Evaluation of Web-Based Recovery Monitoring With Clinical Alerts

Phase 1:Our study team completed Phase I of the project in June, 2011. Our initial plans called for our team to analyze RT data that was collected by Delaware treatment programs to determine specific predictors for early attrition from treatment. Eleven counselors were trained to use RT as part of their standard practice in May, 2011. In an effort to generate a predictive algorithm for Phase II, we analyzed data from three datasets available from IRB approved research projects. However, the results were inconclusive, possibly due to small sample sizes and differences in the recruitment and definition of drop out across samples. In order to proceed with Phase II, we created an algorithm based largely on severity of substance use, a consistent predictor of drop-out across the alcohol and illicit substance literature; the algorithm also addresses when clients are not using but their Risk factors greatly outweigh the influence of Protective factors. We determined that the algorithm did work to identify clients who self-reported using substances or high levels of risk. However, the identification rate of high-risk cases was too low, and so we modified the algorithm to identify: 1) clients with high use at baseline, 2) clients with positive urine samples, 3) clients with any use at subsequent [post-baseline] interviews, and 4) clients whose risk factors outweighed their protective factors. This increased our rate of identification of risk cases by about 32%.

As part of Phase I, we also completed the programming changes to RT to add a Clinical Alert feature that informs counselors when a client is at high risk based on the algorithm. The CBI intervention and training materials for Phase II was created. We developed the CBI into a brief clinical "toolkit" that enabled counselors to respond to clinical risk based on their judgment. This Toolkit allows counselors to exercise clinical judgment to determine whether clients are generating a Clinical Alert because 1) they are using drugs or alcohol, or are at risk to start using, 2) they have unmet psychosocial needs [i.e., need for psychiatric consultation], or 3) they have a poor alliance with the counselor / treatment provider and need some help building on the relationship. The Clinical Alert Toolkit included a series of exercises / interventions that counselors could deploy once they had determined which of the client's needs were most pressing (either based on their own judgment or in agreement with the client).

Phase 2:Three counselors consented to participate in this study and were trained to use RT in May, 2011 and were trained on the CBI intervention in June, 2011. Recruitment of client participants began in July, 2011. The training and intervention materials were well received by the participating counselors. One counselor ended their employment prior to the research staff beginning client recruitment. 30 clients out of 35 who were approached participated in the feasibility trial; 28 clients completed baseline (93%), 23 completed the one month follow up (77%), 21 completed the two month follow up (70%), and 21 completed the three month follow up (70%). Out of these clients, two clients were incarcerated for their 1 month follow up window, 2 clients were incarcerated for their 2 month follow up window, and three clients were incarcerated for their three month follow up window therefore they were not approached to complete the follow-up interview. We used our experiences during Phase II to make several changes in our training and clinical protocol for counselors to follow. We revised several of our Toolkit components (specifically the worksheets to make them more user-friendly and to reduce training burden). We also simplified our feedback strategy which took place during the clinical supervision sessions to focus much less on general therapeutic skills and more specifically on actual compliance with the trained Clinical Alert CBI. Additionally, we found that while our Phase II training counselors verbalized motivation to try to learn to use the CBI techniques, that the learning curve was slower than we desired, and we decided to employ a contingency management feedback strategy to increase their incentive to acquire the basic elements of the CBI more quickly. This revision required IRB approval; in the revised approach, counselors could earn bonuses when a rated audio recorded session showed that they had been at least minimally adherent in delivering the CBI. Finally, during Phase II we completed the Attention Control training (focused on treatment planning) for use in the TAU condition.

Phase 3: Because we needed to move the study from Delaware Site 1 (our original clinical site and partner), our team recruited Alternate Site 1 in Philadelphia and Alternate Site 2 in central New Jersey to conduct the Phase III clinical pilot trial. We enrolled 20 counselors who were trained to use RT on a monthly basis with their clients, starting at the first scheduled individual session and monthly thereafter. We randomly assigned these counselors to receive either TAU treatment planning training or to receive Clinical Alert (CA) training. After training, recruitment began. During the trial, three counselors in the CA condition either were noncompliant with the study protocol (N = 2), or never had any clients report any high risk (N = 1). We worked with all three of these counselors to encourage improved engagement with the intervention and study procedures, but this was not successful. Consequently, we decided that we would over-recruit additional counselors into the CA condition, to create additional opportunities to determine whether the Clinical Alerts + Intervention training would impact outcomes. We recruited an additional three counselors, but did not randomize them; rather, we assigned them directly to the CA condition.

We enrolled 336 clients in Phase 3. 142 self-reported clinical RT data which would result in a Clinical Alert profile (TAU: N = 78, CA: N = 64). Because of a low recruitment rate, the study was severely underpowered for our subanalyses to examine specific effects of the intervention training on dealing with High Risk cases; this is where we believed our strongest effects would present themselves.