1. Implementing evidence-based psychosocial therapies for substance use: Aspects of measuring adherence and competence
Andrea Meier, Mark P. McGovern, Bethany Lasko, and Chantal Lambert-Harris
Geisel School of Medicine at Dartmouth
ABSTRACT
METHODS
RESULTS
DISCUSSION
Background: Treatment quality is a major concern in the implementation of evidence-based treatments for substance use disorders. There is wide variation in methods for assessing quality.
Objective: The present study examines three aspects of clinician adherence and competence. We test the following hypotheses:
Methods: Clinicians from community addiction treatment programs delivered two evidence-based psychosocial therapies in a RCT. Audio-recordings of sessions were rated using adherence/competence scales. Working alliance, number of sessions attended, and patient outcomes at 3- and 6-months were assessed.
Results: Positive relationships were found between adherence and competence levels and clinician characteristics (gender, educational and attitudes), and patient outcomes (PTSD, alcohol and drug severity).
Conclusions: Community clinicians learn to deliver EBTs with quality. Effective implementation of EBTs should consider clinician demographics and attitudes in clinician selection, training and supervision methods.
Consistent with previous research, IAC was delivered with greater adherence than ICBT (p<.05), however both EBTs were delivered above the adequate adherence level.
Significant relationships were found between adherence and competence and clinician gender, educational level and EBPAS scores. Higher ratings were found for:
Therapy Types
Integrated Cognitive Behavioral Therapy (ICBT)
An (8-12 session) manual-guided therapy consisting of 3 learning and skill components designed to reduce PTSD symptoms and substance use: 1) Patient education about PTSD, substance use and their interaction; 2) Anxiety reduction (centering and breathing retraining techniques); and 3) Cognitive restructuring: A cognitive behavioral approach evaluating cognitions, emotions and behaviors.
Individual Addiction Counseling (IAC)
An (8-12 session) manual-guided therapy targeting substance use that is stage-based (Treatment initiation, Early abstinence, Maintaining abstinence, and Recovery). IAC is an adaptation of the Individual Drug Counseling (IDC) treatment used in the NIDA Cocaine Collaborative Study and the Twelve Step Facilitation (TSF) therapy used in the NIAAA Project MATCH study.
1. Women
2. Bachelor’s and Master’s level
3. Positive Appeal and Openness attitudes, and
Negative Divergence attitudes
Clinicians’ adherence and competence varies by therapy type
Adherence and competence are predicted by clinician factors
Adherence and competence ratings are associated with patient outcomes
22 clinicians from seven community addiction treatment programs trained in both ICBT and IAC therapies.
Clinicians completed background characteristics and attitude measures at baseline training.
Clinicians received biweekly researcher-led supervision (feedback, coaching, and session audio review).
Independent raters evaluated 25% of audio-recorded sessions.
Adherence/competence measures for ICBT and IAC have acceptable psychometric properties.
121 patients meeting both PTSD and substance use disorder criteria randomized to IAC or ICBT.
Patients assessed at baseline, 3-month and 6-month.
***p<.001
*p<.05
Clinician adherence and competence ratings did not show a relationship with therapy process (Working Alliance).
Adherence and competence had a significant relationship with patient outcomes. Higher ratings showed greater reductions in:
EBPAS Rating Scale 0 to 4:
0=Not at all
2=Moderate degree
4=To a very great extent
PTSD symptom severity in both ICBT and IAC
Alcohol severity in ICBT
Drug severity in IAC
These findings were surprising given the focus of ICBT (PTSD and substance use) and IAC (substance use only).
CONCLUSIONS
Measures
Clinician Characteristics and Attitudes
Adherence and Competence
Therapy Process
Therapy Outcomes
Clinician Background Survey
Evidence Based Practices Attitudes Scale (EBPAS)
ICBT Adherence and Competence Rating Scale
IAC Adherence and Competence Rating Scale
Working Alliance Inventory (WAI-S)
Clinician Administered PTSD Scale (CAPS)
Addiction Severity Index (ASI)
Timeline Follow-Back (TLFB)
Although differences by therapy type are identified, clinicians deliver both therapies with adherence and competence.
Several clinician factors are associated with adherence and competence ratings: Gender, educational level and attitudes.
Adherence and competence ratings are associated with patient outcomes: PTSD, alcohol and drug symptom severity.
These results have clinician selection, training, supervision, and quality monitoring implications.
Future research is warranted to further examine quality monitoring and EBT implementation in community settings.
INTRODUCTION
Quality Monitoring: Evidence-based treatment quality is routinely monitored by rating clinicians’ adherence and/or competence in delivering the intervention.
*p <.05 **p<.01 ***p<.001
ADHERENCE – Fidelity to model techniques
COMPETENCE – Skillfulness in delivering techniques
RESULTS
Challenges in EBT implementation: Although widely accepted as the standard, there is mixed evidence on clinician factors, as well as EBT training and supervision models, that contribute to adherence/competence levels. Current quality monitoring, training and supervision models tend to be research driven, timely and cost prohibitive.
Evaluating the relationship of therapy process and outcomes with adherence and competence: The relationship to the therapeutic process or patient outcomes has also been minimally addressed.
Current study: Further examination of these relationships could influence practical training, supervision, monitoring and implementation methods, as well as adherence and competence measure development.
REFERENCES CITED
Manuel, JK, Hagedorn, HJ, & Finney, JW. Implementing evidence-based psychosocial treatment in specialty substance use disorder care. Psychology of Addictive Behaviors.2011;25(2):
McGovern, MP, Lambert-Harris, C, Alterman, AI, Xie, H, & Meier, A. A randomized controlled trial comparing integrated cognitive behavioral therapy versus individual addiction counseling for co-occurring substance use and posttraumatic stress disorder. Journal of Dual Diagnosis. 2011;7(4):
Miller, WR, Sorensen, JL, Selzer, JA, Brigham, GS. Disseminating evidence-based practices in substance abuse treatment: A review with suggestions. Journal of Substance Abuse Treatment. 2006;31:25-39.
Morgenstern, J, Morgan, TJ, McCrady, BS, Keller, DS, & Carroll, KM. Manual-guided cognitive-behavioral therapy training: A promising method for disseminating empirically supported substance abuse treatments in the practice community. Psychology of Addictive Behaviors. 2001;15(2):83-88.
Item
% High Fidelity
Item Correlation with Employment
1. Caseload size 76
0.02
2. Exclusively voc services 78
-0.12
3. Vocational generalists 81
0.39
4. Integration with treatment team 63
0.04
5. Contact with treatment team 30
0.03
6. State vocational rehab agency 54
0.12
7. Vocational unit 56
0.27
8. Supervisory role 27
0.16
9. Zero exclusion 37
0.01
10. Agency focus on work 32
11. Agency leadership support 34
-0.05
12. Benefits counseling 49
0.17
13. Disclosure
0.38
14. Individualized assessment 25
15. Rapid search
0.19
16. Individualized job search 35
17. Job development -- frequency 33
0.09
18. Job development -- quality 46
0.21
19. Occupational diversity
0.23
20. Employer diversity 52
0.24
21. Competitive jobs
0.2
22. Individualized supports
0.36
23. Time-unlimited supports
0.28
24. Community-based services
0.11
25. Engagement
Mean SD
Correlations
Local Unemployment
Fidelity
9.8%
2.6
IPS Fidelity
101 13
-0.01
Employment Rate
37%
-0.15
.34**
Measures
Clinician Characteristics and Attitudes
Adherence and Competence
Process
Therapy Outcomes
Clinician Background Survey
ICBT Adherence and Competence Rating Scale
Working Alliance Inventory (WAI-S)
Clinician Administered PTSD Scale (CAPS)
Evidence Based Practices Attitudes Scale (EBPAS)
IAC Adherence & Competence Rating Scale
Addiction Severity Index (ASI)
Timeline Follow-Back (TLFB)
CONFLICT OF INTEREST
The authors declare that they have no conflicts of interest.
GRANT SUPPORT
This research was supported by NIDA R01 DA (McGovern, PI).