1. Toolkits Deni Carise, Ph.D. Tom McLellan, Ph.D. Adam Brooks, Ph.D. Robert Forman, Ph.D Developing a Clinician Resource for Evidence-Based Treatment Delivery Supported by NIDA Grant (R21DA-015977)

2. The Research-Practice Gap 25 years of heavy research investment in: –Medication Development –Behavioral Therapies Numerous treatments have: –Demonstrated efficacy in clinical trials –Demonstrated effectiveness in dissemination studies

3. TRI science addiction The Research-Practice Gap Research shows very low rates of adoption of these evidence-based practices Possible factors leading to non-adoption: –Financial –Complexity of interventions –Provider organizational and policy concerns

4. TRI science addiction The Research-Practice Gap We interviewed counselors and program directors in the Delaware Valley Research Practice Collaborative (PA, DE, NJ) Initial Focus Groups revealed two barriers as greatest contributor to the “gap”: –Finance –Training/Supervision

5. New therapies won’t be adopted if: – the treatments cannot be supported by managers and funding agencies – the training and supervision burdens of the treatments are overwhelming The Research-Practice Gap

6. The Financial Barrier It is a challenge to implement evidence- based treatment practices within the severely challenged infrastructure * Addiction treatment system has experienced a 20-year period of declining funds: ** * McLellan, Carise and Kleber, 2003. **Mark, Levit, et. al. 2007.

7. TRI science addiction The Financial Barrier General healthcare funding declined 12% between 1988 and 1998* During the same period, addiction treatment funding declined 75%* * Galanter, Keller et. al. 2000.

8. Financial Barrier Consequences Program closures or re-organizations Increased counselor and management turn-over* In a survey of 450 treatment programs: 32% faced the threat of closure 5% actually closed during the year of the survey** *Gallon, Gabriel et. a. 2003; McLellan, Carise et al. 2003. Knudsen, Ducharme et. al. 2004. **Roman, Blum 1997.

9. Financial Barrier Outcomes Increased Reliance on Group Therapy Reimbursement for individual therapy is under-funded or not funded Offered as primary treatment modality

10. TRI science addiction Group Psychotherapy Can be as effective as individual therapy* Are a top priority for bridging the gap Evidence-based approaches have often not been adapted for group treatment * Weiss, Jaffee et al. 2004.

11. Supervision Barrier Research shows that clinical supervisors: spend less time mentoring and training spend more time addressing human resource or regulatory compliance issues often have ongoing case loads of their own

12. Training Barrier Of 400 national substance abuse treatment centers, 20% had no staff training budget * Decreased funding+ Increased licensing, accrediting, and funding compliance requirements Decreased clinical training and Increased attention to administrative issues  *Johnson 2000.

13. Training Demands of EBPs Research on Training EBPs shows: – Manuals (if read) are useful for learning interventions, but not enough to change counselor practice* – To be proficient in conducting new interventions, training needs to be ongoing: Requires more time than a standard 2-3 day workshop Requires ongoing, expert supervision *McCarty, Fuller et al. 2007

14. The Big Problem We have complex treatments –Content heavy –Require intensive training –Proven efficacy when delivered correctly We have a resource starved environment We have heavy turn-over in the field

15. TRI science addiction Looking to Other Fields The field of education has experienced similar problems –Minimal resources –Stressed workforce –Complicated interventions (lessons) with little time to prepare

16. Possible Solution Education field Toolkits - An evidence-based approach to addressing the challenge of upgrading skills in a resource restricted environment

17. Possible Solution Teachers use packaged lesson plans to assist teachers in conveying complex concepts They can be taught effectively, require little supervision and have been constructed, tested and refined through a scientific process

18. Possible Solution: Examples

21. Existing Treatment Curricula Currently, there are some existing prepackaged curricula for substance treatment Vary to the degree that they are informed by evidence based content Few, if any, have been rigorously tested to assess their impact on treatment or outcomes

22. TRI science addiction Testing a Curriculum Sample We conducted significant background work to determine provider interest in a curriculum-based approach We developed and pilot-tested a single- session curriculum “Toolkit”

23. Surveying Interest 21-item survey designed for the Clinical Trials Network projects called: “What Do You Need?” Administered in 18 community-based treatment programs in the Delaware Valley Practice Research Collaborative (PA, NJ, DE)

24. Surveying Interest 269 treatment program staff completed the survey 195 employees identified themselves as “treatment providers” –Other options: researcher, managed care or faith-based organizations, support staff, or consumers were not included in the analyses (n=74)

25. Surveying Interest Self-report survey included: Demographic questions: Age, gender, ethnicity, education, certification, years in the field Forced choice of preferred Training Topics (n=26)

26. Sample Demographics Overall demographics: Age (Mean=42; SD=11) Years of Experience (Mean=9; SD=7) Gender: Female = 57% Ethnicity: Caucasian = 71%, African American = 22%, Hispanic/Latino = 3%, Other = 4%

27. Analyses 1 Which Topics were most frequently endorsed? Was endorsement related to counselor background? –Certification was defined as having any of the following certifications: Certified Drug/Alcohol Counselor (PCACB, PCB, etc) ASAM Certification or APA Qualified in Addiction Medicine/Psychiatry Licensed Health Professional Ordained Clergy –Experience was defined as having been in the field for at least 6 years (Range: 0-35 yrs)

28. Analyses 2 Years of Experience and Certification were examined, creating 4 groups: Note: Participants without both Experience and Certification data were omitted from analysis (n=25; 14%) UncertifiedCertified Inexperienced(n=34, 17%)(n=51, 26%) Experienced(n=61, 31%)(n=24, 12%)

29. Results 1 Percent Endorsed: Training Topic 62 -Increasing Client Motivation 17 -Obtaining Project Funding 49 -Relapse Prevention Techniques 17 -Finding Treatment Information 47 -Co-Occurring Disorders 15 -Harm Reduction Strategies 47 -Group Therapy Techniques 15 -Applying Research in Practice 45 -Spirituality & Recovery 14 -Age Specific Treatment 27 -12-Step Oriented Approaches 14 -Sexuality Related Treatment 25 -Ethnicity & Diversity 12 -Documentation Software 25 -Biological Basis of Addiction11 -Tobacco Addiction 21 -Addictions Medications 11 -Use of Incentives & Rewards 19 -Sexual Addiction 11 -Internet Addiction Resources 18 -Gender Specific Treatment9 -Research & Evaluation Methods 18 -Alternative Therapies5 -Using Research Databases 4 -Statistics & Research Findings Most frequently endorsed Research-related Topics analyzed

30. Results 2 Training Topic Preferences by Experience and Certification: Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p <.05 Note: Participants can choose more than one category Binary Logistic Regression % Inexperienced & Uncertified (n=34) % Inexperienced & Certified (n=51) % Experienced & Uncertified (n=61) % Experienced & Certified (n=24) Increasing Client Motivation 68655758 Relapse Prevention 56593650 Co-Occurring Disorders 1 44315658 Group Therapy Techniques 3 59493358

31. Results 3 Training Topic Preferences by Experience and Certification: Note: 1 = Main Effect (ME) for Experience, 2 = ME for Certification, 3 = Interaction, at p <.05 Note: Participants can choose more than one category Binary Logistic Regression % Inexperienced & Uncertified (n=34) % Inexperienced & Certified (n=51) % Experienced & Uncertified (n=61) % Experienced & Certified (n=24) Spirituality & Recovery 32494850 12-Step Oriented Approaches 1 27392017 Addiction Medications 1,2 3518234 Applying Research in Clinical Practice 21101325

32. Results 4 Out of the 26 possible Training Topics, the majority of counselors chose the same 5, making up 45% of the responses These “top 5” topics were most related to the treatment counselors were already providing

33. Survey: “Staff Beliefs about Addiction Treatment and Clinical Trials” NIDA Clinical Trials Network (CTN) – DE Valley 22-items assessing 317 counselors’: Beliefs about addiction treatment Willingness to try new treatment approaches – Endorsed by 80% of surveyed counselors – Willingness to try research-based innovations – Endorsed by 82% of surveyed counselors

34. 129 counselors were asked about their clinic’s videotape usage for group sessions: 100% used videotapes with patients 97% agreed videotapes were “a useful aid in educating patients about addiction and recovery” Survey: Videotape Use in Addiction Treatment Settings

35. 96% wanted “a library of up-to-date, scientifically accurate videotapes...” 63% agreed only showing “brief sections of videotapes with a discussion following them” was best 58% were dissatisfied with their clinic’s current education tools Survey: Videotape Use in Addiction Treatment Settings

36. TRI science addiction Toolkit Development We incorporated provider feedback into the development of a single “Toolkit” Goal was to: – assess provider satisfaction –assess client satisfaction –assess continued provider use of the Toolkit

37. Step 1 – Choosing Content Examples: Relapse prevention HIV risk reduction Biology of Addiction Medications Motivational Interviewing Toolkit Prototype – MI Component - Decisional Balance Developing the Toolkit

38. Step 2: Designing the Toolkit Multiple formats (Digital & Hand-Outs) Created Toolkit prototype Developing the Toolkit “Toolbox” Clinician Guide DVD Worksheet Wallet Cards Components:

39. TRI science addiction Toolkit Components DVDDVD Wallet Card WorksheetClinician’s Guide

40. TRI science addiction Toolkit Video Good Things and Not So Good Things about Recovery

41. Counselors - 26 Counselors from 6 clinics - Experienced Counselors (> 10 years) Procedures - Counselors consented & oriented - Use the Toolkit in Group & Complete Clinician survey - One month later - Asked to use 2nd time - Next 3 months: Record use of Toolkit components Methods - Counselors

42. Group Members/Patients N=230 Procedures - Consented prior to group - Attended Toolkit group - Completed Patient Satisfaction Survey Methods - Patients

43. Will clinicians be able to use the Toolkit after only a brief orientation? Will clinicians continue to use it, even when not required? How will patients evaluate the toolkit group? Research Questions

44. N=26 (counselors) Will Clinicians Be Able To Use It After Only a Brief Orientation? Mean(SD) Was the Toolkit orientation adequate?4.50.8 Were you satisfied with the Toolkit group?4.20.7 Was the Toolkit "User-Friendly?"4.40.7 Do you think other Clinicians would like to use the Toolkit?4.40.7 Scale: 1=Not at all, 2=Somewhat, 3=Moderately, 4=Quite a Bit, 5=Very Much

45. TRI science addiction Counselor Ratings

46. TRI science addiction Patient Satisfaction

47. 96% of counselors have reused at least one component of the Toolkit On at least one occasion, 63% of clinicians used all 5 components of the Toolkit as directed The most frequently and broadly used were the core components: Counselor Guide & Patient Worksheet Will Clinicians Continue To Use It After the Study Ends?

48. Over a 3-month tracking period: Counselor Guide: 83% used it again (M = 2.5 times) Patient Worksheet: 96% used it again (M = 3.2 times) Will Clinicians Continue To Use It After the Study Ends?

49. Toolkit component Clip- board DVD Work- sheet Guide Wallet Cards Mean (sd)5 (9)1 (1)3 (3) 2 (2) Range1 - 421 - 31 - 12 1 - 9 Times Used Toolkit Components

50. Preliminary results in community-based settings support the Toolkit’s: feasibility acceptability sustainability Conclusions

51. TRI science addiction Future Directions Will curriculum impact clinician behavior in treatment? Will curriculum beneficially impact client outcomes?

52. TRI science addiction Translating a Complete Treatment We will translate a full treatment intervention into a Toolkit format Starting with 6 Toolkits on Relapse Prevention Techniques –Content and didactic heavy –Popularly selected by providers –Evidence Based –Useful in group treatment

53. TRI science addiction Testing a Complete Curriculum In a larger sample, we will randomly assign sites to receive: 6 training manuals OR 6 training manuals + 6 Toolkits Curriculum Supplements

54. TRI science addiction Future Directions, Cont’d Toolkits for other evidence-based treatments (12-step, etc) Toolkits to address comorbidity, HIV prevention, etc Arming clinicians with enough evidence- based content to fill 50-75% of group sessions

55. TRI science addiction Future Directions Your Thoughts?