1. Homewood Research Institute 1 The Development and Implementation of a Recovery Monitoring System: In the Addiction Medicine Service, Homewood Health Centre 2015 Addictions & Mental Health Conference Toronto, Ontario Monday, May 25 th, 2015 Presented by Jean Costello, PhD Research & Evaluation Scientist Homewood Research Institute

2. 2 The Partnership: HRI & Homewood Health

3. Homewood Research Institute 3 About Homewood Health Homewood Health Centre (HHC) Inpatient Programs: Addiction Medicine Traumatic Stress Recovery Integrated Mood and Anxiety Eating Disorders Others…

4. Homewood Research Institute 4 Why Recovery Monitoring at HHC? Commitment to evidence-based practice and client-centered treatment measuring & monitoring patient-level outcomes Demonstrate accountability to patients, family members, payers/insurers, etc. Few other facilities routinely measure outcomes post-discharge Research value – better understanding of the effectiveness of MHA services

5. Homewood Research Institute 5 Our Vision To create a learning system where one can learn from practice to improve practice, by… enabling routine evaluation and monitoring of patient outcomes post-discharge; facilitating applied or clinical research (within and across organizations); and, aligning with provincial-level efforts to facilitate comparisons across populations and contribute to system change.

6. Homewood Research Institute 6 Our Approach Build upon best practices and align with larger efforts Hazelden (Dr. Valerie Slaymaker) – U.S. Leader Ontario (Dr. Brian Rush) – Canadian & International Leader Participatory / Consultative Administrative-level Program-level Patient-level Other external experts: Drs. Brian Rush; Valerie Slaymaker; James MacKillop; Don Meichenbaum

7. Homewood Research Institute 7 STEP 2: Develop data collection tools and protocols STEP 3: Pre-test tools and protocols STEP 4: Implement recovery monitoring system STEP 5: Evaluate and refine tools and protocols STEP 1: Develop a program logic model Manage routine operations of the system Our Process

8. Homewood Research Institute 8 Manage routine operations of the system STEP 2: Develop data collection tools and protocols STEP 3: Pre-test tools and protocols STEP 4: Implement recovery monitoring system STEP 5: Evaluate and refine tools and protocols STEP 1: Develop a program logic model Our Process: Step 1

9. Homewood Research Institute 9 Step 1: AMS Logic Model

10. Homewood Research Institute 10 Step 1: Program Outcomes Outcome DomainExample Outcome(s) Substance Use BehavioursAbstinence from alcohol and/or drugs Mental Health StatusImproved mental health status Aftercare InvolvementAdherence to aftercare plan; Engagement in AA/NA/CA Life Functioning PsychologicalGreater sense of hope; meaning; self-efficacy Physical healthImproved physical health status Social roles & responsibilityGreater fulfilment of social roles (e.g., parent, spouse) Activities of daily living (ADL)Improved ability to perform ADL Occupational/vocationalImproved performance at work Quality of life/life satisfactionImproved quality of life/satisfaction with life Use of Health & Social ServicesDecreased use of emergency room services

11. Homewood Research Institute 11 Manage routine operations of the system STEP 2: Develop data collection tools and protocols STEP 3: Pre-test tools and protocols STEP 4: Implement recovery monitoring system STEP 5: Evaluate and refine tools and protocols STEP 1: Develop a program logic model Our Process: Step 2

12. Homewood Research Institute 12 Step 2: System & Tool Development Reviewed existing protocols and tools/sub-scales Adopted, refined, and created new protocols/items where appropriate with internal and external stakeholder feedback Created an infrastructure: Staff Software (i.e., data capture, participant tracking) Equipment (i.e., tablets, phones & headsets)

13. Homewood Research Institute 13 Step 2: Data Collection Protocol AdmissionDischarge 1-month post- discharge 3-months post- discharge 6-months post- discharge 12-months post- discharge Self-administered, in-person, group-based via tablets Primary recruitment Re-recruitment opportunity Telephone via trained research assistant OR email via electronic questionnaire Telephone via trained research assistant OR email via electronic questionnaire

14. Homewood Research Institute 14 Manage routine operations of the system STEP 2: Develop data collection tools and protocols STEP 3: Pre-test tools and protocols STEP 4: Implement recovery monitoring system STEP 5: Evaluate and refine tools and protocols STEP 1: Develop a program logic model Our Process: Step 3

15. Homewood Research Institute 15 Pre-tested with former AMS patients Group 1, n=21: self-completed, in-person via tablets Group 2, n=25: completed over phone w/ a trained interviewer All followed up 3-7 days later by phone and asked to complete the questionnaire again (Group 1, n=17; Group 2, n=21) Informed revisions to question wording, instructions, and administration protocols Test-retest reliability analysis Step 3: Pre-testing System & Tools

16. Homewood Research Institute 16 Manage routine operations of the system STEP 2: Develop data collection tools and protocols STEP 3: Pre-test tools and protocols STEP 4: Implement recovery monitoring system STEP 5: Evaluate and refine tools and protocols STEP 1: Develop a program logic model Our Process: Step 4

17. Homewood Research Institute 17 Step 4: Early Implementation AdmissionDischarge 1-month post- discharge 3-months post- discharge 6-months post- discharge 12-months post- discharge Self-administered, in-person, group-based via tablets Primary recruitment Re-recruitment opportunity Telephone via trained research assistant OR email via electronic questionnaire Telephone via trained research assistant OR email via electronic questionnaire WE ARE HERE

18. Homewood Research Institute 18 Reflections on Critical Success Factors and Early Lessons Learned

19. Homewood Research Institute 19 Critical Success Factors Buy-in at the level of ownership & senior administrators Committed resources for developing & testing the system HRI is arms length  objective data collection & reporting Working initially at the program-level (e.g., AMS) Not trying to re-invent the wheel; building on/contributing to… Engaging expert reviewers and advisors Incorporating the lived experience Commitment to translating findings back to decision-makers Plans to embed outcome monitoring within standard practices

20. Homewood Research Institute 20 Create a system that is meaningful to the participant Value in “giving back”; continual engagement Start with a good electronic capture product(s) & embrace the limitations Tablets facilitate data collection & entry, but require facilitation Move to a tailored product when needs are clear Embed data collection within treatment Need to balance expectations & multi-uses of outcome data Early Lessons Learned

21. Homewood Research Institute 21 HRI’s Recovery Monitoring Team Jean Costello, Research & Evaluation Scientist Courtney Ropp, Research Associate Sarah Sousa, Research Associate Kayla Deroux, Research Assistant Katie Junkin, Research Assistant

22. Homewood Research Institute 22 The aim of Homewood Research Institute (HRI) is to improve outcomes of mental health and addictions treatments and services. Initial areas of focus are addictions, PTSD, concurrent treatment and return to work. HRI is initiating networks to enable researchers, evaluators and practice leaders to collaborate with patients, clients, worksites, clinicians, and donors in work that advances both science and practice. Homewood Health is HRI’s primary practice partner. www.homewoodresearch.org