1. Participate in the Research & Evaluation track Complete two sessions in the track, and the CFHA research committee would like to recognize your efforts with a certificate. Please fill in your email on the guidebook app and/or attendance sheet to receive the certificate.

2. We need your feedback! As many of you know this is a new component of CFHA conference and we need your feedback to allow us to improve our program and decide whether to continue to offer it in the future! We have developed our own survey, here are the instructions to complete it: – Enter guidebook app – Click on this particular session (e.g., Unlocking Implementation in Primary Healthcare) – Scroll down past abstract and objectives – Click on Research Committee Survey – Complete the quick 5 questions Note: If you don’t have the app or would like to complete it in paper, ask one of the co-chairs: Drs. Polaha or Funderburk

3. Let’s Talk! Questions and Answers Regarding How to Convert Clinical Practice Into Research and Program Evaluation Jeffrey L. Goodie, Ph.D, ABPP Jennifer Funderburk, Ph.D. Christina Studts, Ph.D., LCSW R. William Lusenhop, MSW, Ph.D., LICSW Collaborative Family Healthcare Association 17 th Annual Conference October 15-17, 2015 Portland, Oregon U.S.A. Session # B4 October 17, 2015

4. Faculty Disclosure The presenters of this session have NOT had any relevant financial relationships during the past 12 months.

5. Learning Objectives At the conclusion of this session, the participant will be able to: Identify barriers to conducting research and program evaluation in primary care clinics Identify effective strategies to implement research or program evaluation protocols in primary care settings Describe plans to prepare for or initiate research or program evaluation methods in their own practice settings

6. Bibliography / Reference Funderburk, J.S., Spinola, S., & Maisto, S.A. (in press). Mental health predictors of Veterans willingness to consider research participation. Military Medicine. Pigeon, W. & Funderburk, J.S. (2014). Delivering a brief insomnia intervention to depressed VA primary care patients. Cognitive and Behavioral Practice, 21(3), 252260. doi: 10.1016/j.cbpra.2013.10.007 Goodie, J. L., Kanzler, K. E., Hunter, C. L., Glotfelter, M. A., & Bodart, J. (2013). Ethical and effectiveness considerations with primary care behavioral health research in the medical home. Families, Systems, and Health, 31, 86-95. doi:10.1037/a0031852 Cigrang, J. A., Rauch, S. A. M., Avila, L. L., Bryan, C. J., Goodie, J. L., Hryshko­Mullen, A. Peterson, A. L., and the STRONG STAR Consortium. (2011). Treatment of active­ duty military with PTSD in primary care: Early findings. Psychological Services 8(2), 104­113. doi:10.1037/a0022740 Mendenhall, T. J., & Ballard, J. E. (2014). Including the family in research evaluating integrated care: A call for expanding investigators’ scope beyond single person measures. Families, Systems, & Health, 32, 291–302. doi:10.1037/fsh0000018

7. Learning Assessment A learning assessment is required for CE credit. A question and answer period will be conducted at the end of this presentation.

8. Plan Overview of common barriers and effective strategies to conducting research and program evaluation in primary care settings (20 mins) Four groups (30 mins) – Here’s my research/evaluation question; I need help with the method! – Here’s my research/evaluation question & method; I have hit some obstacles! – Here’s my research/evaluation question & method; how can I measure outcomes? – I have no idea where to start/I just need help! Groups are rearranged for new question (25 mins) Large group discussion, summary, and resources

9. Introductions

10. Barriers to research in primary care Nearly 20 years ago, Stange (1996) described many of the same challenges to conducting research in primary care as we see today: – Difficulties in translating practice-based wisdom into methodologically sound research – Lack of a critical mass of researchers – Poorly developed research culture – Competing demands faced by investigators

11. Barriers to research in primary care Your experiences?

12. Example #1 Problem: – Need Data Because At Risk of Losing Current Financial Investment in BHP – Desire to know penetration and reach: how many students involved in services with no experience with mental health – EMR is not advanced and no one knows at the clinic how to effectively pull data

13. Solution Excel Patient #Patient Name Date of Visit Scheduled appt (1) or Walk-in (2) Medical Provider Inter’l student Hx of tx

14. Example #2 Problem: – Asked by leadership to conduct program evaluation of a new telephone care management system, but no idea how to go about doing it – This system collected data within EMR because used structured clinical interview to assess patients prior to linking those patients with services or care management

15. Solution RE-AIM framework helped to provide them with the appropriate questions to ask Decided to conduct initial assessment when patient completed care management as a way to obtain effectiveness data Obtained instruction on how to easily pull the data and analyze it.

16. Small group break-outs 1)Here’s my research/evaluation question; I need help with the method! 2)Here’s my research/evaluation question & method; I have hit some obstacles! 3)Here’s my research/evaluation question & method; how can I measure outcomes? 4)I have no idea where to start/I just need help! **Groups will shift in 30 minutes

17. Summary and themes from small groups

18. Strategies that work Get your foot in the door Understand practice readiness-to-engage – Leadership – Resources – Culture Obtain practice buy-in Work with early adopters/opinion leaders AHRQ (2015), Engaging Primary Care Practices in Quality Improvement: Strategies for Practice Facilitators

19. Strategies that work Approach conversations as “respectful negotiations.” Be transparent Tailor your messages – Office staff – Managers – Clinicians AHRQ (2015), Engaging Primary Care Practices in Quality Improvement: Strategies for Practice Facilitators

20. Strategies that work Identify “pain points” and offer solutions Use data feedback Start with an “early win” Build capacity incrementally Select feasible methods and research/evaluation designs – Randomization? – Interrupted time series? AHRQ (2015), Engaging Primary Care Practices in Quality Improvement: Strategies for Practice Facilitators

21. AHRQ (2014), A Guide to Real-World Evaluations of Primary Care Interventions: Some Practical Advice Resource TypeExamples Leadership buy-inMotivation and support for trying the intervention Financial resourcesFunding available for the intervention Technical assistanceData extraction and feedback, practice facilitation/ coaching, expert consultation, learning collaboratives, information technology (IT) expertise, etc. ToolsIntervention materials, registries, health IT, shared decision-making tools, etc. TimeAllocated time of staff to implement the intervention; sufficient elapsed time for outcomes to change Table 1. Inventory of Resources Needed for Testing a Primary Care Intervention: RESOURCES FOR INTERVENTION:

22. Resource TypeExamples Leadership buy-inMotivation and support for evaluating the intervention Financial resourcesFunding available for the evaluation Research skills/commitmentSkills in design, analysis, interpretation. Motivation and buy-in of evaluation staff and other relevant stakeholders, such as clinicians, staff, and patients who will provide data. DataE.g., claims, electronic medical records, paper charts, patient intake forms, care plans, patient surveys, clinician and practice staff surveys, registries, care management tracking data, qualitative data from site visit observations and interviews, costs, and other information TimeTime to obtain and analyze data and for outcomes to change Table 1. Inventory of Resources Needed for Testing a Primary Care Intervention: RESOURCES FOR EVALUATION: AHRQ (2014), A Guide to Real-World Evaluations of Primary Care Interventions: Some Practical Advice

23. Resources Engaging Primary Care Practices in Quality Improvement: Strategies for Practice Facilitators (AHRQ, 2015: https://pcmh.ahrq.gov/sites/default/files/atta chments/QI-strategies-practices.pdf) https://pcmh.ahrq.gov/sites/default/files/atta chments/QI-strategies-practices.pdf A Guide to Real-World Evaluations of Primary Care Interventions (AHRQ, 2014: https://www.pcmh.ahrq.gov/sites/default/file s/attachments/PCMH_Evaluation_Guide.pdf) https://www.pcmh.ahrq.gov/sites/default/file s/attachments/PCMH_Evaluation_Guide.pdf

24. Resources Connect with (and join!) the CFHA Research Committee: http://www.cfha.net/?page=ResearchCommittee http://www.cfha.net/?page=ResearchCommittee Resource pages from past and present CFHA Conferences (including presentation slides): http://www.cfha.net/?page=Conference http://www.cfha.net/?page=Conference Apply to the CFHA Early Career Mentorship Program: http://www.cfha.net/?page=EarlyCareer http://www.cfha.net/?page=EarlyCareer

25. Participate in the Research & Evaluation track Complete two sessions in the track, and the CFHA research committee would like to recognize your efforts with a certificate. Please fill in your email on the guidebook app and/or attendance sheet to receive the certificate.

26. We need your feedback! As many of you know this is a new component of CFHA conference and we need your feedback to allow us to improve our program and decide whether to continue to offer it in the future! We have developed our own survey, here are the instructions to complete it: – Enter guidebook app – Click on this particular session (e.g., Unlocking Implementation in Primary Healthcare) – Scroll down past abstract and objectives – Click on Research Committee Survey – Complete the quick 5 questions Note: If you don’t have the app or would like to complete it in paper, ask one of the co-chairs: Drs. Polaha or Funderburk

27. Session Evaluation Please complete and return the evaluation form to the classroom monitor before leaving this session. Thank you!