1. Toward a Learning Healthcare Delivery System: Leveraging Implementation, Improvement and Delivery System Science to Improve Performance November 11, 2015 Brian S. Mittman, PhD Dept of Research and Evaluation, Kaiser Permanente Southern California US Dept of Veterans Affairs Quality Enhancement Research Initiative (QUERI) UCLA CTSI Implementation and Improvement Science Initiative Kaiser Permanente RESEARCH

2. Defining a Learning Healthcare System 1.Outcomes and performance goals 2.Features and performance processes

3. Institute of Medicine’s Crossing the Quality Chasm (2001)  Safe: avoiding injuries  Effective: services based on scientific knowledge; avoiding underuse and overuse  Patient-centered: responsive to individual patient preferences, needs, and values  Timely: reducing waits and delays  Efficient: avoiding waste [value, affordability]  Equitable: across gender, ethnicity, geography, SES

4. Commonwealth Fund’s Framework for a High Performance Health System (2006)  Quality and Safety the right health care, avoiding underuse, overuse and misuse safe, reliable coordinated patient-centered: timely, excellent service, active and informed patients  Access to Care universal participation financial protection, established benefits, affordable equitable

5. Commonwealth Fund’s Framework for a High Performance Health System (2006)  Efficient, High Value Care efficient right time, right setting ongoing evaluation of new technologies; defined processes for introduction, surveillance, reevaluation  System Capacity to Improve investment in innovation and research information infrastructure effective educational system rapid response to threats and disasters culture of improvement balance between autonomy and accountability

6. Summary of desired features 1.Safe and reliable 2.High-quality, effective, evidence-based 3.Patient-centered, excellent service 4.Timely, accessible 5.Efficient, cost-effective, high-value 6.Equitable 7.System is technologically advanced, research- and improvement-oriented, balancing autonomy and accountability

7. Learning Healthcare Delivery System features  Explicit performance, improvement, learning goals  Comprehensive performance monitoring against goals  Explicit care management plans, policies, practices  Active environmental scanning  Explicit policies and processes for locating, vetting, evaluating, refining, scaling/spreading external innovations  “ “ “ for internal innovation, experimentation  Supportive leadership, culture, training, resources (staff, expertise, space, equipment, funds, etc.), rewards, etc.

8. Achieving learning and improvement: Role of improvement, implementation, delivery sciences? 1.Reliable evidence-based strategies (interventions) for delivering, improving care (FDA-approved, formulary-listed) 2.Strategies for working to improve care (e.g., PDSA/rapid- cycle improvement); analytical approaches and tools for monitoring and guiding improvement 3.Insights into barriers to change, requirements or conditions for improvement (environment, organization, team, ind’l) 4.Insights into the behavior of delivery systems and organizations, teams, clinicians and staff

9. What is implementation science? 1.Clinical research produces new evidence, innovation 2.Initial efforts to promote implementation 3.Measurement of rates of implementation – and implementation (quality) gaps 4.Research to develop and evaluate implementation programs* to increase adoption *quality improvement programs, practice change programs (interventions)

10. Health benefits of research Improved Health Processes, Outcomes Basic Science Clinical Research ? ?

11. The Clinical Research Crisis  AAMC Clinical Research Summit: Clinical Research: A National Call to Action (Nov 1999)  IoM Clinical Research Roundtable (2000-2004)  UK Cooksey Report (2006), other US and non-US reports

12. Translational research Improved Health Processes, Outcomes Basic Science Clinical Research Type 1 Translation Type 2 Translation

13. Translational research Basic Science Pre-Clinical/ Translational Research Type 2 Translation Implementation Research Clinical Research Type 1 Translation Improved Health Processes, Outcomes

14. Implementation research Basic Science Pre-Clinical/ Translational Research Implementation Research Clinical Research Improved Health Processes, Outcomes

15. Implementation science definition Implementation research is the scientific study of methods to promote the systematic uptake of research findings and other evidence-based practices into routine practice, and, hence, to improve the quality and effectiveness of health services. It includes the study of influences on healthcare professional and organizational behavior. Eccles and Mittman, 2006

16. Implementation science aims 1.Develop reliable strategies for improving health- related processes and outcomes; facilitate widespread adoption of these strategies 2.Produce insights and generalizable knowledge regarding implementation processes, barriers, facilitators, strategies 3.Develop, test and refine implementation theories and hypotheses; methods and measures

17. Clinical research vs. implementation research Study type Study feature Clinical research Implementation research Aim: evaluate a / an … clinical intervention implementation strategy Typical intervention drug, procedure, therapy clinician, organizational practice change Typical outcomes symptoms, health outcomes, patient behavior adoption, adherence, fidelity Typical unit of analysis, randomization patient clinician, team, facility

18. US, international resources  NIH Conference on the Science of Dissemination and Implementation (2007 — 2015 )  NIH grant funding, review committee, training programs  Journals: Implementation Science, Translational Behavioral Medicine, special issues of general and specialty journals, new SIRC journal in development  NIH CTSAs (selected), PBRNs, ACTION, VA QUERI  Patient-Centered Outcomes Research Institute (PCORI), AAMC Research on Care Community (ROCC)  Knowledge Translation Canada, Kings College London Centre for Implementation Science, etc.

19. Local resources  Community Health, Health Behavior  Health Services, Management  GIM, Family/Prev Medicine, Subspec  Nursing, Dentistry, Psychology, Social Work, OT, PT, other allied  Psychology, Sociology, Anthropology, Political Science, Economics  Management, Education, Public Policy Health Sciences: Main Campus:

20. Practice-focused research: Emerging models  VA QUERI  Academic Health System / School of Medicine “3I Institutes” (Improvement, Implementation, Innovation)  Integrated delivery system (Health Care Systems Research Network) embedded/partnership research: –KPSC Care Improvement Research Team –AcademyHealth Delivery System Science fellowship  Key features: joint governance, internal funding, negotiated scope, goals, standards (timeline, rigor)

21. The “Quality Chasm”  Institute of Medicine (1999, 2001)  Quality “report cards” (US, international)

22. Emergence and evolution of research interest 50+ years of research to identify causes and develop solutions to slow, uneven adoption of effective practices Changing physician behavior (1970s/80s: CME, reminders, incentives) Quality improvement, patient safety (1980s, 1990s, etc.) Implementation science (2000s to present) Findings, insights, recommendations are rich and valuable, yet difficult to apply VA, Kaiser and other systems have improved, but significant quality and performance gaps remain

23. The Tower of Babel problem Knowledge translation Translational research Research utilization, knowledge utilization Knowledge-to-action, knowledge transfer & exchange Technology transfer Dissemination research Quality improvement research T-1, T-2, T-3, T-4 Etc.

24. Achieving learning and improvement: Contributions from improvement science (QI research) vs. implementation science (1) QI often focuses on the “here and now” – immediate, local improvement needs via rapid-cycle, iterative improvement IS often attempts to develop, deploy and rigorously evaluate a fixed implementation strategy across multiple sites, emphasizing theory, contextual factors, (sometimes) mediators, moderators, mechanisms IS aims to develop generalizable knowledge

25. Achieving learning and improvement: Contributions from QI vs. implementation science (2) QI is pragmatic, improvement-oriented (often at the cost of limited confidence in interpretation and attribution and useful knowledge); IS is scientific, research/knowledge-oriented (often at the cost of improvement outcomes and useful knowledge) Neither has made much headway in achieving either goal

26. QI often ignores contextual factors, fundamental insights into organizational/professional behavior, cross-site differences and implications for improvement success IS usually ignores heterogeneity and dominance of context over intervention main effects, and – too often – mediators, moderators, mechanisms Neither has made much headway in achieving either goal Achieving learning and improvement: Contributions from QI vs. implementation science (3)

27. Achieving learning and improvement: Contributions from QI vs. implementation science (4) QI offers tools for persisting until improvement is achieved, driven by a desire to solve an identified quality problem IS offers theories, designs, methods, conceptual clarity for building from effectiveness/innovation work to implementation, to reap the benefits of innovation and research discovery and development

28. Achieving learning and improvement: Contributions from QI research and implementation science Despite some overlap, QI research and implementation science are largely complementary, and each could (should) learn and benefit from the other

29. Necessary conditions for practice change: insights from QI and implementation research 1.Valid, legitimate (accepted) evidence 2.Evidence of deviations 3.External expectations, interest (monitoring), pressure 4.Supportive professional norms 5.Etiology of practices, deviations 6.Information, evidence, education 7.Feasible methods/systems

30. Necessary conditions for practice change 1.Valid, legitimate (accepted) evidence 2.Evidence of deviations 3.External expectations, interest (monitoring), pressure 4.Supportive professional norms 5.Etiology of practices, deviations 6.Information, evidence, education 7.Feasible methods/systems

31. Implementation and QI in local settings factors contributing to success  Exceptional (non-routine, unsustainable, non- scalable) resources and support from central project team: –site-by-site, individualized technical assistance –funding for new staff, services –recruitment, hiring, training, supervision, support for new staff  Hawthorne effect (enhanced attention from monitoring, evaluation, external/internal interest)

32. Challenges to planned scale-up and spread Lack of exceptional resources coupled with: 1.Features of innovations 2.Features of target adopters 3.Features of the environment 4.Features of innovation champions 5.Features of scale-up/spread strategies ______________ Source: WHO and ExpandNet, Practical Guidance for Scaling Up Health Innovations, 2009. http://expandnet.net/PDFs/WHO_ExpandNet_Practical_Guide_published.pdf

33. Barriers to progress? Critiques and commentaries on the “state of the science” in implementation science often cite:  Lack of rigor; limited internal validity; too few RCTs  Limited external validity; too many RCTs (or too many flawed RCTs); use of “black box” evaluation approaches  Lack of theory; lack of appropriate theory  Too many theories; lack of guidance in using theory  Implementation and improvement problems and phenomena are extraordinarily complex (simple vs. complex vs. wicked problems)

34. An alternative (re-stated) hypothesis Implementation and improvement science study phenomena characterized by:  Heterogeneity and variability of program (intervention) content across time and place  Heterogeneity of program implementation across time and place  Significant and variable contextual influences (leadership, culture, experience/capacity, staff/budget sufficiency)  Strong mediator effects (indirect impacts) and attenuation of effects  Weak main effects (other than for robust programs)

35. Studying complex social interventions: What is our goal? Two very different questions 1.Does it work? Is it “effective”? Should it be approved? Included in the formulary? Should I use it? 2. How, why, when and where does it work? How should I use it? How do I make it work? For many or most implementation strategies, Q1 is meaningless

36. Developing insights and guidance for implementation and improvement  How do I choose an appropriate implementation or improvement strategy given my context?  How do I implement (deploy) the strategy to increase effectiveness?  How do I adapt and customize the strategy to increase effectiveness (initially and over time)?  How do I modify (manage) the organization or setting to increase effectiveness (initially and over time)?  How, why, when and where does it work?