Adult and Child Consortium for Outcomes Research and Delivery Science Implementation Science: How It Can Complement, Extend, and Challenge How You Do Science (and increase your success) Dan D. Matlock, MD, MPH Russell E. Glasgow, PhD VA Eastern Colorado Geriatric Research Education and Clinical Center, Denver, Colorado Dissemination and Implementation Science Program, and Shared Decision Making Core Adult and Child Consortium for Outcomes Research and Delivery Science Changed to my full name Added our program/ core at ACCORDS

Conflicts of Interest FINANCIAL DISCLOSURE: Matlock: Grant support from the ACC Foundation, NIH, PCORI, AFAR, the Hartford Foundation, the Colorado Health Foundation Glasgow: National Institutes of Health (NIH), Agency for Healthcare Research and Quality (AHRQ), and Robert Wood Johnson Foundation (RWJF). UNLABELED/UNAPPROVED USES DISCLOSURE: None

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A Added discussion

Need for Implementation Science? Need for IS Need for Implementation Science? How Different Examples Tools Conclusions

Bench to Bookshelf Need for IS How Different Examples Tools Conclusions Bench to Bookshelf

Need for Implementation Science? Need for IS Need for Implementation Science? How Different Examples Tools Conclusions SLOW Traditional RCTs are slow and expensive Rarely produce findings that are easily put into practice Pragmatic research methods were developed in part to address common challenges to traditional efficacy research and clinical trials. The first one is that much clinical research is very slow It takes an average of 17 years before 14% of research findings lead to widespread changes in care.

Research to Practice Pipeline Need for IS How Different Examples Tools Conclusions Research to Practice Pipeline (MAY DROP)...I say this is figure showing drop of and ‘leakage across different steps in research translation..... Unfortunately- it is often more like a million gallon oil spill- then add in the background oil sheen... Than a leak Green, LW et al. Diffusion theory and knowledge dissemination…Annu Rev Public Health 2009;30:151-74

Need for Implementation Science Need for IS Need for Implementation Science How Different Examples Tools Conclusions Traditional RCTs study the effectiveness of treatments delivered to carefully selected populations under ideal conditions. Even when we do implement a tested intervention into everyday clinical practice, we often see a “voltage drop”—a dramatic decrease in effectiveness. “If we want more evidence-based practice, we need more practice-based evidence.” Green LW. Am J Pub Health 2006 Another challenge is that many practitioners and consumers do not see traditional research as relevant to their settings. As Larry Green has said….. Rothwell PM. External validity of randomised controlled trials…Lancet 2005;365:82-93.

Need for IS Science How Different Examples Tools Conclusions knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method.

Need for IS Science How Different Examples Tools Conclusions knowledge or a system of knowledge covering general truths or the operation of general laws especially as obtained and tested through scientific method.

Implementation Science Need for IS Implementation Science How Different Examples Tools Conclusions Implementation science is the study of methods to promote the integration of research findings and evidence into healthcare policy and practice. Dissemination science is the study of methods of distribution of information and intervention materials to a specific public health or clinical practice audience.

When are you ready to study Implementation Science? Need for IS How Different Examples Tools Conclusions When are you ready to study Implementation Science?

When are you ready to study D&I? Need for IS When are you ready to study D&I? How Different Examples Tools Conclusions Want to cite Brownson.. And also ad caveat that ‘as you will hear later.. 1) you do need evidence of effectiveness or efficacy) before implemenation, BUT It is never too early to plan for disseination/implementation/ sustainability..... Adapted from Figure 11.1 Implementation and dissemination of Prevention Programs (2009)

Effectiveness/Implementation hybrids Need for IS Effectiveness/Implementation hybrids How Different Examples Tools Conclusions Curran et al. Medical Care. 2012

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A

Need for IS How Different Examples Tools Conclusions “The significant problems we face cannot be solved by the same level of thinking that created them.” A. Einstein

Need for IS How Different Examples Tools Conclusions PCTs: Fewer Exclusions Allow for a Broader Subset of Settings, Staff, and Participants Traditional RCT PCT Efficacy, among a defined subset Eligible population Exclusions, non-response, etc. Effectiveness, in a broad subset Hard to read grey header against black background Figure provided by Gloria Coronado, PhD, Kaiser Permanente Center for Health Research

External Validity/ Pragmatic Criteria— Need for IS How Different Examples Tools Conclusions External Validity/ Pragmatic Criteria— Often Ignored Participant Representativeness Setting Representativeness Context and Setting Community/Setting Engagement Adaptation/change Sustainability Costs/Feasibility of Tx Comparison Conditions In our own work and a reference provided in our materials, we have found that the CONSORT PRECIS criteria are helpful, but not completely comprehensive. We also felt the dimensions above were important, and found these dimensions, like PRECIS ones, could be reliably rated after brief training

A Different Approach: Pragmatic Research Need for IS How Different Examples Tools Conclusions A Different Approach: Pragmatic Research Explanatory trial: Specialized experiment in a specialized population Pragmatic trial: Real-world test in a real-world population Pragmatic designs emphasize: Participation or reach Adoption by diverse settings Ease of Implementation Maintenance Generalizability Emphasize: Participation, timelines, generalizability Maclure, M. (2009). Explaining pragmatic trials to pragmatic policy-makers. Canadian Medical Association Journal, 180(10), 1001-1003.

Need for IS How Different Examples Tools Conclusions Key differences between Traditional Randomized Control Trials (RCT) and Pragmatic Controlled Trials (PCT) A traditional RCT tests a hypothesis under ideal conditions A PCT compares treatments under everyday clinical conditions GOALS To determine causes and effects of treatment To improve practice and inform clinical & policy decisions DESIGN Tests the intervention against placebo using rigid study protocols & minimal variation Tests two or more real-world treatments using flexible protocols & local customization PARTICIPANTS Highly defined & carefully selected More representative because eligibility criteria are less strict MEASURES Require data collection outside routine clinical care Brief and designed so data can be easily collected in clinical settings RESULTS Rarely relevant to everyday practice Useful in everyday practice, especially clinical decision making

Need for IS How Different Examples Tools Conclusions

Need for IS Other models How Different Examples Tools Conclusions Added comment on commonalities- 91 frameworks: http://dissemination-implementation.org/index.aspx Most Common at NIH: REAIM and DOI Many commonalities across models and theories

Readiness for Translation? RE-AIM Need for IS How Different Examples Tools Conclusions Readiness for Translation? RE-AIM Internal validity perspective The magnitude of effect as the key indicator of readiness for translation and adheres to the principles of evidence rating for determining efficacy External validity perspective Attention to intervention features that can be adopted and delivered broadly, have the ability for sustained and consistent implementation at a reasonable cost, reach large numbers of people, especially those who can most benefit, and produce replicable and long-lasting effects “Key RE-AIM dimensions are in RED” Some of the problems with what is known as the pipeline model include the application of an internal validity perspective as a the key indicator of readiness for translation… when in fact a balanced understanding of both internal and external validity is necessary to fully inform community and clinical health professionals (or others interested in adopting a program or policy) when they are considering changes in their environment. (talk through bullet points explicitly). Glasgow RE, Vogt TM, Boles SM. Evaluating the Public Health Impact…Am J Public Health, 1999;89:1322-1327

RE-AIM Precision (Personalized) Medicine Questions Need for IS How Different Examples Tools Conclusions RE-AIM Precision (Personalized) Medicine Questions Determine What percent and types of patients are Reached; For whom among them is the intervention Effective, in improving what outcomes, with what unanticipated consequences; In what percent and types of settings and staff is this approach Adopted; How consistently are different parts of it Implemented at what cost to different parties; And how well are the intervention components and their effects Maintained? Pawson R, et al. J Health Serv Res Policy 2005;10(S1)S21-S39. Gaglio B, Glasgow RE. Evaluation approaches…In:Brownson R, Colditz G, Procter E, (Eds). Dissemination and implantation research in health: Translating science to practice. New York: Oxford University Press; 2012. Pages 327-356

Key pragmatic questions to consider and answer Reach Need for IS How Different Examples Tools Conclusions RE-AIM Dimension Key pragmatic questions to consider and answer Reach WHO is (was) intended to benefit and who actually participates or is exposed to the intervention? Effectiveness Adoption WHAT is (was) the most important benefits you are trying to achieve and what is (was) the likelihood of negative outcomes? WHERE is (was) the program or policy applied and WHO applied it? Implementation HOW consistently is (was) the program or policy delivered, HOW will (was) it be adapted, OWHOW HOW much will (did) it cost, and WHY will (did) the results come about? Maintenance WHEN will (was) the initiative become operational; how long will (was) it be sustained (Setting level); and how long are the results sustained (Individual level)?

Why is this important? impact Loss at each RE-AIM step Need for IS How Different Examples Tools Conclusions Why is this important? impact Loss at each RE-AIM step Example of Translation of Interventions into Practice Dissemination Step Concept % Impact RG- what do you think about animation this one so blank table with columns comes up first…(also to second column- change tilte to ‘RE-AIM Concept” **YES_ precisely- I was just going to ask this *** Then have first two lines come in; then reach and implemntation lines; Then effectiveness; then maintenance (and in this line make red and BOLD the final 1.6%) So have 6 steps… lets see if this works or is too much  Re-aim.org

Why is this important? impact Loss at each RE-AIM step Need for IS How Different Examples Tools Conclusions Why is this important? impact Loss at each RE-AIM step Example of Translation of Interventions into Practice Dissemination Step Concept % Impact 50% of clinics use intervention Adoption 50.0% 50% of clinicians/staff take part 25.0% RG- what do you think about animation this one so blank table with columns comes up first…(also to second column- change tilte to ‘RE-AIM Concept” **YES_ precisely- I was just going to ask this *** Then have first two lines come in; then reach and implemntation lines; Then effectiveness; then maintenance (and in this line make red and BOLD the final 1.6%) So have 6 steps… lets see if this works or is too much  Re-aim.org

Why is this important? impact Loss at each RE-AIM step Need for IS How Different Examples Tools Conclusions Why is this important? impact Loss at each RE-AIM step Example of Translation of Interventions into Practice Dissemination Step Concept % Impact 50% of clinics use intervention Adoption 50.0% 50% of clinicians/staff take part 25.0% 50% of patients identified accept Reach 12.5% 50% follow regimen correctly Implementation 6.2% RG- what do you think about animation this one so blank table with columns comes up first…(also to second column- change tilte to ‘RE-AIM Concept” **YES_ precisely- I was just going to ask this *** Then have first two lines come in; then reach and implemntation lines; Then effectiveness; then maintenance (and in this line make red and BOLD the final 1.6%) So have 6 steps… lets see if this works or is too much  Re-aim.org

Why is this important? impact Loss at each RE-AIM step Need for IS How Different Examples Tools Conclusions Why is this important? impact Loss at each RE-AIM step Example of Translation of Interventions into Practice Dissemination Step Concept % Impact 50% of clinics use intervention Adoption 50.0% 50% of clinicians/staff take part 25.0% 50% of patients identified accept Reach 12.5% 50% follow regimen correctly Implementation 6.2% 50% benefit from the intervention Effectiveness 3.2% 50% continue to benefit after 6 months Maintenance 1.6% RG- what do you think about animation this one so blank table with columns comes up first…(also to second column- change tilte to ‘RE-AIM Concept” **YES_ precisely- I was just going to ask this *** Then have first two lines come in; then reach and implemntation lines; Then effectiveness; then maintenance (and in this line make red and BOLD the final 1.6%) So have 6 steps… lets see if this works or is too much  Re-aim.org

Ultimate Impact of a Weight Management Program Need for IS How Different Examples Tools Conclusions Ultimate Impact of a Weight Management Program Dissemination Step Concept Percent Impacted 8.8% of Weight Management sites participated Adoption 8.80% 5.9% of members participated Reach 0.52% 91.4% program components implemented Implementation 0.47% 43.8% of participants showed weight loss Effectiveness 0.21% 21.2% individuals maintained benefit (individual) Maintenance 0.04% Are they representative? **THIS slide should go right after the hypothetical example slide of sequence... Then immediately after insert that says’ WHAT IS THE MORAL OF THE STORY/THESE RESULTS? Then have the extended CONSORT slide Lets look at a real-world example of a weight loss intervention study that applied RE-AIM Note the right had side which shows the overall percent of the population impacted or improved by the intervention as we go down different RE AIM issues or program phases Abildso CG, Zizzi SJ, Reger-Nash B. Prev Chronic Dis 2010 May;7(3):A46

WHAT IS THE MORAL OF THIS STORY? Need for IS How Different Examples Tools Conclusions WHAT IS THE MORAL OF THIS STORY?

Comparison of Two Different Types of Programs Need for IS How Different Examples Tools Conclusions Comparison of Two Different Types of Programs Glasgow et al. AJPH, September 1999, Vol. 89, No. 9

Extended CONSORT Diagram Need for IS Extended CONSORT Diagram How Different Examples Tools Conclusions RG- NICE- can we make font on text in boxes more readable- maybe bold or ??? THIS IS IT- THANKS!...IF CAN MANIPULATE IT; MIGHT ADD ONE MORE LEVEL- SHOW SETTING AS IS; THEN ANOTHER LINE SHOWING STAFF WHO PARTICIPATE (UNLESS MAKES TOO BUSY/IMPOSSIBLE TO SEE) How is this? Can edit further if need be. re-aim.org: https://www.re-aim.hnfe.vt.edu/resources_and_tools/figures_and_tables/consort.pdf

What are unique considerations about this framework? Need for IS How Different Examples Tools Conclusions What are unique considerations about this framework? Intended to facilitate translation of research to practice Internal and external validity and emphasizes representativeness Individual and organizational factors Public health impact depends on all elements (reach x effectiveness, etc) Resource materials for researchers and community leaders (www.re-aim.org)

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A

Planned Parenthood Smoking Cessation Example Need for IS How Different Examples Tools Conclusions Planned Parenthood Smoking Cessation Example Patient randomized study (n = 1154) in low income Planned Parenthood clinics Eligible and target population = women smokers coming into clinic for contraception, wellness, or non-pregnancy follow-up INT= 9-minute tailored video, clinician advice to quit, brief behavioral counseling, follow-up phone calls Control = Advice to Quit and Stop Smoking brochure Problem with the reference at bottom Glasgow R et al. A brief smoking cessation intervention. AJPH, 2000, 90: 786-789

Planned Parenthood Smoking Cessation Results Need for IS How Different Examples Tools Conclusions Planned Parenthood Smoking Cessation Results Reach: 99% had smoking identified, 76% of smokers approached participated, no differences on demographics for participants vs. decliners Effectiveness: 10.2% quit INT vs. 6.9% CON at 6-week follow-up, p<.05 Adoption: Approached 4 clinics in lowest SES neighborhoods in area (most below 125% poverty level), also most diverse clinics—all participated Use yellow- males often color blind to red-green?

Results (cont.) Implementation: Need for IS How Different Examples Tools Conclusions Results (cont.) Implementation: Excellent (>85%) on all components except phone calls, which were problematic—only 43% successfully contacted Maintenance: Individual level: Higher, but NS different levels cessation (18.3 vs. 14.9%, p=.09) in INT condition at 6- month follow-up Setting level: Not reported Larger font- yellow

DECIDE – LVAD trial Need for IS How Different Examples Tools Conclusions

Need for IS DECIDE-LVAD Trial How Different Examples Tools Conclusions Objective: Understand the effectiveness and implementation of a shared decision support intervention for advanced heart failure patients considering DT LVAD.

DECIDE-LVAD Trial Key Considerations: Need for IS DECIDE-LVAD Trial How Different Examples Tools Conclusions Key Considerations: Desire to evaluate implementation in multiple real-world settings Specific small population (DT LVAD 25-50 per site/year) Max 5-6 sites due to budget constraints ($2M) Randomization needed to assess effectiveness Intervention involves both patients and clinicians/program Maybe have key terms in color? Note that multi-level focus- and study of context is key aspect of D&I

Study Design Options for DECIDE-LVAD Trial Need for IS How Different Examples Tools Conclusions Study Design Options for DECIDE-LVAD Trial Classic patient-level randomization Intervention is patient AND program-based; not at individual-level Diffusion among participants at each site is probable Cluster randomization Concerns about statistical power with only 6 total sites 3 sites intervention, 3 sites control Homogeneity of intervention participants and control participants Stepped wedge cluster randomization . . .

Need for IS DECIDE-LVAD Trial How Different Examples Tools Conclusions

Stepped Wedge Design Ideal When… Need for IS Stepped Wedge Design Ideal When… How Different Examples Tools Conclusions Diffusion of intervention to control participants is likely Implementation is a focus Iterative adjustment possible at each phase All clusters go through implementation (not half) Staggered rollout provides logistical, practical, or financial advantages All sites wish to receive the intervention Randomization itself can be a reason sites don’t participate NICE- might add when randomizaton is a problem- reason why sites may not participate

Evaluation Procedures Need for IS Evaluation Procedures How Different Examples Tools Conclusions Reach & Effectiveness: Patients and caregivers Surveys: Baseline 1 month 6 months Adoption, Implementation & Maintenance Qualitative interviews: Post-intervention implementation Post-study completion Checklist of education materials/procedures for each patient

Implementation Intervention Need for IS Implementation Intervention How Different Examples Tools Conclusions Pre-implementation: Planning, identifying key people Implementation visit 1 hour: Grand rounds presentation (large audience) 1 hour: Communication Training (heart failure team) 1 hour: Discuss new process Already a delivery process “plug and play” Post-implementation Ongoing site support Follow-up visit

Communication Training Need for IS Communication Training How Different Examples Tools Conclusions Four key communication concepts How to introduce the LVAD How to frame the options: How to respond to emotions: How to provide recommendations “Bad” “Good”

Need for IS How Different Examples Tools Conclusions RE-AIM Summary Points RE-AIM is an outcomes framework that can be used for planning and evaluation Each dimension is an opportunity for intervention RE-AIM can be used for efficacy, effectiveness, and implementation science projects All dimensions can be addressed within a given study (though likely not all intervened upon) Methods exist to combine and summarize RE- AIM outcomes MAYBE? After this.... The PRISM figure- to show newer directions- role of CONTEXT? So lets quickly summarize some key things about RE-AIM Hard to read grey header **HERE- or earlier?? Present Extended CONSORT? Dr. Paul Estabrooks - 540-857-6664

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A

Need for IS How Different Examples Tools Conclusions The Pragmatic-Explanatory Continuum Indicator Summary (PRECIS) Planning Tool How pragmatic is your study? Tool to help in planning and reporting. (see next slide) Something missing? Figure? DAN: Dunno – I referred them to the next slide. Gaglio, B., et al. (2014). How pragmatic is it? Lessons learned using PRECIS and RE-AIM for determining pragmatic characteristics of research. Implementation Science, 9(1), 1. Thorpe KE, et al. A pragmatic-explanatory continuum indicator summary (PRECIS)…CMAJ 2009;180(10):E47-E57.

https://www.precis-2.org/

Need for IS How Different Examples Tools Conclusions Seems ***OUT OF ORDER??? Talked about Pragmatic trials before??? Dan: Dunno – move or delete? Example from: Little P, Moore M, Kelly J, Williamson I, Leydon G, McDermott L, Mullee M, Stuart B: Ibuprofen, paracetamol, and steam for patients with respiratory tract infections in primary care: pragmatic randomised factorial trial. BMJ 2013, 347:f6041.

Overview Need for Implementation Science Need for IS Overview How Different Examples Tools Conclusions Need for Implementation Science How is Implementation Science Different? Examples Tools and Resources Conclusions, Discussion; Q & A

NEW AREAS Evolution of RE-AIM Focus on Context! Need for IS How Different Examples Tools Conclusions Evolution of RE-AIM Reviews documenting use over time Applicability to many different content areas Used for both planning and evaluation Underreporting of key components Setting level factors reported much less often (e.g., adoption) Maintenance (sustainability) reported least often NEW AREAS Health Policy Multilevel community interventions Built environment Patient centered outcomes research Focus on Context! Gaglio et al. The Re-AIM framework….AJPH 2013; 103: 38-46.

Replicability (and Generalizability) Need for IS Replicability (and Generalizability) How Different Examples Tools Conclusions Important to report conditions under which program was delivered To what extent is the program replicable: In similar settings? In different settings? Goal – what intervention do you compare it to (real world alternative)? RG- on second thought, not sure PICOT adds- introduces somewhat tangential point I do not have time to address… lets take it out upon reflection- sorr **NEW thought- can we ‘animate this slide- so that bottom line question comes in in a second step? *YES- EXACTLY AND YES- REMOVE PICOTS Let think if want this on any otrh slide- should keep it minimal but may be helpful? y ALSO CITE PICOS HERE? I’ve added this as a bullet. Does this get at what you had in mind?

Replicability (and Generalizability) Need for IS Replicability (and Generalizability) How Different Examples Tools Conclusions Important to report conditions under which program was delivered To what extent is the program replicable: In similar settings? In different settings? Goal – what intervention do you compare it to (real world alternative)? Bottom Line and Ultimate Use question: “What program/policy components are most effective for producing what outcomes for which populations/recipients when implemented by what type of persons under what conditions, with how many resources and how/why do these results come about?” RG- on second thought, not sure PICOT adds- introduces somewhat tangential point I do not have time to address… lets take it out upon reflection- sorr **NEW thought- can we ‘animate this slide- so that bottom line question comes in in a second step? *YES- EXACTLY AND YES- REMOVE PICOTS Let think if want this on any otrh slide- should keep it minimal but may be helpful? y ALSO CITE PICOS HERE? I’ve added this as a bullet. Does this get at what you had in mind?

Resources (SEE YOUR HANDOUT) Need for IS Resources (SEE YOUR HANDOUT) How Different Examples Tools Conclusions Brownson, R. C., Colditz, G. A., & Proctor, E. K. (2012). Dissemination and implementation research in health: translating science to practice. Oxford University Press. Re-aim.org https://www.precis-2.org/ www.ucdenver.edu/accords/implementation www.Dissemination-Implementation.org

Questions? We’re all ears! Need for IS How Different Examples Tools Conclusions Questions? We’re all ears!

Common questions? “Implementation Science is about context (and context change); generalizability and applicability; adaptation; and designing for sustainability and dissemination How does ‘adaptation’ fit with fidelity to protocol? How do I plan or design for dissemination? Is this feasible or realistic to do in my situation?