The Promise and Challenge of Dissemination and Implementation Science (DIS) Russell E. Glasgow, PhD Director, ACCORDS DISSEMINATION AND IMPLEMETATION SCIENCE Program University of Colorado School of Medicine https://goo.gl/fTUyrM Acknowledgments: Ross Brownson, David Chambers, PAUL ESTABROOKS, BRYAN FORD, Amy Kilbourne, Borsika Rabin and the great researchers cited throughout this talk

Everything I know (about DIS) Dissemination and Implementation Science (DIS) is about: Multi-level, contextual issues, and external validity Relevant, pragmatic models, research methods and measures Real world implementation and adaptation Designing for dissemination, sustainability and equity (Normal science (T1– T2) is necessary but not sufficient)

Objectives Why do we need DIS? How can DIS inform your work? What might you expect if you engage in DIS? ....better outcomes: better fit to local context, better uptake, etc. Current and Evolving Issues in DIS Overview some current DIS research areas and future opportunities

“PUBLICATION PATHWAY” Balas & Boren, 2000 Original research 18% variable Negative results Dickersin, 1987 Submission 46% 0.5 year Kumar, 1992 Koren, 1989 Acceptance Negative results 0.6 year Kumar, 1992 Publication Expert opinion 35% 0.3 year Poyer, 1982 Balas, 1995 Lack of numbers Bibliographic databases Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation “PUBLICATION PATHWAY”

It takes 17 years to transform 14% of original research Balas & Boren, 2000 Original research 18% variable Negative results Dickersin, 1987 Submission 46% 0.5 year Kumar, 1992 Koren, 1989 Acceptance It takes 17 years to transform 14% of original research into the benefit of patient care Negative results 0.6 year Kumar, 1992 Publication Expert opinion 35% 0.3 year Poyer, 1982 Balas, 1995 Lack of numbers Bibliographic databases Reviews, guidelines, textbook 9.3 years Inconsistent indexing Implementation “PUBLICATION PATHWAY”

Need for Pragmatic DIS Research Traditional biomedical RCTs study the effectiveness of treatments delivered to carefully selected populations under ideal conditions Even when we do implement a tested intervention into everyday clinical practice, we often see a “voltage drop”…a dramatic decrease in effectiveness Most common reason evidence- based programs are not adopted…they are not seen as relevant “If we want more evidence-based practice, we need more practice-based evidence.” Green LW Am J Pub Health 2006 Rothwell PM. External validity of randomised controlled trials. Lancet 2005;365:82-93.

Key Differences Between Traditional RCTs and Pragmatic DIS Controlled Trials (PCTs) A traditional RCT tests a hypothesis under ideal conditions A DIS Pragmatic Trial compares treatments under everyday clinical conditions GOALS To determine causes and effects of treatment To improve practice and inform clinical and policy decisions DESIGN Tests the intervention against placebo using rigid study protocols and minimal variation *Tests two or more real-world using flexible protocols & local customization* PARTICIPANTS Highly defined and carefully selected More representative because eligibility criteria are less strict MEASURES Require data collection outside routine clinical care Brief and designed so data can be easily collected in clinical settings RESULTS Rarely relevant to everyday practice Useful in everyday practice, especially clinical decision-making

T1-T4 Research Which type (T) is most common?

Bench to Bookshelf The end result of these challenges is that the most common type of research today is not ‘bench to bedside’; or ‘bedside to community’, but ‘bench to bookshelf’.

Some Remedies and Key Terms Implementation science is the study of methods to promote the integration of research findings and evidence into healthcare policy and practice. Dissemination research is the scientific study of targeted distribution of information and intervention materials to a specific public health or clinical practice audience. The intent is to understand how best to spread and sustain knowledge and the associated evidence-based interventions. Implementation research is the scientific study of the use of strategies to adopt and integrate evidence-based health interventions into clinical and community settings to improve patient outcomes and benefit population health. Pragmatic research is the use of real-world tests in real-world populations and situations. *Brownson, Colditz & Proctor. Dissemination and Implementation Research in Health 2018

A Big Tent of Terms (and ovals) Population Health (and Community) Services Health Services Research Implementation Science Implementation Research Dissemination Research Pragmatic Research Health Communication Research Quality Improvement Science QI . Adapted from Mitchell S, Chambers, D. https://doi.org/10.1200/JOP. 2017.024729.

Key Characteristics of D&I Science Glasgow RE, Chambers D. Developing robust, sustainable, implementation systems using rigorous, rapid and relevant science. Clin Transl Sci. 2012;5(1):48

Too often, we have assumed, “If you build it…and if you have evidence”

An Evidence-Based Cancer Prevention. or Weight Loss. or Mental Health An Evidence-Based Cancer Prevention... or Weight Loss... or Mental Health.....or (fill in blank) Story Even if 100% effective...is only so good as how and whether: it is adopted practitioners are trained to deliver it trained practitioners choose to deliver it eligible populations receive it it can be sustained If we assume 50% threshold for each step… (even with perfect access/adherence/dosage/maintenance) Impact: .5x .5x .5x .5 x .5 = 3% benefit www.Re-aim.org Glasgow RE, Vogt TM, Boles SM. Evaluating the public health impact of health promotion interventions: The RE-AIM framework. Am J Public Health. 1999;89(9):1322.

Pragmatic DIS Use of RE-AIM RE-AIM Dimension Key Pragmatic Priorities and Key DIS Questions to Consider and Answer Reach WHO is/was intended to benefit, and who actually participates or is exposed to the intervention? Effectiveness Adoption WHAT is/was the most important benefits you are trying to achieve and what is/was the likelihood of negative outcomes? WHERE is/was the program or policy applied and WHO applied it? Implementation HOW consistently is/was the program or policy delivered, HOW will it be adapted, HOW much will/did it cost, and WHY will/did the results come about? Maintenance WHEN will/was the initiative become operational; how long will/was it be sustained (setting level); and how long are the results sustained (individual level)? Glasgow R & Estabrooks P. Pragmatic applications of RE-AIM for health care initiatives in community and clinical settings. Preventing Chronic Disease. 2018; 15(1): E02.

The 5 Rs to Enhance Pragmatism, DIS and Likelihood of Translation Research that is: Relevant Rapid and Recursive Redefines Rigor Reports Resources Required Replicable LINDA)_ PLEASE MAKE LARGER TO FILL SLIDE Peek, C.J, et al. (2014). The 5 Rs: An Emerging Bold Standard for Conducting Relevant Research in a Changing World. Annals Of Family Medicine, 12(5), 447-55. doi:10.1370/afm.1688 deGruy, F.V, et al. (2015). A plan for useful and timely family medicine and primary care research. Family Medicine, 47(8), 636-42.

Reporting Resources Required Reporting on cost and other resources in a standardized manner is useful in: Demonstrating value Promoting rigor, transparency and relevance to stakeholders Present from perspective of stakeholders and decision makers Simple is fine – sophisticated economic analyses are not needed Report costs of conducting or replicating interventions Beyond money, costs can include clinician and staff time, training, infrastructure, startup costs, opportunity costs Ritzwoller, D P, et al. (2009). Costing behavioral interventions: A practical guide to enhance translation. Annals of Behavioral Medicine, 37(2), 218-27.

How well do research and practice connect? Dissemination

Dissemination Disconnects How local public health agencies learn about research findings? How researchers perceive they most effectively reach practitioners? Professional associations Journal articles Seminars/workshops Face-to-face meetings Email alerts Media interviews Press releases

Dissemination: What we know Dissemination generally does not occur spontaneously and naturally; Passive approaches to dissemination (diffusion) are usually ineffective; Single-source prevention messages are generally less effective than comprehensive, multi-level approaches; Stakeholder involvement in the research or evaluation process is likely to enhance dissemination; Theory and frameworks for dissemination are beneficial; and The process of dissemination needs to be tailored to various audiences From a broad set of literature, both within and outside health, here are some key points on what we know about dissemination of evidence-based programs and policies. You will hear these themes addressed thru-out our course. Many of these apply as much to implementation science as to dissemination science. Brownson RC, et al. Designing for dissemination among public health researchers...... Am J Public Health. 2013;103(9):1693-1699.

DIS Models + Frameworks are available! Outlined in Tabak, et al., proliferation of models: 61 reviewed, now more than 150!! Context is critical Focus on external validity Begin with stakeholders—take their perspective(s) Find balance between fidelity to EB program and adaptation to local setting Unlikely you need to create a new model www.dissemination-implementation.org website by Rabin et al. Tabak RG, et al. Bridging research and practice. Am J Prev Med. Sep 2012;43(3):337-350 Mitchell S & Chambers, D. Leveraging Implementation Science...J Oncology Practice, 2017.024729

Mode http://www.dissemination-implementation.org/

Types of Outcomes in DIS Research (Proctor, et al., 2010) Implementation Outcomes Service Outcomes Client Outcomes Acceptability Efficiency Satisfaction Adoption Effectiveness Function and QoL Appropriateness Equity Symptoms Costs Patient-centeredness Feasibility Timeliness Penetration Safety Sustainability Proctor E, Silmere H, Hensley M, et al. Outcomes for implementation research: Administration and Policy in Mental Health [serial online]. March 2011;38(2):65-76.

Key Outcomes Related to Health Equity Penetration (Reach)...It is ‘about the denominator’ Representativeness- levels of community, setting, staff, patient Costs and Feasibility- for setting; staff; and patients/citizens/users Other

Evidence-based...but on what? Sorely needed on external validity and pragmatic criteria…often ignored, including: Participant Representativeness Setting Representativeness Context and Setting Community/Setting Engagement Adaptation/Change Sustainability Costs/Feasibility of Treatment Comparison Conditions

Reporting for D&I vs. Usual HSR Focus Internal validity (some EV) External validity (some IV) Priority Fidelity Adaptation Format CONSORT Expanded CONSORT; StaRI

Questions? “I am all ears!”

Future Opportunities Examples of how DIS Research can help address current and future prevention, healthcare, health equity and population health

Example One: Learning Healthcare Systems (and Communities) Big Data, e-Health/m-Health, geo-coding Consumer-driven Healthcare policy reform Exchanges/ACOs Medicaid expansion Learning Healthcare Systems Aligning science with clinical priority goals Emphasis on costs and value Conducting more rapid and efficient studies Leveraging existing data to deploy and evaluate innovations and best practices ALSO VERY RELEVANT – VERY STRONG DESIGN LAB (USED TO BE LED BY KEVIN PATRICK NOW TAKEN OVER BY VERY TALENTED RELATIVELY NEW RECRUIT ERIC HECKLER - https://designlab.ucsd.edu/

The Long Road to Learning Healthcare Systems and Communities (and why we need DIS and pragmatic application) New research takes too long for adoption (if at all) Research is often not aligned to address critical health and health care problems or feasible Research is often not designed with stakeholders Providers lack practical tools/technical assistance and strategies to implement EB treatments (data are not enough) Large programs are often rolled out without adequate planning to maximize effectiveness, learning and sustainability Variation and patient-centered care Treatments work differently for different people and different settings

Example Two: Precision Medicine (Health) VERY RELEVANT – WE HAD JUST 2 WEEKS AGO THE PUBLIC HEALTH RESEARCH DAY PLENNARY SPEAKER TALK ABOUT : “ Putting Genomics in Context to Promote Health ” Ellen Wright Clayton, MD, JD Rosalind E. Franklin Professor of Genetics & Craig - Weaver Professor of Pediatrics Department of Health Policy, Vanderbilt University School of Medicine Professor of Law, Vanderbilt University School of Law How does clinical practice incorporate PMI findings? How do you implement evidence that will evolve? What does it cost and what are opportunity costs? How do you train and support the workforce? How do you ensure that PM findings reduce, not exacerbate, health inequities?

Convergence of Precision Health, DIS & Learning Health Care Systems and Communities GREAT!! Chambers DA, Feero WG, Khoury MJ. Conversion of implementation science, precision medicine, and the learning health care system. JAMA. 2016, 315: 1941-1942.

RE-AIM Precision Medicine (PM- or Precision Health) Questions Determine What percent and types of patients are Reached (equity); For whom among them is the PM intervention Effective, in improving what outcomes, with what unanticipated consequences (is health equity improved or decreased )? In what percent and types of settings is this approach Adopted; How consistently are different PM Implemented at what cost to different parties; And how well are the intervention components and their effects Maintained at both setting and individual levels? Gaglio B, Glasgow RE. Evaluation approaches…In: Brownson R, Colditz G, Proctor E, (Eds). Dissemination and implementation research in health. New York: Oxford University Press; 2018. 317-334.

Current DIS Funding Opportunities NIH: PAR-18-017; 18-007; (18 ICs & Offices) Purpose: To support innovative approaches to identifying strategies for the adoption, adaptation, integration, scale-up and sustainability of evidence-based interventions, tools, policies, and guidelines. Also benefit in studying how to “de-implement” or reduce the use of strategies and interventions that are not evidence-based, yield sub- optimal benefits for patients, and are harmful or wasteful. PCORI: Dissemination and Communication and Large Pragmatic Trials NHLBI: Center on Translational Research and Implementation Science (training and research); upcoming discussion of new RFA Add Fogarty?

Key Science Questions- New and Old Traditional science and evidence question: (necessary but not sufficient): “What intervention produces the largest average effect in tightly controlled trials on the major (clinical) outcome? Pragmatic DIS question: (contextual) “What program/policy components are most effective for producing what outcomes for which populations/recipients when implemented by what type of persons using what strategies under what conditions, with how many resources and how/why do these results occur?” Should I include this here...or not...save for longer talk? Or good to point ow how DIS relates to T2- T3??? ABSOLUTELY YES – WE CAN EVEN MOVE IT UP AND HAVE ONE SLIDE ON T1-T4 AS WELL

Everything I know (about DIS) Dissemination and Implementation Science (DIS) is about: Multi-level, contextual issues, and external validity Relevant, pragmatic models, research methods and measures Real world implementation and adaptation Designing for dissemination, sustainability and equity (Normal science (T1– T2) is necessary but not sufficient)

USE FOR DAY 1 AS WELL!!!

russell.glasgow@ucdenver.edu University of Colorado School of Medicine ACCORDS D&I Program SOME FROM BELOW SHOULD MOVE UP

Future Opportunities (and biased speculations) *Getting serious about CONTEXT: Measures; designs; international collab. *Real integration of health policy, public health, and biomedicine; clinical and community; impact on policy Focus on social determinants of health, health equity, generalizability of results to low resource settings, individuals and LMIC (and vice versa) Focus on COSTS, resources, de-implementation, and comparative effectiveness research Precision medicine, big data, AI and learning health systems

Expanded CONSORT Figure Existing CONSORT Figures for RCTs focus almost exclusively on internal validity Use of CONSORT- required by most medical journals, has improved quality and reporting on internal validity Until now, no comparable resource has addressed external validity. URL: https://goo.gl/8X2gzL Glasgow, Huebschmann, Brownson. Amer J Prev Med, 2018, DOI: https://doi.org/10.1016/j.amepre.2018.04.044

General Resources Brownson, RC, Colditz, GA, & Proctor, EK (2018). Dissemination and implementation research in health: translating science to practice. Oxford University Press. re-aim.org https://rtips.cancer.gov/rtips/index.do www.ucdenver.edu/accords/implementation www.Dissemination-Implementation.org RUSS- something on pragmatic

ACCORDS Dissemination and Implementation Science Program, University of Colorado Denver School of Medicine Our goal is to: Provide local consultation on D&I related research to increase funding and publication success Create collaborative learning partnerships with embedded research settings to translate research into practice more quickly and successfully Conduct cutting edge T3-T4 research on: pragmatic research and measures, adaptation of interventions, designing for dissemination, shared decision making, planning for and evaluation of reach, implementation and sustainability Use interactive on-line resources and support for patients, medical and public health students, trainees and faculty researchers Communicate the latest information on DIS related conferences, articles, grant opportunities, events, webinars, talks, and training opportunities www.ucdenver.edu/accords/implementation    

Bethany- want to do this Bethany- want to do this?- either live or maybe better to just use screen shot- and we could have the website online during Q&A... Up to you. 

T1 – T4 Extra