Systematic Review Module 2: Analytic Frameworks Melissa McPheeters, PhD, MPH Associate Director for Methods, Vanderbilt University EPC Assistant Professor, Institute for Medicine and Public Health Assistant Professor, Obstetrics & Gynecology

Learning Objectives To understand the use of analytic frameworks within the systematic review process To develop an understanding of principles for developing analytic frameworks To review examples of formats for analytic frameworks 1

CER Process Overview 2

What Is an Analytic Framework? A type of evidence model, linking and defining clinical concepts, evidence, populations, and interventions that affect health outcomes (AHRQ Methods Guide; White and Ovid, 2009). Also known as – – Conceptual frameworks – – Influence diagrams – – Theoretical frameworks – – Simulation models – – Causal pathways 3

Why Use Analytic Frameworks? Frames the review in the clinical context Allows the reviewer to easily identify logical flaws as key questions are being developed Provides the clinical and biological underpinnings for the mechanism of action of the intervention under review Clarifies implicit assumptions in the review 4

The Importance of an Analytic Framework Helps describe an appropriate set of outcomes Identifies likely modifiers of effectiveness Can clarify likely links between intermediate and health outcomes, such as cholesterol and myocardial infarctions (Woolf et al., 2006) 5

When to Use Analytic Frameworks Directing literature searches – – (Bravata et al., 2005) Constructing key questions – – (Woolf et al.,1996) Targeting future research – – (Bravata et al., 2005) Demonstrating logic – – (Harris et al., 2001) Making knowledge gaps visible – – (Mulrow, Langhorne, and Grimshaw, 1997) 6

Building the Framework 7 Harris et al. (2001) We most often focus on the treatment component of the overall analytic framework model

Model CER Framework for Behavioral Outcomes 8

Sample Screening and Treatment Framework 9 Hartmann et al. (2009) PopulationInterventionOutcomes

Capturing the Context 10 Analytic frameworks can provide a means of capturing contextual issues important to a review

Defining the Framework Model Graphical chain of logic Graphical chain of logic – Arrows ● Linkages ● Preventive service or treatment ● Questions – Dotted lines ● Associations – Rectangles ● Intermediate outcomes u Rounded corners ● Health states u Square corners – Curved arrows ● Lead to ovals u Harms 11 Harris et al. (2001)

Exercise 12 Overall functioningTherapySustained health behavior change TreatmentInterventionIndividual diagnosed with condition HarmsPerson at riskLong-term outcomes Patient presenting symptomsReduced morbidity or mortalityShort-term outcomes Intermediate outcomesAdverse effectsSurrogate outcomes

Key Research Questions Key questions should be numbered on the diagram in the corresponding area of the framework (Whitlock et al., 2002). Key questions should be numbered on the diagram in the corresponding area of the framework (Whitlock et al., 2002)

Building the Framework Analytic frameworks can be developed more easily by adapting them from previously established similar frameworks, which is particularly helpful for young research topics (Bravata et al., 2005). Using PICOTS, make sure you can easily identify at least the population, intervention(s), outcomes, and timing on the framework; sometimes the comparator and setting are also made clear Place your key questions on the draft framework if they don’t fit easily; either the questions or framework may be off target Stand back and consider: the process of developing an analytic framework and associated key questions is iterative 14

Using the Framework Don’t build it and forget it—use the framework throughout the review process to – – Revisit inclusion/exclusion criteria – – Keep a handle on scope – – Reconsider the appropriateness of key questions Let the framework help in structuring the report and results 15

Summary Analytic frameworks provide patients, reviewers, decisionmakers, and providers with a model of evidence linking populations, interventions, and outcomes Frameworks direct literature searches, pinpoint key questions, target future research, demonstrate logic, and build arguments for legitimacy of therapeutic care Frameworks are created by building a graph of arrows, dotted lines, rectangles, and ovals, and by plugging in corresponding key research questions – – These can be easily adapted from previously constructed frameworks in relevant topic areas 16

References Bravata DM, McDonald KM, Shojania KG, et al. Challenges in systematic reviews: Synthesis of topics related to the delivery, organization, and financing of health care. Ann Intern Med (12 Pt. 2): Harris RP, Helfand M, Woolf SH, et al. Current methods of the US Preventive Services Task Force: A review of the process. Am J Prev Med (3 Suppl): Hartmann KE, McPheeters ML, Biller DH, et al. Treatment of overactive bladder in women. Evidence Report/Technology Assessment, No Rockville, MD: Agency for Healthcare Research and Quality Aug. AHRQ Publication No. 09-E017. Mulrow C, Langhorne P, Grimshaw J. Integrating heterogeneous pieces of evidence in systematic reviews. Ann Intern Med (11): Whitlock EP, Orleans CT, Pender N, et al. Evaluating primary care behavioral counseling interventions: An evidence-based approach. Am J Prev Med 2002;22(4): White CM, Ovid SP. AHRQ methods guide. Rockville, MD: Agency for Healthcare Research and Quality. Search 2009 July 23. Chapter 2, Finding evidence. Woolf SH, DiGuiseppi CG, Atkins D, et al. Developing evidence-based clinical practice guidelines: Lessons learned by the US Preventive Services Task Force. Annu Rev Public Health :