Systematic Review Module 12: Presentation of Findings Melissa McPheeters, PhD, MPH Jeff Seroogy, BS Vanderbilt University EPC Joseph Lau, MD Thomas Trikalinos, MD, PhD Tufts EPC

Learning Objectives To have a basic understanding of the ways that results are commonly presented in a CER To understand the benefit of synthesizing nonquantitative results To review examples of graphical presentations of nonquantitative findings 1

CER Process Overview 2

Organizing and Reporting Findings Once the evidence tables are completed, there is an enormous (sometimes overwhelming) amount of data yet to synthesize Various types of heterogeneity may preclude a quantitative synthesis Nonetheless, qualitatively synthesizing heterogeneous studies can capture underlying similarities to support conclusions Making sense of the data requires good tabular presentation and clear organization and writing in the text 3

Combining Studies Synthesizing results from different studies allows for broader conclusions to be drawn – – PICOTS can be employed to determine what criterion to focus on when combining results Population, intervention, comparator, outcome, timing, setting The nature of the review and the substance of the literature will dictate the most appropriate criterion to use Not all studies can be (or should be) combined – – Various statistical models are useful for identifying the level of heterogeneity between studies (see Methods Guide) – – Even with statistically significant variation, studies still may be combined unless systematic differences among studies are evident or when combining would yield meaningless, skewed, or biased results 4

Challenges to Combining Studies Even studies meeting the same inclusion criteria can vary substantially: – – Clinical heterogeneity: variation in the study population, interventions, and outcomes – – Methodological heterogeneity: variation in study design – – Statistical heterogeneity: variation in observed treatment effect (for trials) Many factors can contribute to variation in seemingly similar studies: – – Evolving diagnostic criteria, evolving diseases, difference in baseline characteristics, differences in care, etc. 5

Solution … Qualitative synthesis through well- constructed text and summary tables Content and order of report depend on the available evidence, but should adhere to a set of guiding principles Flow and clarity of the document support development of clear conclusions and summary statements, even with heterogeneous studies 6

Writing the Report A standard order for the overall report is recommended Subsection ordering may vary but should adhere to principles of clarity, be consistent with key questions, and may be guided by PICO(TS) Summary tables should be presented within each section; where multiple sections present comparable data, comparable tables should be available 7

Overall Order of the Document Sections prior to and after results are standard Results are then usually structured per the key questions Order of results presented within key questions can vary, but when there is a primary outcome of interest (generally efficacy), it is presented first Modifying questions are generally presented after direct efficacy and effectiveness questions, with the order possibly guided by PICOTS 8

Document Structure Chapter 1: Introduction – – The nuts and bolts: defining the project, purpose and scope of the CER, key questions, analytic framework, experts involved, etc. Chapter 2: Methods – – Literature review methods: literature search methods, inclusion and exclusion criteria, developing (NOT presenting) evidence tables, approach to quality, data abstraction, and data synthesis process No “results” (i.e., tables) presented in this chapter, but rather a guide as to how the information was collected and the tables created Chapter 3: Results – – Broken down by key questions, with subsections when applicable (focus of this module) Chapter 4: Discussion – – Sections discussing strength of the literature and evidence, principal findings (broken down by key questions), areas of future research, and any conclusions that can be drawn 9

Presenting Results Chapter 3: Results: example from Overactive Bladder review Subsections determined over course of the review, dictated by the substance of the literature, technical expert panel guidance, and PICOTS Depending on findings: some key questions will have several subsections (KQ2/KQ3); some will have few (KQ4/KQ5) 10

Presenting Results: Levels of Organization Level 1: Key questions Level 1: Key questions Level 2: Intervention type Level 2: Intervention type Levels 3+: Use key questions and PICO(TS) to organize … Levels 3+: Use key questions and PICO(TS) to organize … 11

PICOTS PICOTS can be used to establish ordering of results For example, if multiple interventions are being examined, interventions (I) might be the meta-organizational level, with other PICOTS reflected within Or, if the same intervention is being studied in different populations (P), the report could be structured by population (e.g., results seen in women followed by results seen in men) with interventions (I) ordered within populations (P) 12

Organizing by and within PICO(TS) Population: primary populations of interest followed by subpopulations Intervention: hierarchy of interventions may reflect most common to least Comparator: where multiple comparisons are made, one might present the more “usual” decision first Outcome: a hierarchy of outcomes may reflect higher to lower acuity, or target outcomes followed by collateral ones; harms generally presented last Timing: generally short term before long term Setting: an intervention may be available in various settings (e.g., inpatient or outpatient) 13

Principles The order of sections and subsections should reflect the key questions and a conscious attention to PICOTS. Importantly, it is: – – Not necessarily a ranking of importance – – Structured to reflect the strength of the data and outcomes – – Useful in determining the applicability of different treatments or the likelihood of different outcomes 14

Organization within Intervention Category (Level 3) Within intervention category : – Content of the literature (types of studies, total N, outcomes included) – Outcomes (efficacy, effectiveness and other) – Harms Treatment Category A – Treatment A1 u Literature content u Results –RCTs –Outcome A –Outcome B –… –Cohort Studies –Outcome A –Outcome B –… –Harms – Treatment A2 15

Summary Tables Within the structure described thus far, represent a map of the outcomes Provide a snapshot of global characteristics of the CER; tables will differ in their structure and content based on the included literature of the CER Highlight gaps in the present research and practical goals for future studies 16

Summary Tables Whereas evidence tables represent an individual study, summary tables combine multiple studies to illustrate overarching trends in the data – – Can be designed to include characteristics of all included studies i.e., funding sources, assessment method (mailed questionnaire, telephone questionnaire, etc), country of study, etc. – – Can be designed for subsets of included studies i.e., summary tables for RCTs, prevalence studies, harms/side effects, outcomes for specific treatments, etc. – –. 17

Overall Summary Table 18

Summary Tables Another basic summary table is the “Study Characteristics” table Another basic summary table is the “Study Characteristics” table This will vary some between projects, but each should have some form of this table This will vary some between projects, but each should have some form of this table Fields can be taken from the global descriptors in the evidence tables or at the recommendation of content experts Fields can be taken from the global descriptors in the evidence tables or at the recommendation of content experts 19

Balk et al. Effects of Statins on Nonlipid Serum Markers Associated with Cardiovascular Disease. Ann Intern Med

Summary Matrix Primary prevention: cohort study of fish consumption and all cause mortality ( EPC Report # 94, Effects of omega-3 fatty acids and cardiovascular disease 2004) 21

Summary Tables Summarizes information for each study answering one question Simplified entry (one row) for each study Some dimensions for table columns – – PICO (may be listed in table title or headers) – – Methodological quality – – Applicability – – Study size (weight) – – Magnitude of effect One study may belong to multiple summary tables (e.g., different outcomes) 22

Summary Tables Summary tables can be specialized for different subgroups of the literature Summary tables can be specialized for different subgroups of the literature Example: a table presenting the findings from only the included RCT studies of one treatment type Example: a table presenting the findings from only the included RCT studies of one treatment type 23

Summary Table Example Secondary prevention RCTs: supplements 24

Summary Table Example Primary prevention: cohort study (EPC Report # 94, Effects of omega-3 fatty acids and cardiovascular disease 2004) 25

Summary Tables Summary tables can be specialized for different subgroups of the literature Summary tables can be specialized for different subgroups of the literature Example: a table presenting the findings from prevalence literature Example: a table presenting the findings from prevalence literature 26

Summary Tables Summary tables can be specialized for different subgroups of the literature Summary tables can be specialized for different subgroups of the literature Example: a table presenting the proportion of participants experiencing harms and side effects from treatment studies Example: a table presenting the proportion of participants experiencing harms and side effects from treatment studies 27

Summary Tables Summary tables can be specialized for different subgroups of the literature Summary tables can be specialized for different subgroups of the literature Example: a table presenting the outcomes of a certain treatment Example: a table presenting the outcomes of a certain treatment 28

Summary Table Example Secondary prevention RCTs: diet/dietary advice 29

Forest Plot without Summary 30 Favors WACA Favors PVI Wide area circumferential ablation (WACA) vs. pulmonary vein isolation (PVI): Freedom from atrial fibrillation recurrence; Evidence Report “Comparative Effectiveness of Radiofrequency Catheter Ablation for Atrial Fibrillation” by Tufts EPC

Evidence Map A succinct visualization of studies available to address specific question(s) along one or more variables of interest – – Results of outcomes typically not included Can help to identify comparisons where evidence is sufficient to be analyzed To guide readers in knowing what comparisons and outcomes are available in the report To quickly appreciate evidence gaps 31

Update on Acute Bacterial Rhinosinusitis. Evidence Report

Primary Studies on Vitamin D Intake or Concentration 33

Example of Evidence Map Figure 2. Current clinical evidence on particle beam therapies. Terasawa T, et al. Ann Intern Med

Evidence Map: VKORC1 Single Nucleotide Polymorphisms and Mean Warfarin Maintenance Dose or Anticoagulation Outcomes 35

Wrapping Up PRISMA checklist – – Preferred Reporting Items for Systematic Reviews and Meta-Analyses – – A list of items that should be included in any evidence review and flow diagram detailing the reporting of included studies i.e., title, study selection process, synthesis of results, etc. (27 total items in the 2009 version) – – Very useful tool to reference during the writing of a review, and a necessary checklist to work through during the closing stages of a project – – Available at 36