1. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 18 Systematic Review and Meta-Analysis

2. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter Overview How to complete a systematic review. Hierarchy of evidence. Identify the domains and elements of a systematic review. Difference between narrative and systematic reviews. Identify obstacles clinicians must overcome to practice evidence-based medicine. Differences between systematic review and meta-analysis. How to conduct a meta-analysis. Concept of validity in systematic review. Role of the Cochrane Collaboration in evidence-based health care. Concept and implications of publication bias in systematic review.

3. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Systematic Review Systematic reviews are clinician friendly and summarize the evidence into a single paper—the most compelling evidence in most circumstances. Systematic review is a research process in which investigators identify previous studies that address a particular question, summarize findings, and combine data for meta-analysis. Traditional Literature Review vs. Systematic Review Systematic review: the investigators begin with an answerable question and work through a planned process. Traditional literature reviews: discuss multiple references related to an issue but often limit the included literature to that which supports the authors’ position. The Agency for Healthcare Quality and Research outlined the essential elements of systematic review.

4. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Systematic Review as a Research Process Systematic review is a research process, so the research methods impact the validity of data and conclusions drawn. Clinicians should evaluate systematic reviews carefully before accepting them at face value because of their high rankings on evidence hierarchies. A systematic review offers a comprehensive analysis of the clinical literature available at the time it’s prepared. Systematic review may lead to practice recommendations and clinical practice guidelines.

5. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Systematic Review as a Research Process In a systematic review, the literature search strategy must be described in a manner that is fully reproducible. Include a detailed description of the criteria and methods employed to include or exclude papers from the final analysis, as well as of how final papers were included. Flow diagrams can help the reader follow the process in the review. Methods used to extract data from individual papers for analysis should be described, including a description of investigators blinded to the assessment of others. There are multiple grading scales of methodological quality of diagnostic and intervention studies.

6. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Data Synthesis Data Synthesis is the process by which data from multiple studies are combined. Data from studies using similar research methods and measurements can be combined through meta-analysis. Meta-analysis can substantially increase the size of the sample contributing data for analysis, thus increasing statistical power and narrowing confidence intervals. The similarity among treatments and among patients included in individual studies warrant combining data. The variables being measured and the units of measurement may also preclude meta-analysis. When meta-analysis is not possible, qualitative analysis can yield important information.

7. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Data Analysis (continued) Once decisions regarding pooling of data are made, qualitative data analysis or quantitative meta- analysis is performed. The issue of harm related to diagnostic procedures and interventions is an important and sometimes overlooked issue. –Many diagnostic procedures and treatments have potential side effects. Although surgical management may have benefits, those benefits must be weighed against the frequency and severity of side effects in the clinical decision process. Researchers directing clinical trials and systematic reviews must report adverse events, and the researcher consumer needs to consider the incidence and morbidity associated with the events.

8. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Meta-Analysis Meta-analysis: raw data or mean difference data from multiple studies are pooled. The reader of meta-analyses will find reference in the methods section of these papers to fixed or random effects models. Fixed effects models: ask the question “is there evidence here (within the data available) of an outcome effect?” Random effects models: address whether “the available data indicate that the larger population of data from which they were drawn provides evidence of an outcome effect.” Random effects analysis projects greater generalizability of results. When mean difference values are pooled, the values should be weighted based upon the sample size of the study.

9. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Considerations: Applying Results from Systematic Review to Patient Care The Systematic Review (of therapy) Worksheet illustrates a process clinicians can use to make decisions about the extent to which the results of a systematic review should influence a plan of care. After completing the worksheet questions, clinicians consider the likelihood that the patient will benefit from the intervention, as well as the potential for adverse responses to treatment. The response to an intervention can be either preventative or therapeutic. The response can be conveyed as: –The magnitude of change on one or more measures. –A probability of a favorable or adverse outcome.

10. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Is this a systematic review of randomised trials? Does it include a methods section that describes: a)finding and including all relevant trials? b)assessing their individual validity? Were the results consistent from study to study? Were the individual patient data used in the analysis (or aggregate data)? Are the results of this systematic review valid? Sample Questions from SYSTEMATIC REVIEW (OF THERAPY) WORKSHEET (Reprinted with permission from University of Toronto’s University Health Network, available at http://www.cebm.utoronto.ca/teach/materials/sr.htm.)

11. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Do these results apply to your patient? Is your patient so different from those in the study that its results cannot apply? Is the treatment feasible in your setting? What are your patient’s potential benefits and harms from the therapy? Method I: In the OR tables above, find the intersection of the closest odds ratio from the systematic review and your patient’s expected event rate (PEER). Method II: To calculate the NNT from any OR and PEER: Are your patient’s values and preferences satisfied by the regimen and its consequences? Do you and your patient have a clear assessment of his or her values and preferences? Are they met by this regimen and its consequences? Can you apply this valid, important evidence from a systematic review in caring for your patient? (Reprinted with permission from University of Toronto’s University Health Network, available at http://www.cebm.utoronto.ca/teach/materials/sr.htm.)

12. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Do they really make biologic and clinical sense? Is the qualitative difference both clinically (beneficial for some but useless or harmful for others) and statistically significant? Was this difference hypothesised before the study began (rather than the product of dredging the data), and has it been confirmed in other, independent studies? Was this one of just a few subgroup analyses carried out in this study? Should you believe apparent qualitative differences in the efficacy of therapy in some subgroups of patients? Only if you can say “yes” to all of the following: (Reprinted with permission from University of Toronto’s University Health Network, available at http://www.cebm.utoronto.ca/teach/materials/sr.htm.)

13. Copyright © 2011 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter Summary and Key Points Systematic review provides the clinician with results compiled from multiple clinical trials in a single document—sometimes with the effect of defining current best practice. A literature review is often limited because it supports a position taken by the author, whereas a systematic review begins with an answerable question and works through a planned process. Because systematic review is a research process, the research methods affect the validity of the data and the conclusions drawn. Data from studies using similar research methods and measurements can be combined through meta-analysis. Systematic reviews are used to provide clinicians with the most well- supported evidence of solutions to clinical problems. Because larger samples are more likely to represent true population values, a systemic review is at the top of the evidence value hierarchy. Consideration of patient values serves as a reminder that clinicians should not get carried away interpreting numbers and forget that the patient is the focus of attention.