1. Systematic Review & Meta-Analysis 系统综述和meta 分析
Xu Xiong, MD, DrPH
School of Public Health and Tropical Medicine
Tulane University

2. Narrative Reviews, Systematic Reviews, and Meta-Analysis
Narrative Review: traditional expert review
Subjective, no formal rules in selecting studies, no standard statistical methods for combining studies
Systematic Review: review in which there is a comprehensive search for relevant studies on a specific topic, and those identified are then appraised and synthesized according to a predetermined and explicit method.
Meta-Analysis: systematic review that employs statistical methods (a quantitative summary) to combine and summarize the results of several studies.

3. Definition: Meta-Analysis
Coined by Glass in 1976 from the Greek prefix “meta” meaning “after,” “more comprehensive,” or “transcending” and the root, “analysis”
The statistical analysis of a large collection of analysis results from individual studies for the purpose of integrating the findings (Glass, 1976)
A statistical synthesis of the data from separate but similar (i.e., comparable) studies, leading to a quantitative summary of the pooled results (Last, 2001)
Synonyms: Research synthesis, Pooled analysis, Quantitative review, Overview
Gene Glass
Pooled analysis is sometimes used as a synonym. However, this term incorrectly implies that source data are merged, when in fact summary estimates are traditionally combined.

4. Number of Papers Referencing Meta-Analysis, 1985-2011
Results from MEDLINE search using MeSH or text word “meta-analysis”

5. Historical Note
1904 – Karl Pearson derived formulas to combine correlations from different samples
1932 – R.A. Fisher developed a method to combine p-values from different studies
1976 – Gene Glass coined the term meta-analysis
1977 – Smith & Glass published the first meta-analysis paper cited in MEDLINE
Meta-analysis of psychotherapy outcome studies (Am Psychol 1977;32: )
1989 – Meta-analysis was adopted by MEDLINE as a subject heading
1993 – Meta-analysis was adopted by MEDLINE as a publication type

6. Strength of Evidence Concerning Efficacy of Treatment

7. Meta-Analysis Meta-analysis differs from:
Primary analysis: the original analysis of data from a research study
Secondary analysis: the re-analysis of data to answer new research questions
Meta-analysis methods focus on:
Contrasting and comparing results from different studies in anticipation of identifying consistent patterns and sources of disagreements among the results

8. Why Use Meta-analysis
To provide a more objective appraisal of the evidence
To reduce the probability of false negative results
To test treatment effects in subgroups of patients
To explore and explain heterogeneity between study results
To generate research questions to be addressed in future studies

9. Meta-Analysis Objectives
Synthetic goal (estimation of summary)
Analytic goal (estimation of differences)
Meta-analysis is the part of the review process that concerns itself with the analysis of data extracted from the primary research included, and which uses quantitative methods to explore the heterogeneity of study results, estimate the overall measures of association or effect and assess the sensitivity of the results to possible threats from validity such as publication bias and study quality.

10. When to Use Meta-Analysis
When individual trials or studies are too small to give reliable answers
When large trials or studies are impractical or impossible
When there have been many trials or studies showing small effects are important
When trial or study results are inconclusive or conflicting

11. Potential Limitations of Meta-Analysis
Problems associated with design or reporting original studies
Publication bias
Limitations of using published data
Retrospective research
Variation of standard treatments over time
Heterogeneity of studies
Statistical methods

12. Steps of Meta-Analysis
Formulate research question
Develop a proposal
Comprehensive literature search
Selection of study
Critical appraisal of study
Extraction of data
Synthesis of data
Sensitivity and subgroup analyses if appropriate and possible
Preparing a structured report

13. Formulating a Research Question
What are the study objectives?
To validate results in a large population
To guide new studies
What are the operational definitions?
Disease or condition of interest
Population and setting
Treatment, intervention or exposures (e.g. risk factor, medication, diagnostic test)
Outcomes of interest (both beneficial and harmful)
What types of study designs?
Randomized controlled trials: e.g., Cochrane Review
Observational studies

14. Literature Scoping Useful for formulating the research question
Preliminary assessment of potentially relevant literature
Search for existing reviews and primary studies relevant to the topic
Usually only undertaken in a small range of databases relevant to the topic
Cochrane Controlled Trials Register (CCTR)
MEDLINE and EMBASE for medical topics
PsycLIT for reviews of psychological and psychiatric topics

15. Study Design Population Interventions Comparisons Outcomes
Inclusion Criteria
Study Design
Population
Interventions
Comparisons
Outcomes

16. Practical Considerations in Defining Eligibility for a Meta-Analysis
Study designs to be included
Years of publication or study conduct
Languages
Choice among multiple publications
Restrictions due to sample size or follow-up duration
Similarity of treatment and/or exposure
Completeness of information

17. Comprehensive Data Search
Need a well formulated and coordinated effort
Seek guidance from a librarian
Specify language constraints
Requirements for comprehensiveness of the search depends on the field and question to be addressed

18. Tulane Medical Library

19. Literature Search Computerized bibliographic database (MEDLINE)
Bibliography searches
Current contents
Dissertations
Textbooks
Databases of unpublished work
Citation searches
Expert survey
Meeting proceedings and abstracts
Granting agencies
Trial registries
Industry
Journal hand-searching

22. Literature Search Challenges
Database Bias - “No single database is likely to contain all published studies on a given subject.”
Publication Bias - selective publication of articles that show positive treatment of effects and statistical significance. It is important to search for unpublished studies through a manual search of conference proceedings, correspondence with experts, and a search of clinical trials registries.
English-language bias - occurs when reviewers exclude papers published in languages other than English
Citation bias - occurs when studies with significant or positive results are referenced in other publications, compared with studies with inconclusive or negative findings

23. Unbiased Selection and Extraction Process
Study Selection
Two independent reviewers select studies
Based on a priori specification of the population, intervention, outcomes and study design
Level of agreement: kappa
Differences are resolved by consensus
Specify reasons for rejecting studies

24. Data Extraction
Two independent reviewers extract data using pre-established forms
Should be explicit, unbiased, and reproducible
Include all relevant measures of benefit and harm of the intervention
Contact investigators of the studies for clarification in published methods, data
Extract individual patient data when published data do not answer questions about: intention to treat analyses, time-to-event analyses, subgroups, dose-response relationships
Methodological quality
Level of agreement: kappa
Differences in data extraction are resolved by consensus

25. Data Extraction: Study Characteristics
Types of publication (journal article, abstract or unpublished data)
Publication year and country of origin
Study participants (sample size, age, gender, race, health status)
Design details (case-control, cohort, parallel or cross-over, randomization, blinding)
Nature of treatment and control
Study duration
Measurement of compliance
Definition and measurement of outcome
Other confounders

27. Data Extraction: Study Outcome
Continuous variables
Mean difference between treatment and control groups
Binary variables
Odds ratios
Relative risks
Hazard ratios
Absolute risk reduction or number of patients needed to be treated to prevent one event

28. Critical Appraisal of Data
Description of Studies
Size of study
Characteristics of study participants
Details of specific interventions used
Details of outcomes assessed

29. Study Quality Assessment
Choose a method of assessment of quality of original studies, for example, Chalmers RCT Quality Score (Controlled Clinical Trials 1981;2:31-49)
Assess quality of each study in uniform, systematic and complete manner
Identify acceptable studies and give score to their quality
Keep a list of unacceptable studies
Consider weighting each study result by quality score, or stratifying by quality

30. Synthesis of Data Graphic displays Flow diagram Forest plot
Pooling data
Fixed-effects model
Random-effects model
Test for heterogeneity
Subgroup analysis
Meta-regression
Statistical techniques for meta-analysis can broadly be classified into two models, differing in how the variability of the results between studies is treated. The fixed effects models assumes variability is exclusively due to random variation or chance and that if all studies were infinitely large their results would be the same. The random effects model assumes a different underlying effect for each study and takes this into consideration as an additional source of variation.

31. Active Management of 3rd Stage: Cochrane Review/Meta-Analysis
A Forest Plot
Active Management of 3rd Stage: Cochrane Review/Meta-Analysis
Cochrane Review

33. Data Analysis
Include all relevant and clinically useful measures of treatment effect
Perform a narrative, qualitative summary when data are too sparse, or too low quality or too heterogeneous to proceed with a meta-analysis
Specify if fixed or random effects model is used
Describe proportion of participants used in final analysis
Use confidence intervals
Include a power analysis
Consider cumulative meta-analysis (by order of publication date, baseline risk, study quality) to assess the contribution of successive studies

34. Software for Meta-analysis: e.g., WinPEPI

37. Subgroup Analyses
Pre-specify hypothesis-testing subgroup analyses and keep few in number
Label all posteriori subgroup analyses
When subgroup differences are detected, interpret in light of whether they were:
Established a priori
Few in number
Supported by plausible mechanisms
Important (qualitative vs. quantitative)
Consistent across studies
Statistically significant (adjusted for multiple testing)

38. Sensitivity Analysis
Test robustness of results relative to key features of the studies and key assumptions and decisions
Include tests of bias due to retrospective nature (e.g., with/without studies of lower methodological quality)

39. Publication bias: A Funnel Plot
“A funnel plot is used as a way to assess publication bias in meta-analysis.”
Kevin C. Chung, MD, Patricia B. Burns, MPH, H. Myra Kim, ScD. “Clinical Perspective: A Practical Guide to Meta-Analysis.” The Journal of Hand Surgery. Vol.31A No.10 December p. 1676

40. Prepare a Structured Report
Include a structured abstract
Include a table of the key elements of each study
Include a flow diagram detailing the study selection process
Include summary data from which the measures are computed
Employ formative graphic displays representing confidence intervals, group event rates, sample sizes

41. Interpretation of Findings
Interpret results in context of current health care
State methodological limitations of the individual studies included and in the meta-analysis
Consider size of effect in studies and meta-analysis, consistency of effect sizes and any dose-response relationship
Consider interpreting results in context of temporal cumulative meta-analysis
Interpret results in light of other valuable evidence
Make recommendations clear and practical
Propose future research agenda (clinical and methodological requirements)

42. Summary
A well conducted meta-analysis allows for a more objective appraisal of the evidence than traditional narrative reviews
Meta-analysis may resolve uncertainties and disagreements in original research
Meta-analysis may enhance the precision of estimates of treatment effects
Exploratory analyses (i.e., subgroups who are likely to respond well to a treatment) may guide cost effective treatment decisions
Meta-analyses may demonstrate areas where the evidence is inadequate and thus identity areas where further research is needed