A Meta-Analysis Primer Rick Chappell - 641, 12/15/2010

Outline I. Definition II. Example 1 - Cervical Carcinoma at UW III. How to Numerically Summarize Evidence IV. Examples of Presentation V. Reasons for Meta-Analysis VI. Problems with Meta-Analysis

I. Definition n “ Meta-analysis is a relatively new method for reviewing and combining results from multiple clinical trials. Whereas other review methods usually involve narrative discussions of individual trials, a meta- analysis … systematically combines and evaluates the results of clinical trials that have been completed.” - Spilker, Guide to Clinical Trials, 1991, Chapter 104

n A “systematic review” is: –Numeric –Complete n Often a separate publication n Can be used in the Discussion of a trial publication to compare its results with others’ and to summarize evidence

II. Example 1 - Cervical Ca at UW n Does treatment prolongation change outcome? n If so, among which subgroups of patients? (Example presented on Wed. 12/8)

n Moral: the scientific method is not a democratic process. Meta-analysis does not consist of “voting” for the most popular result. P-values do not provide a proper summary of evidence - estimates do.

III. How to Numerically Summarize Evidence A. Continuous Outcomes - Difference in Means (usually) or medians. –Means are easy to combine: averaging means gives you a new mean (unlike with medians) –E.g., mean income is the average of mean male and mean female incomes (average may need to be weighted if # of males isn’t equal to # of females).

B. Time to Event (“Survival”) Outcomes - Hazard Ratio (HR) HR = (Probability of event in a short time period for a ‘treated’ patient) / (Probability of event in a short time period for an ‘untreated’ patient). – Sometimes also called relative risk. – HRs may are often constant over a wide range of risk levels.

C. Binary Outcomes - Odds Ratio (OR) Odds = (Probability of event) / (Probability of no event) OR of event due to ‘treatment’ = (Odds of event given ‘treatment’) / (Odds when not given ‘treatment’)

Why are odds ratios commonly used? n They are useful in case-control studies (e.g., the odds of being an asbestos miner if you have mesothelioma are the same as the odds of having mesothelioma given you are an asbestos minor). This isn’t the reason here. n Statistical Reasons - ORs are easy to combine. n Like hazard ratios, ORs are often constant over a wide range of risk levels.

IV. Examples of Presentation n Table of “O - E” (observed - expected in the ‘treatment’ group = excess incidence in that cohort). These can be added up. n Plot of trial # vs. odds ratio n Both combined: “Forest plot”

Presentation of Meta-analysis: Example n Clahsen, et al. JCO 115. pp (1997). Perioperative polychemotherapy (PeCT) in breast cancer. n Forest plot of hazard ratio (PeCT vs. no PeCT) of disease-free survival.

Example 3 - Berry, et al. (JCO, to appear)

V. Reasons for Meta-Analysis (Armitage & Colton, Enc. of Biostatistics Vol. 4, 1999) n Narrative Reviews can be distorted and misleading. n Explosion of research evidence cannot be easily assimilated without a formal review. n Since individual CTs’ sample sizes may be too small to reliably detect clinically important effects, synthesis is necessary. Meta-analysis provides statistical power.

VI. Problems with Meta-Analysis n Intertrial heterogeneity (see Table of Spilker). n “Garbage in garbage out.” n Publication bias - any relation between the direction of study’s results and their dissemination.

Publication Bias n E.g., ORG-2766 protected nerves from cytotoxic injury in 55 women with ovarian cancer - NEJM lead article (van der Hoop, et al., 1990); a subsequent negative study of 133 women - ASCO Proceedings abstract (Neijt, et al., 1994). n Vickers, et al. (1998) show that the problem is widespread: in some countries, 100% of publications show treatment effects.

An attempted Cure - The Cochrane Collaboration n An international organization which maintains complete registries and prepares reviews of research on clinical interventions. n Home page: orwww.cochrane.de n Cochrane library: