PH 401: Meta-analysis Eunice Pyon, PharmD (718) , HS 506

Meta-analysis Quantitative systematic review Combines data from previously conducted clinical trials (and epidemiologic research) and performs statistical analyses on pooled results NOTE: different from a review article

Meta-analysis Useful when: 1. definitive clinical trials are impossible, unethical or impractical 2. randomized trials have been performed but results are conflicting 3. results from definitive trials are being awaited 4. new questions not posed at the beginning of the trial need to answered 5. sample sizes are too small

Meta-analysis Purposes include: 1. to increase statistical power for primary endpoint and or subgroups 2. to resolve uncertainty when reports disagree 3. to improve estimates of size of effect 4. to answer new questions not posed at the start of the trials 5. to bring about improvements in quality of primary research

Meta-analysis: SSRIs Whittington CJ, et al. Selective serotonin reuptake inhibitors in childhood depression: systematic review of published versus unpublished data. Lancet 2004;363:

Meta-analysis: Cox-2 Furberg CD, et al. Parecoxib, valdecoxib, and cardiovascular risk. Circulation 2005;111:249.

Meta-analysis: the process 1. Problem formulation 2. Data collection 3. Evaluation of the collected data 4. Analysis and interpretation 5. Presentation of Results

Problem formulation Clearly define the clinical question specify variables evaluate relationship between variables (cause and effect)

Data collection Describe details of literature search databases published vs. unpublished additional sources (i.e.., reference lists, meetings) Describe inclusion/exclusion criteria study design participants treatment outcome measures

Evaluation of data Even before the “data” Author Funding Relevant information Important part of data evaluation. Different ways to incorporate into meta- analysis: exclusion, weighting, stratifying

Evaluation of data Raw data, individual patient data preferred difficult to obtain Summary data more commonly utilized

Evaluation of data Homogeneity vs. heterogeneity L’abbe plot Cochran-Q Publication Bias Funnel plot

Funnel Plot

Meta-analysis continued Remember: Meta-analysis is an observational study of evidence. It is retrospective.

Evaluation of data Scrutinize validity of trials randomization techniques sample size compliance blinding intention to treat vs. per protocol Primary studies may be weighted to reflect quality of research design. Weighting of data is controversial. Investigators should be blinded to: authors, institutions, journals, funding, acknowledgements.

Analysis and interpretation Appropriate statistical analyses standardized outcome measure Continuous (i.e., blood pressure): differences, standard deviations Binary (i.e., dead or alive): odds ratio, relative risk overall effect; combining data fixed effects model--assumes same effect across studies random effects model--assumes different underlying effect for all studies and others…

Analysis and interpretation Odds Ratio Sample GroupDiseaseNo Disease Treatment or exposureab Control or no exposurecd Totala+cb+d

Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii ). Results for men. Lung cancer casesControls Smokers Non-smokers 227 OR=? Analysis and Interpretation: Odds Ratio

Cigarette smoking and lung cancer (Doll and Hill BMJ 1950 ii ). Results for men. Lung cancer casesControls Smokers Non-smokers 227 Odds ratio = (647x27) / (2x622) = Lung cancer cases 14 x more likely to be smokers. Analysis and Interpretation: Odds Ratio

Analysis and interpretation Relative Risk Sample GroupDiseaseNo DiseaseTotal Treatment or exposure aba+b Control or no exposure cDc+d Totala+cb+d

Analysis and interpretation Sensitivity analysis Overall effect calculated by different methods (fixed vs. random) Reanalysis with exclusion of poor-quality studies Reanalysis with exclusion of small studies Reanalysis of exclusion of studies with short duration of follow-up

Presentation of results Often graphically displayed with confidence intervals Type I and II error should be discussed Robustness of findings/sensitivity should be discussed

Strengths Can summarize from available studies the effects of interventions across many patients Can reveal research designs as moderators of study results Can reduce false negative results Can clarify heterogeneity between study results

Strengths Can assist in accurate calculation of sample size needed in future studies Can suggest promising research questions for future study Can allow more objective assessment of evidence and thereby reduce disagreement

Weaknesses Can pass along inflated estimates of size effects based previously reported results Cannot overcome subjectivity in choice of outcomes and their weighting in analysis Can be compromised by publication bias

Weaknesses Arithmetic nature of meta-analysis can produce false impression of certainty in an inherently uncertain process with many subjective elements

Cochrane Collaboration The Cochrane Collaboration is an international not-for-profit organization, providing information about the effects of health care The Cochrane Collaboration Source of qualitative and quantitative systematic reviews with good methodological rigor

Conclusions Interpret with caution remembering that conclusions depend on the quality of the studies included Findings of subsequent randomized controlled trials may differ

References Malone PM et al. Drug information: a guide for pharmacists. McGraw-Hill. New York. 2 nd edition Noble Jr JH. Meta-analysis: methods, strengths, weaknesses, and political uses. J Lab Clin Med 2006;147:7-20.