Systematic Review Module 10: Quantitative Synthesis II Thomas Trikalinos, MD, PhD Joseph Lau, MD Tufts EPC

CER Process Overview

Learning objectives of this module Dealing with between-study heterogeneity Dealing with between-study heterogeneity Promise and danger of subgroup analyses Promise and danger of subgroup analyses Meta-regression Meta-regression Control rate meta-regression Control rate meta-regression

Homogeneity From Cochrane Database Syst Rev. 2000;(2):CD000505

Heterogeneity: Patellar resurfacing in total knee arthroplasty for pain J Bone Joint Surg Am. 2005;87(7):

Heterogeneity Diversity of studies in a meta-analysis Diversity of studies in a meta-analysis Typically abundant Typically abundant Arguably the most important role of meta- analytic methodologies is to quantify, explore, and explain between-study heterogeneity Arguably the most important role of meta- analytic methodologies is to quantify, explore, and explain between-study heterogeneity

Heterogeneity Methodological heterogeneity Pertains to specifics of study design and analysis (e.g., type of study, length of follow-up, proportion of dropouts and handling thereof) Pertains to specifics of study design and analysis (e.g., type of study, length of follow-up, proportion of dropouts and handling thereof) Clinical heterogeneity Pertains to differences in the populations, intervention and co-interventions, outcomes Pertains to differences in the populations, intervention and co-interventions, outcomes

Statistical heterogeneity Statistical heterogeneity exists when the results of the individual studies are not consistent among themselves Clinical heterogeneity Methodological heterogeneity Biases Chance Statistical heterogeneity

Clinical vs. statistical heterogeneity Clinical and methodological heterogeneity is abundant. Our aim is to explore it, and use these observations to formulate interesting hypotheses. Clinical and methodological heterogeneity is abundant. Our aim is to explore it, and use these observations to formulate interesting hypotheses. Often, but not always, clinical and methodological heterogeneity will result in a statistically significant test Often, but not always, clinical and methodological heterogeneity will result in a statistically significant test Chance, technical issues or biases may result in statistically significant results in heterogeneity tests Chance, technical issues or biases may result in statistically significant results in heterogeneity tests

META-REGRESSION modeling summary data OVERALL ESTIMATE combining summary data RESPONSE SURFACE modeling individual patient data SUBGROUP ANALYSES differentiating effects in subgroups

Promises of subgroup analyses

J Am Coll Card 1990

Mortality of thrombolytic therapy for AMI meantime to treatment (0-3 hours)

Mortality of thrombolytic therapy for AMI meantime to treatment (3.1-5 hours)

Mortality of thrombolytic therapy for AMI meantime to treatment ( hours)

Mortality of thrombolytic therapy for AMI meantime to treatment (> 10 hours)

Vit E and all cause mortality Ann Intern Med. 2005;142(1):37-46.

Hazards of subgroup analyses

From Fibrinolytic Therapy Trialists Collaborative Group: Indications for Fibrinolytic Therapy Lancet 343: 311,1994

ISIS-2. Lancet 1988;ii: Subgroup analyses

Beyond subgroup analyses: meta-regression

Subgroup analysis Ann Intern Med. 2005;142(1):37-46.

Univariate meta-regression Ann Intern Med. 2005;142(1):37-46.

Meta-regression: Zidovudine monotherapy vs. placebo ~τ 2

Multivariate meta-regression: Effect of Soy on LDL Dose Baseline LDL

Control Rate Meta-Regression Single covariate included is event rate in the control group (control rate) Single covariate included is event rate in the control group (control rate) – Control rate is surrogate for all baseline differences between the studies, in terms of baseline risk for the event of interest. – Can show that underlying risk of event (severity of illness) may explain differences in the treatment effect across studies

Control rate meta-regression in the streptokinase example Stat Med. 1998;17(17):

Two types of covariates in meta- regressions Study level covariates vs. participant level covariates Study level: presence/absence of blinding, intervention dose (in experimental studies) Study level: presence/absence of blinding, intervention dose (in experimental studies) Participant level: mean age, proportion of diabetics, mean intake of vitamin D (in observational studies) Participant level: mean age, proportion of diabetics, mean intake of vitamin D (in observational studies)

Spurious associations in meta- regressions and subgroup analyses Meta-regressions that use participant-level covariates can mislead, as they are susceptible to ecological fallacy Associations of treatment effect and participant-level covariates should be interpreted with caution See the quiz

Summary Subgroup analyses, meta-regressions and control-rate meta-regressions are tools to explore between-study heterogeneity. Do use them to understand your data. Subgroup analyses, meta-regressions and control-rate meta-regressions are tools to explore between-study heterogeneity. Do use them to understand your data. They are mostly hypothesis forming tools. Especially for meta-regressions on patient-level covariates, ecological fallacy may mislead. They are mostly hypothesis forming tools. Especially for meta-regressions on patient-level covariates, ecological fallacy may mislead. Beware when interpreting their results. Beware when interpreting their results.