INTRODUCTION TO META- ANALYSIS

Meta-analysis  A statistical approach (after systematic review) to compare and combine effect sizes from a pool of independent effect sizes.  After which, conclusions on the overall effect sizes, variability of the effect sizes and potential study characteristics that influences the effect sizes may be drawn.

Key Steps 1. Identify relevant studies 2. Extract and code the studies 3. Convert reported statistics into appropriate effect sizes 4. Conduct statistical analysis 5. Interpret and report findings according to standards

Effect Sizes

What are effect sizes (y i ) ?  Effect sizes are quantitative indices that measure the strength of the effect in individual studies.  A good effect size should be able to:  Indicate the direction and strength of the effect being studied  Be comparable across studies.  Sampling variances (v i ) of the effect sizes are also required since studies are usually weighted by the precision of the estimated effect sizes.

How do we get the effect sizes (y i ) ?  Authors might have reported them in their studies.  Summary statistics reported such as means, standard deviations, proportions as well as t and F statistics could be used to calculate the effect sizes.  More than one type of effect size can be applied in any particular research question.

Common Effect Sizes

Proportion

Relative Risk (RR), Odds Ratio(OR)

Raw Mean Difference (RMD), Standardised Mean Difference (SMD)

Correlation (r), Fisher’s z Transformed Score (z)

Distribution of effect sizes (y i )  Assumed to be normally distributed with known variances  Accuracy of normal approximation depends on several factors such as type of effect size, population value of the effect size, and sample size of the study.  Appropriate transformation may have to be applied to the effect sizes to make the sampling distribution closer to a normal distribution.

Statistical Models for Meta-Analysis

Fixed-effects model

Fixed-effects model – Weighted least squares

Random-effects model

Mixed-effects model  Using study characteristics to predict variation in population effect sizes.  Known as mixed-effects meta-analysis or meta- regression.  R 2 index can be calculated to indicate the percentage of variance explained by the predictors.

Illustration with Real Data Set using

Introduction to  A programming language/environment for data processing, statistical computing, and graphics  Free & open-source  R (  Cross-platform (UNIX/Linux, Windows, MacOS, …)  Command-driven & object-oriented  User community & packages (8000+)

Binary Outcome

Extraction of Data  Create a Excel spread sheet  For each study, create the following columns: 1. Study ID 2. Author –Year published 3. Number of participants who received intervention 4. Number of participants who were in control arm 5. Number who developed outcomes in intervention 6. Number who developed outcomes in control arm

Example 1: Mak et al. study  Objective: To compare the incidents of atrial fibrillation among bisphosphonate and non- bisphophonate users  Atria fibrillation (Afib) – an irregular and often very fast heart rate.  Bisphosphonates – anti-resorptive medicines, slow or stop the natural process that dissolves bone tissues, resulting in maintained or increased bone density and strength.

 Created six column trial: Study (Study ID), Type (Type of Intervention), AF.BP (number who developed outcome in intervention), Tot.BP (Total of participant who received intervention), AF.non.BP (Number who developed outcome in intervention), Tot.non.BP ( Total of participant who received intervention) Example 1: Mak et al. study

Example 1: Import Excel file using R package Command: 1.install.packages(“xlsx") 2.require(xlsx) 3.data1 = read.xlsx("C:\\journalclub\\data1.xlsx", sheetName=‘data1’) Command: 1.install.packages(“xlsx") 2.require(xlsx) 3.data1 = read.xlsx("C:\\journalclub\\data1.xlsx", sheetName=‘data1’)

Command 1: Fixed-Effects Model Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.MH(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.MH(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) model.FE <- meta.MH(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Name of the output file Name of the test (Mantel-Haenszel test) for fixed-effects models List of variablesPrint the study ID Name of the dataset

Results 1: Fixed-Effects Model Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.MH(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.MH(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1)

Command 2: Random-Effects Model Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.DSL(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Command: 1. install.packages(“rmeta") 2. require(rmeta) 3. model.FE <- meta.DSL(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) model.RE <- meta.DSL(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Name of the output file Name of the test (DerSimonian & Laird test) for random-effects models List of variablesPrint Name of the dataset

Results 2: Random-Effects Model Command: 1.require(rmeta) 2.model.RE <- meta.DSL(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1) Command: 1.require(rmeta) 2.model.RE <- meta.DSL(Tot.BP,Tot.non.BP,AF.BP,AF.non.BP, names=Study, data=data1)

Key results from meta-analysis  Effect Size (Odds Ratio)  Variance with 95% Confidence Interval  Test of heterogeneity  Plots  Forest Plot  Funnel Plot

Summary Estimates for Mak Study 1.Effect size OR= % Confidence Interval [1.13, 1.27] 3.Test of Heterogeneity:  DerSimonian & Laird Test Chi-square (df=7) = 7.16 P-Value =  Woolf’s Test Chi-square (df=7) = 7.37 P-Value = The pooled Odds Ratio shows that the risk of atrial fibrillation is 20% higher among bisphosphonate than non- bisphosphonate users In 95 out of 100 such meta analyses, the pooled Odds Ratio would lie between 1.13 and 1.27, indicating a statistically significant protective effect These studies were not significantly heterogeneous

Command 3: Test of heterogeneity Command: 1.require(meta) 2. homogeneity.results <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data1,sm="OR", method.tau="DL") Command: 1.require(meta) 2. homogeneity.results <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data1,sm="OR", method.tau="DL") List of variables To analyze type of variable. 1.metabin = binary 2.metacon = continuous 3.metagen = log hazard ratios Type of summary measure (“OR”, “RR”, “RD”, “ASD” “HR”, “MD”) To estimate the between study variance (“DL”, “PM”, “REML”, “ML”, “HS”, “SJ”, “HE”, or “EB”)

Results 3: Test of heterogeneity Command: 1. require(meta) 2. homogeneity.results <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data1,sm="OR",method.tau="DL") 3. summary(homogeneity.results) Command: 1. require(meta) 2. homogeneity.results <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data1,sm="OR",method.tau="DL") 3. summary(homogeneity.results)

Command 4: Mixed-effects meta- analysis Command: 1.require(meta) 2.Require(data2) 3.data2 4. meta2 <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data2, sm="OR", method=“MH”, method.tau="DL") 5. metareg.age <- metareg(meta2, age) Command: 1.require(meta) 2.Require(data2) 3.data2 4. meta2 <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data2, sm="OR", method=“MH”, method.tau="DL") 5. metareg.age <- metareg(meta2, age) Meta regression Original results derived from ‘metabin’ Predictor

Results 4: Mixed-effects meta- analysis Command: 1. require(meta) 2. meta.reg.age <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data2,sm="OR", method=“MH”, method.tau="DL") 3. summary(homogeneity.results) Command: 1. require(meta) 2. meta.reg.age <- metabin(AF.BP,Tot.BP,AF.non.BP,Tot.non.BP,data=data2,sm="OR", method=“MH”, method.tau="DL") 3. summary(homogeneity.results)

Other important commands not discussed Command: 1.require(meta) 2.Data(Fleiss93) Command: 1.require(meta) 2.Data(Fleiss93) metacont(n.e, mean.e, sd.e, n.c, mean.c, sd.c, data=Fleiss93cont, sm="SMD", studlab=study) “metacont”: Meta-analysis of continuous outcome metacor(c(0.85, 0.7, 0.95), c(20, 40, 10)) “metacor”: Meta-analysis of correlations metacr(Fleiss93cont) “metacr”: Meta-analysis of outcome data metaprop(4:1, c(10, 20, 30, 40)) “metaprop”: Meta-analysis of single proportions

Forest and Funnel Plots

How to read Forest Plot The dotted line passes across null, or 1.0 The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the line The diamond is the summary estimate The two ends of the diamond indicate 95% CI The dotted line passes across null, or 1.0 The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the line The diamond is the summary estimate The two ends of the diamond indicate 95% CI The size of the black square box indicates weight of the study

Figure 1: Forest plot for Mak study Command: 1. plot(model.FE) Command: 1. plot(model.FE)

Forest plot for Mak study Command: 1. require(meta) 2. meta1 <- metabin(AF.BP, Tot.BP, AF.non.BP, Tot.non.BP, data=data1) 3. forest(meta1) Command: 1. require(meta) 2. meta1 <- metabin(AF.BP, Tot.BP, AF.non.BP, Tot.non.BP, data=data1) 3. forest(meta1)

How to read Funnel Plot Plots the effect size against the sample size of the study To study a funnel plot, look at its LOWER LEFT corner, that’s where negative or null studies are located If EMPTY, this indicates “PUBLICATION BIAS” Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias Plots the effect size against the sample size of the study To study a funnel plot, look at its LOWER LEFT corner, that’s where negative or null studies are located If EMPTY, this indicates “PUBLICATION BIAS” Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias

Figure 2: Funnel Plot for Mak Study Command: 1. plot(model.FE) Command: 1. plot(model.FE)

Fixed Effects or Random Effects Model? Fixed Effects ModelRandom Effects Model Conduct if it is reasonable to assume underlying Rx effect is SAME for all studies Conduct if test of heterogeneity is significant (shows heterogeneity) Test: test of heterogeneity. If significant, go for random effects model Assume that TRUE log odds ratio comes from a normal distribution Method: Mantel Haenszel method of calculating OR Method: DerSimonian Lair’s method (DSL) of calculating OR OR=1.2 [1.13,1.27OR=1.2 [1.12,1.27]

Summary  Outlined basic steps  Information retrieval  Data Abstraction  Data Analysis  Model Selection: Fixed Effects or Random effects  Outlined some issues and listed software

Issues in meta-analysis  Choosing a model  Fixed effects model or random effects?  Bias in meta-analysis  Poor quality of trials  Publication bias  Quality control in meta-analysis  QUOROM guidelines  Statistical Software for meta-analysis

Bias in Meta-analysis  Poor Quality of Trials  To avoid them, learn more at CONSORT statement [  Publication Bias  Study showing beneficial effects of new treatment more likely to be published than one showing no effect  Negative trials assumed to contribute less; never show up in the literature base  Use several approaches to avoid this  Use Funnel Plots to examine the influence of publication bias

Statistical Software for Meta-analysis  Free Software: 1. “rmeta”, “meta” or “metafor” package in R for statistical computing 2. EpiMeta: from Epi Info 3. Revman: from Cochrane Collaboration  Commercial Software 1. Comprehensive Meta-Analysis 2. meta module in STATA 3. MLviN 4. HLM 5. MPLUS  Other list of softwares: analysis-software-sites/

Thank you!