  Meta-Analysis of Correlated Data

Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same control Phylogenetic non-independence Measurements of multiple responses to a common treatment Unknown correlations…

Multiple Sample Points per Study! StudyExperiment in StudyHedges DV Hedges D Ramos & Pinto Ramos & Pinto Ramos & Pinto Ellner & Vadas Ellner & Vadas Moria & Melian

Hierarchical Models Study-level random effect Study-level variation in coefficients Covariates at experiment and study level

Hierarchical Models Random variation within study (j) and between studies (i) T ij  ij,  ij 2 )  ij  j,  j 2   j ,  2 

Study Level Clustering

Hierarchical Partitioning of One Study Grand Mean Study Mean Variation due to  Variation due to 

Example: Data Set 1 Group Effect Variance 1 A A A A B B B C

A Two-Step Solution T ij  ij,  ij 2 )  ij  j,  j 2   j ,  2  library(plyr) data1_study <- ddply(data1,.(Group), function(adf){ mod <- rma(Effect, Variance, data=adf) cbind(theta_j = coef(mod), se_theta_j = coef(summary(mod))[1,2], omega2 = mod$tau2) })

A Two-Step Solution T ij  ij,  ij 2 )  ij  j,  j 2   j ,  2  > data1_study Group theta_j se_theta_j omega2 1 A B C D E jj jj

A Two-Step Solution T ij  ij,  ij 2 )  ij  j,  j 2   j ,  2  > rma(theta_j, I(se_theta_j^2), data=data1_study) Random-Effects Model (k = 5; tau^2 estimator: REML) tau^2 (estimated amount of total heterogeneity): (SE = )... estimate se zval pval ci.lb ci.ub < *** 22 

Multiple Effects per Research Group

Solutions to Multiple Hierarchies Multiple-Step Meta-analyses Multi-level hierarchical model fits –Better estimator –Accommodates more complex data structures –May need to go Bayesian (don't be scared!) Model correlation…

Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same control Phylogenetic non-independence Measurements of multiple responses to a common treatment Unknown correlations…

Multiple Effect Sizes with Common Control Effect of each treatment calculated using same control!

The Control Keeps Showing Up! n c and sd c are going to be the same for all treatments Effect sizes will covary

Calculating Covariance Formulae available or derivable for all effect sizes

A Mixed Effect Group Model Group means, random study effect, and then everything else is error T i  im,  i 2 ) where  im  m,  2 

A Mixed Effect Group Model Group means, random study effect, and then everything else is error T i  MVN  i,  i ) where  i  MVN  X i ,  2 

What are  i and  i ? i =i=i =i= T i  MVN  i,  i )

What about the treatment effects? X i =   i =  i  MVN  X i ,  2 

What if treatments are correlated?  i = T i  MVN  i,  i )

Why does covariance matter?   x-y =   x +   y + 2   x,y In asking if two treatments differ, cov helps tighten confidence intervals High cov  more weight for a study as treatments share information

Multiple Treatments study trt m1i m2i sdpi n1i n2i Common Control!

Calculating the Variance/Covariance Matrix [,1] [,2] [,3] [,4] [,5] [,6] [1,] [2,] [3,] [4,] [5,] [6,]

Fitting a Model with a VCOV Matrix > rma.mv(yi ~ factor(trt)-1, V, random =~ 1|study, data=dat)

Comparison to No Correlation Model With correlation estimate se zval pval ci.lb ci.ub factor(trt) < factor(trt) < Without correlation estimate se zval pval ci.lb ci.ub factor(trt) < factor(trt) <

Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same control Phylogenetic non-independence Measurements of multiple responses to a common treatment Unknown correlations…

Effect Size on Related Organisms Not Independent Warming on Litterfall Pine Trees Redwoods Fir Trees Oak Trees {

Phylogenetic Distances Determines Covariances for Weights

What about Multiple Studies of Some Species?

Common Forms of Dependence Multiple effects per study –Or per research group! Multiple effect sizes using same control Phylogenetic non-independence Measurements of multiple responses to a common treatment Unknown correlations…

Common Treatments Treatment Response 1Response 2Response 3

Common Treatments CO 2 CO 2 Assimilation GS Stomatal Conductance PN

Correlation Between Responses

What does Correlation between effects mean? X i =   i =  i  MVN  X i ,  2 

What Do We Do? 1. Create a 'composite' measure –Average –Weighted Average 2. Estimate different coefficients directly 3. Robust Variance Estimation (RVE)

The CO 2 Effect Data experiment Paper Measurement Hedges Var GS PN GS PN GS PN GS PN GS PN

Direct Estimation rma.mv(Hedges ~ Measurement, Var, random =~ Measurement|Paper, data=co2data, struct="HCS")

 and Different Correlation Structures Different structures for different data We do not always know which one is correct!

Estimates of Variance, Covariance Multivariate Meta-Analysis Model (k = 68; method: REML) Variance Components: outer factor: Paper (nlvls = 18) inner factor: Measurement (nlvls = 2) estim sqrt k.lvl fixed level tau^ no GS tau^ no PN rho no

Disadvantages to Multivariate Meta-Analysis 1. Difficult to estimate with few studies 2. Additional assumptions of covariance structure 3. Often little improvement over univariate meta-analysis 4. Publication bias exacerbated if data not missing at random Jackson et al Satist. Med.

Robust Variance Estimation Essentially, bound weights within a group j to 1/mean var j and assume a value of  –Test sensitivity to choice of  –Correct DF for small sample sizes Methods developed by Hedges, Tipton, and others robumeta package in R

robumeta & RVE library(robumeta) robu(Hedges ~ Measurement, data=co2data, studynum=Paper, var.eff.size=Var)

RVE: Correlated Effects Model with Small-Sample Corrections Model: Hedges ~ Measurement Number of studies = 18 Number of outcomes = 68 (min = 2, mean = 3.78, median = 4, max = 10 ) Rho = 0.8 I2 = Tau.Sq = Struct="CS" only so far

Often, Choice of  Matters Little > sensitivity(co2modRVE) Type Variable rho=0 rho=0.2 rho=0.4 rho=0.6 rho=0.8 rho=1 1 Estimate intercept MeasurementPN Std. Err. intercept MeasurementPN Tau.Sq

Results May Differ… Multivariate Meta-Analysis Model Results: estimate se zval pval ci.lb ci.ub intrcpt MeasurementPN * Robust Variance Estimation Model Results: Estimate StdErr t-value df P(|t|>) 95% CI.L 95% CI.U Sig 1 intercept MeasurementPN

Other Sources of Unknown Correlations Shared system types Shared environmental events Labs or investigators Re-sampling experiments Experiments repeated in a region More…

Why Model Correlation instead of Hierarchy? Depends on question Analytical difficulty Leveraging correlation to aid with missing data