Lecture 9 EPSY 642 Meta Analysis Fall 2009 Victor L. Willson, Instructor

Current Issues Multi-level models: Raudenbush & Bryk analysis in HLM6 Structural equation modeling in meta-analysis Clustering of effects: cluster analysis vs. latent class modeling Multiple studies by same authors- how to treat (beyond ignoring follow-on studies), the study dependence problem Multiple meta-analyses: consecutive, overlapping Multiple outcomes per study

Multilevel Models Raudenbush & Bryk HLM 6 One effect per study Two level model, mediators and moderators at the second level Known variance for first level (w i ) Mixed model analysis: requires 30+ studies for reasonable estimation, per power analysis Maximum likelihood estimation of effects

Multilevel Models Model: Level 1:g i = g i + e i where there is one effect g per study i Level 2: g i =  0 +  1 W + u i where W is a study-level predictor such as design in our earlier example Assumption: the variance of g i is known = w i

Structural Equation Modeling in SEM New area- early work in progress: Cheung & Chan (2005, Psych Methods), (2009, Struc Eqn Modeling)- 2-step approach using correlation matrices (variables with different scales) or covariance matrices (variables measured on the same scale/scaling) Stage 1: create pooled correlation (covariance) matrix Stage 2: fit SEM model to Stage 1 result

Structural Equation Modeling in SEM Pooling correlation matrices: Get average r: r mean(jk) =  w i r iij /  w ijk I i where j and k are the subscripts for the correlation between variables j and k, where i is the ith data set being pooled Cheung & Chan propose transforming all r’s to Fisher Z- statistics and computing above in Z If using Z, then the SE for Zi is (1-r 2 )/n ½ and

Structural Equation Modeling in SEM Pooling correlation matrices: for each study, COV g (r ij, r kl ) = [.5r ij rkl (r 2 ik + r 2 il + r 2 jk + r 2 jl ) + r ik *r jl + r il *r jk – (r ij *r ik *r il + r ji *r jk *r jl + r ki *r kj *r kl + r li *r lj *r lk )]/n Let  i = covariance matrix for study i, G = {0,1} matrix that selects a particular correlation for examination, Then G = [ |G 1 |’ G 2 |’…| G k |’]’ and  = diag [  1,  2, …  k]

Structural Equation Modeling in SEM Beretvas & Furlow (2006) recommended transformations of the variances and covariances: SDrtrans = log(s) + 1/(2(n-1) COV(r i,r j )trans = r 2 ij /(2(n-1)) The transformed covariance matrices for each study are then stacked as earlier

Clustering of effects: cluster analysis vs. latent class modeling Suppose Q is significant. This implies some subset of effects is not equal to some other subset Cluster analysis uses study-level variables to empirically cluster the effects into either overlapping or non-overlapping subsets Latent class analysis uses mixture modeling to group into a specified # of classes Neither is fully theoretically developed- existing theory is used, not clear how well they work

Multiple studies by same authors- how to treat (beyond ignoring follow-on studies), the study dependence problem Example: in storybook telling literature, Zevenberge, Whitehurst, & Zevenbergen (2003) was a subset of Whitehurst, Zevenbergen, Crone, Schultz, Velging, & Fischel (1999), which was a subset of Whitehurst, Arnold, Epstein, Angell, Smith, & Fischel (1994) Should 1999 and 2003 be excluded, or included with adjustments to 1994? Problem is similar to ANOVA: omnibus vs. contrasts Currently, most people exclude later subset articles

Multiple meta-analyses: consecutive, overlapping The problem of consecutive meta-analyses is now arising: Follow-ons typically time-limited (after last m-a) Some m-a’s partially overlap others: how should they be compared/integrated/evaluated? Are there statistical methods, such as the correlational approach detailed above, that might include partial dependence? Can time-relatedness be a predictor? Willson (1985)

Multiple Outcomes per study Multilevel Approach to study dependency: Requires assumption of homogeneous error variance across studies Variation within “cluster” (study) is the same for all studies If above is reasonable, MLM may be a reasonable model Weight function acts as “sample size” equivalent for second level analysis- use weighted average W 1/2 or N for the effects within the study (unclear which will be more appropriate here) If all effects within-study have the same sample size(s) the W for all are equal to each other

Multiple Outcomes per study Study effect e Effect  Indep Var W 1/2

CONCLUSIONS Meta-analysis continues to evolve Focus in future on complex modeling of outcomes (SEM, for example) More work on integration of qualitative studies with meta-analysis findings