Moderators

Definition Moderator - A third variable that conditions the relations of two other variables Example: SAT-Quant and math grades in school Correlation for females greater than for males Sex is a moderator of the relations between SAT and grades

Types of Moderators Categorical or nominal Analogous to factors in ANOVA Sex (F v M), race, study type (published vs. dissertation vs. not) Analyzed by analog to ANOVA Continuous Time between test & retest, age of participants, number of visits or duration of therapy Analyzed by weighted regression (meta-regression) Handled as weighted GLM (no correlated errors)

Types of Analysis Fixed (common effects) model The moderator(s) are expected to account for all systematic variance in effect sizes. One analysis only. Mixed model The moderators(s) account for some, but not all, variance in effect sizes. Some REVC left over. Recalculate weights for residual REVC and re- estimate conditional means. Multiple ways to consider error and weights

Number of Independent Variables In theory, a meta-analysis model can contain both continuous and categorical moderator variables. In theory, a model can contain an unlimited number of independent variables – statistical control for IVs In practice, independent variables are usually modeled one at a time, and there are often only a few IVs There are problems with missing data and capitalization on chance. Often problems with both Type I and Type II errors

Categorical Example Correlation of SAT and Grades for Males (studies 1-3) and Females (studies 4-6)

Hypothetical SAT data StudyrzNwSex M M M F F F What is the correlation overall? Is it different for males and females?

Studyzww**2w*z w*(z-zbar)** Sum

Male Subset Studyzww**2w*z w*(z-zbar)** Sum

Female Subset Studyzww**2w*z w*(z-zbar)** Sum

Test of Moderator TotalMaleFemale Zbar Q The test of the moderator is the test of Q B. The test has df = (Number groups –1). Here df=1, Q B =18.99, p<.05.

Group Means GroupSum (w) SE zbar CI 95L CI 95U rbarCI 95L CI 95U Male Female Total Note that the difference in means is.25, which is quite large (and hypothetical; fictional data).

Mixed Model In the previous example, the moderators accounted for all the variance in the effect sizes (excepting sampling error). Suppose there was remaining variance, e.g., Q W = 6.

Mixed Model (2) Studyzww2w2*z Sum In this example, it makes little difference whether fixed or mixed. Sometimes it matters.

Analog to ANOVA Review the Excel and R analog to ANOVA programs. Software note: Avoid interpreting Q tests from the random- effects models if the REVC weights were applied. The values of Q only conform to the chi-square statistic when the fixed weights are used. Test the moderator with fixed effects. If you want random effects (usually you do) estimate the means with pooled or separate estimates of REVC, but do not interpret the resulting values of Q. Interpret the confidence and prediction intervals.

Input File zw VSex

Results from R (1) The overall mean is estimated to be.58 Lots of variability in these data. Test for heterogeneity is significant. Overall mean with no moderators (sex is not part of the analysis). My results are slightly different because of rounding z for input.

Results from R (2) Sex is fixed, study is random. Little residual heterogeneity This is the test of the moderator. In this case, M vs. F difference. The estimate in the intercept is the mean for males. The second estimate is the DIFFERENCE between males and females.

Results from R (3) Intercept is suppressed. The Male and Female means are estimated. This is a test of the simultaneous effects – the joint test of both male and female means being zero. rma doesn’t allow separate REVC estimates

Subsets R command ‘subset’ allows you to partition data for separate analysis Suppose we want to run a meta-analysis solely on females.

Analog to F In ANOVA (primary data analysis), we would compute an overall F regardless of the number of factors. With meta-analysis, we substitute a chi-square test. As you saw in the example, metafor tests different things when you include or suppress the intercept {mods = ~factor(Sex) vs. mods = ~factor(Sex)-1} Be sure to test what you want and to interpret your results correctly In the particular case, we probably are more interested in whether there is a difference between men and women than whether the joint test of slopes is zero.

Class Exercise Find the Excel file DevineR. It contains 54 studies comparing length of stay in the hospital for a control group and an educational treatment group given information about the benefits of going home. Positive d means the treatment appears helpful. The moderator is published article (J for journal) versus dissertation (D for dissertation). Analyze Import the data to R. Compute the overall mean effect size. Test for the moderator (difference in means). Also look at variability within each group. Prepare to share your findings with the group.

Weighted Regression OLS regression Assume equal error variances (homoscedasticity) Estimate magnitude of error, minimize SSE Weighted regression Error variances assumed known Error variances are unequal In meta-analysis, we know the sampling (error) variances, so can use weighted regression Minimize weighted SSE

Weighted Regression Defined OLS WLS Assume uncorrelated errors not all equal to one another. If c jj is S.E. 2, the jth diagonal element of If c jj is the jth diagonal element of p is the number of IVs

Hypothetical SAT-Q and pct quant courses in GPA StudyrzNwPct Q Does the percentage of quantitative courses influence the size of the correlation between SAT- Q and GPA?

SAT-Q Matrices (1) Xy X’X’ V

SAT-Q Matrices (2) V Intercept Slope

SAT-Q (3) S.E. = sqrt(c jj ) S. E. t (or z) = b/S.E t Intercept Slope Intercept Slope Intercept Slope

Mixed Model In the event that homogeneity analysis reveals a large Q value for the residual, you can use the Q value to estimate the residual REVC. The REVC can be used to recalculate the weights for estimating specific values. Generally, however, researchers use weighted regression to compute significance tests for continuous moderators.

Weighted Regression in R Review the weighted regression program in R. Run the PowerPoint example. Run an example with 2 independent variables and discuss output. R will produce a mixed model by default (method = “REML” or “DL”). If you want a fixed-effects model (that assumes all REVC is accounted for by the moderator(s), then request a common effects model (method = “FE”).

Input Data zwvPctQ

Results from R My results are slightly different because of rounding z for input. Test of percent of quant classes. This would be test of all coefficients if we had more. Regression estimates and test of each coefficient. Moderator appears to account for all REVC.

Example 2 Study of relations between LMX (quality of Leader- Member eXchange and other variables. In this sheet, only affective organizational commitment (AC) is included. Correlation between LMX and AC Country classification as individualistic (Western) or collectivistic (Eastern) culture by country by Hofstede Reliability of measurement of AC in the sample

R code & Results Here we have a model analogous to Analysis of Covariance. Culture is a categorical variable, and alpha is a continuous variable. Lots of random variance. No impact of moderators.

What if… No effect for alpha by itself.

What else? Culture is not significant by itself

Exercise Download the file Rockstuhl2012LMX.xlsx Select those studies that are for job satisfaction (variable = JS); delete the rest (Excel: data sort, select, delete; or subset in R) Compute z and v (or use n, r, and ZCOR in R) Test for moderators Culture and Alpha (reliability) Prepare to share your results