1. Univariate Meta-Analysis vs. Meta-Analytic Structural Equation Modeling
Tessa van den Berg, University of Amsterdam Suzanne Jak, University of Amsterdam Machteld Hoeve, University of Amsterdam Veroni Eichelsheim, Netherlands Institute for the Study of Crime and Law Enforcement (NSCR)

2. Meta-Analysis
Integrate research findings  summary effect size (Glass, 1976)
Univariate meta-analysis
1 association between 2 variables
Meta-Analytic Structural Equation Modeling (MASEM)
Combination of meta-analysis and SEM (Viswesvaran & Ones, 1995)
MASEM vs. Univariate Meta-Analysis

3. SEM Investigates set of variables at once Model based on theory
Model fit
Regression coefficients
Controlling for other variables in the model
MASEM
Answers questions that univariate meta-analysis cannot answer
Primary studies don’t need to include all associations
Can answer RQ’s that were not explored before
MASEM vs. Univariate Meta-Analysis

4. Research Question
Differences between univariate meta-analysis and MASEM
Advantages and disadvantages of both methods
MASEM vs. Univariate Meta-Analysis

5. Application
MASEM vs. Univariate Meta-Analysis

6. Project Former meta-analysis
k = 161 (Hoeve, Dubas, Eichelsheim, Van Der Laan, Smeenk, & Gerris, 2009)
Subset:
k = 88
N = 154,176
MASEM vs. Univariate Meta-Analysis

7. Methods Univariate meta-analysis
Summary effect sizes (pooled correlation)
R package metafor
Random effects model
MASEM: Two Stage SEM approach (Cheung, 2014)
R package metaSEM
MASEM vs. Univariate Meta-Analysis

8. Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix
MASEM vs. Univariate Meta-Analysis

9. Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix
MASEM vs. Univariate Meta-Analysis

10. Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix
MASEM vs. Univariate Meta-Analysis

11. Results – Univariate Meta-Analysis
Parental delinquency
Support
k = 6, r = −0.02 [−0.04; 0.01]
Parental delinquency
Authoritarian control
k = 2, r = [0.01; 0.15]
Parental delinquency
Behavioral control
k = 6, r = −0.03 [−0.06; 0.001]
Parental delinquency
Psychological control
k = 1, r = 0.04 [−0.01; 0.09]
Parental delinquency
Indirect parenting
k = 2, r = −0.07 [−0.11; −0.04]
MASEM vs. Univariate Meta-Analysis

12. Results – Univariate Meta-Analysis
Parental delinquency
Support
k = 6, r = −0.02 [−0.04; 0.01]
Parental delinquency
Authoritarian control
k = 2, r = [0.01; 0.15]
Parental delinquency
Behavioral control
k = 6, r = −0.03 [−0.06; 0.001]
Parental delinquency
Psychological control
k = 1, r = 0.04 [−0.01; 0.09]
Parental delinquency
Indirect parenting
k = 2, r = −0.07 [−0.11; −0.04]
MASEM vs. Univariate Meta-Analysis

13. Results – Univariate Meta-Analysis
Parental delinquency
Juvenile delinquency
k = 19, r = 0.19 [0.13; 0.25]
Support
Juvenile delinquency
k = 39, r = −0.15 [−0.19; −0.11]
Authoritarian control
Juvenile delinquency
k = 10, r = 0.15 [0.10; 0.20]
Behavioral control
Juvenile delinquency
k = 39, r = −0.16 [−0.21; −0.12]
Psychological control
Juvenile delinquency
k = 7, r = 0.08 [0.02; 0.14]
Indirect parenting
Juvenile delinquency
k = 25, r = −0.18 [−0.25; −0.11]
MASEM vs. Univariate Meta-Analysis

14. Results – Univariate Meta-Analysis
Parental delinquency
Juvenile delinquency
k = 19, r = 0.19 [0.13; 0.25]
Support
Juvenile delinquency
k = 39, r = −0.15 [−0.19; −0.11]
Authoritarian control
Juvenile delinquency
k = 10, r = 0.15 [0.10; 0.20]
Behavioral control
Juvenile delinquency
k = 39, r = −0.16 [−0.21; −0.12]
Psychological control
Juvenile delinquency
k = 7, r = 0.08 [0.02; 0.14]
Indirect parenting
Juvenile delinquency
k = 25, r = −0.18 [−0.25; −0.11]
MASEM vs. Univariate Meta-Analysis

15. Results – MASEM
k = 88 N = 154,176
MASEM vs. Univariate Meta-Analysis

16. Results – MASEM
k = 88 N = 154,176
MASEM vs. Univariate Meta-Analysis

17. Results – MASEM Indirect effects Support: β = 0.002 [−0.0002; 0.004]
Authoritarian: β = [0.001; 0.019]
Behavioral: β = [−0.0002; 0.009]
Psychological: β = −0.001 [−0.008; 0.004]
Indirect parenting: β = [−0.0002; 0.015]
MASEM vs. Univariate Meta-Analysis

18. Results – Compared
MASEM vs. Univariate Meta-Analysis

19. Results – Compared
MASEM vs. Univariate Meta-Analysis

20. Results – Compared
MASEM vs. Univariate Meta-Analysis

21. Results – Compared
MASEM vs. Univariate Meta-Analysis

22. Moderator Analysis Univariate meta-analysis
All kinds of moderator variables
MASEM vs. Univariate Meta-Analysis

23. Moderator Analysis MASEM
Subgroup analysis (Jak & Cheung, in press)  compare groups of studies
Continuous moderator variable  create subgroups
Loss of information
Group 1
Group 2
MASEM vs. Univariate Meta-Analysis

24. Moderator Analysis Continuous: Age
Univariate: significant effect on some associations
MASEM: no significant effects
Categorical: SES
Univariate: no significant effects
MASEM vs. Univariate Meta-Analysis

25. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

26. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

27. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

28. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

29. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

30. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

31. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

32. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

33. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

34. Conclusion
Different methods  different effect sizes  different conclusions
MASEM
Univariate Meta-Analysis
Set of variables in hypothesized model
2 variables – 1 association
Multiple regression coefficients
Bivariate correlations
Can control for other variables
Cannot control for other variables
Can estimate indirect / mediation effects
Only bivariate relationships
Can answer RQ’s that were not explored before
Only summary effect size of associations that were examined before
Multivariate analysis
Univariate analysis
Moderator analysis with only categorical variables
Moderator analysis with all kinds of variables
Rather one moderator at a time
Several moderators at once
MASEM vs. Univariate Meta-Analysis

35. References
Cheung, M. W. L. (2014). Fixed-and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8. doi: / X Hoeve, M., Dubas, J. S., Eichelsheim, V. I., Van Der Laan, P. H., Smeenk, W., & Gerris, J. R. (2009). The relationship between parenting and delinquency: A meta-analysis. Journal of Abnormal Child Psychology, 37, Jak, S. (2015). Meta-Analytic Structural Equation Modeling. Springer International Publishing. doi: / Jak, S., & Cheung, M. W.-L. (in press). Testing moderator hypotheses in meta-analytic structural equation modeling using subgroup analysis. Behavior Research Methods. Viswesvaran, C., & Ones, D. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, doi: /j tb01784.x
MASEM vs. Univariate Meta-Analysis