Presentation is loading. Please wait.

Presentation is loading. Please wait.

Univariate Meta-Analysis vs. Meta-Analytic Structural Equation Modeling Tessa van den Berg, University of Amsterdam Suzanne Jak, University of Amsterdam.

Similar presentations


Presentation on theme: "Univariate Meta-Analysis vs. Meta-Analytic Structural Equation Modeling Tessa van den Berg, University of Amsterdam Suzanne Jak, University of Amsterdam."— Presentation transcript:

1 Univariate Meta-Analysis vs. Meta-Analytic Structural Equation Modeling
Tessa van den Berg, University of Amsterdam Suzanne Jak, University of Amsterdam Machteld Hoeve, University of Amsterdam Veroni Eichelsheim, Netherlands Institute for the Study of Crime and Law Enforcement (NSCR)

2 Meta-Analysis Integrate research findings  summary effect size (Glass, 1976) Univariate meta-analysis 1 association between 2 variables Meta-Analytic Structural Equation Modeling (MASEM) Combination of meta-analysis and SEM (Viswesvaran & Ones, 1995) MASEM vs. Univariate Meta-Analysis

3 SEM Investigates set of variables at once Model based on theory
Model fit Regression coefficients Controlling for other variables in the model MASEM Answers questions that univariate meta-analysis cannot answer Primary studies don’t need to include all associations Can answer RQ’s that were not explored before MASEM vs. Univariate Meta-Analysis

4 Research Question Differences between univariate meta-analysis and MASEM Advantages and disadvantages of both methods MASEM vs. Univariate Meta-Analysis

5 Application MASEM vs. Univariate Meta-Analysis

6 Project Former meta-analysis
k = 161 (Hoeve, Dubas, Eichelsheim, Van Der Laan, Smeenk, & Gerris, 2009) Subset: k = 88 N = 154,176 MASEM vs. Univariate Meta-Analysis

7 Methods Univariate meta-analysis
Summary effect sizes (pooled correlation) R package metafor Random effects model MASEM: Two Stage SEM approach (Cheung, 2014) R package metaSEM MASEM vs. Univariate Meta-Analysis

8 Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix MASEM vs. Univariate Meta-Analysis

9 Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix MASEM vs. Univariate Meta-Analysis

10 Methods MASEM Step 1: Pooled correlation matrix
Step 2: Fit hypothesized model to matrix MASEM vs. Univariate Meta-Analysis

11 Results – Univariate Meta-Analysis
Parental delinquency Support k = 6, r = −0.02 [−0.04; 0.01] Parental delinquency Authoritarian control k = 2, r = [0.01; 0.15] Parental delinquency Behavioral control k = 6, r = −0.03 [−0.06; 0.001] Parental delinquency Psychological control k = 1, r = 0.04 [−0.01; 0.09] Parental delinquency Indirect parenting k = 2, r = −0.07 [−0.11; −0.04] MASEM vs. Univariate Meta-Analysis

12 Results – Univariate Meta-Analysis
Parental delinquency Support k = 6, r = −0.02 [−0.04; 0.01] Parental delinquency Authoritarian control k = 2, r = [0.01; 0.15] Parental delinquency Behavioral control k = 6, r = −0.03 [−0.06; 0.001] Parental delinquency Psychological control k = 1, r = 0.04 [−0.01; 0.09] Parental delinquency Indirect parenting k = 2, r = −0.07 [−0.11; −0.04] MASEM vs. Univariate Meta-Analysis

13 Results – Univariate Meta-Analysis
Parental delinquency Juvenile delinquency k = 19, r = 0.19 [0.13; 0.25] Support Juvenile delinquency k = 39, r = −0.15 [−0.19; −0.11] Authoritarian control Juvenile delinquency k = 10, r = 0.15 [0.10; 0.20] Behavioral control Juvenile delinquency k = 39, r = −0.16 [−0.21; −0.12] Psychological control Juvenile delinquency k = 7, r = 0.08 [0.02; 0.14] Indirect parenting Juvenile delinquency k = 25, r = −0.18 [−0.25; −0.11] MASEM vs. Univariate Meta-Analysis

14 Results – Univariate Meta-Analysis
Parental delinquency Juvenile delinquency k = 19, r = 0.19 [0.13; 0.25] Support Juvenile delinquency k = 39, r = −0.15 [−0.19; −0.11] Authoritarian control Juvenile delinquency k = 10, r = 0.15 [0.10; 0.20] Behavioral control Juvenile delinquency k = 39, r = −0.16 [−0.21; −0.12] Psychological control Juvenile delinquency k = 7, r = 0.08 [0.02; 0.14] Indirect parenting Juvenile delinquency k = 25, r = −0.18 [−0.25; −0.11] MASEM vs. Univariate Meta-Analysis

15 Results – MASEM k = 88 N = 154,176 MASEM vs. Univariate Meta-Analysis

16 Results – MASEM k = 88 N = 154,176 MASEM vs. Univariate Meta-Analysis

17 Results – MASEM Indirect effects Support: β = 0.002 [−0.0002; 0.004]
Authoritarian: β = [0.001; 0.019] Behavioral: β = [−0.0002; 0.009] Psychological: β = −0.001 [−0.008; 0.004] Indirect parenting: β = [−0.0002; 0.015] MASEM vs. Univariate Meta-Analysis

18 Results – Compared MASEM vs. Univariate Meta-Analysis

19 Results – Compared MASEM vs. Univariate Meta-Analysis

20 Results – Compared MASEM vs. Univariate Meta-Analysis

21 Results – Compared MASEM vs. Univariate Meta-Analysis

22 Moderator Analysis Univariate meta-analysis
All kinds of moderator variables MASEM vs. Univariate Meta-Analysis

23 Moderator Analysis MASEM
Subgroup analysis (Jak & Cheung, in press)  compare groups of studies Continuous moderator variable  create subgroups Loss of information Group 1 Group 2 MASEM vs. Univariate Meta-Analysis

24 Moderator Analysis Continuous: Age
Univariate: significant effect on some associations MASEM: no significant effects Categorical: SES Univariate: no significant effects MASEM vs. Univariate Meta-Analysis

25 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

26 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

27 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

28 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

29 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

30 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

31 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

32 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

33 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

34 Conclusion Different methods  different effect sizes  different conclusions MASEM Univariate Meta-Analysis Set of variables in hypothesized model 2 variables – 1 association Multiple regression coefficients Bivariate correlations Can control for other variables Cannot control for other variables Can estimate indirect / mediation effects Only bivariate relationships Can answer RQ’s that were not explored before Only summary effect size of associations that were examined before Multivariate analysis Univariate analysis Moderator analysis with only categorical variables Moderator analysis with all kinds of variables Rather one moderator at a time Several moderators at once MASEM vs. Univariate Meta-Analysis

35 References Cheung, M. W. L. (2014). Fixed-and random-effects meta-analytic structural equation modeling: Examples and analyses in R. Behavior Research Methods, 46(1), Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5, 3-8. doi: / X Hoeve, M., Dubas, J. S., Eichelsheim, V. I., Van Der Laan, P. H., Smeenk, W., & Gerris, J. R. (2009). The relationship between parenting and delinquency: A meta-analysis. Journal of Abnormal Child Psychology, 37, Jak, S. (2015). Meta-Analytic Structural Equation Modeling. Springer International Publishing. doi: / Jak, S., & Cheung, M. W.-L. (in press). Testing moderator hypotheses in meta-analytic structural equation modeling using subgroup analysis. Behavior Research Methods. Viswesvaran, C., & Ones, D. (1995). Theory testing: Combining psychometric meta-analysis and structural equations modeling. Personnel Psychology, 48, doi: /j tb01784.x MASEM vs. Univariate Meta-Analysis


Download ppt "Univariate Meta-Analysis vs. Meta-Analytic Structural Equation Modeling Tessa van den Berg, University of Amsterdam Suzanne Jak, University of Amsterdam."

Similar presentations


Ads by Google