1. Conclusions/Future Directions
Testing the Mediated Effect in a Pretest-Posttest Control Group Design
Matthew J. Valente & David P. MacKinnon
Arizona State University
Introduction
Results
Pretest-posttest control group designs are common experimental designs used to assess change (Bonate, 2002)
Three main parts of a pretest-posttest control group design:
Measure theoretically relevant variables at one time point (pretest)
Randomly assign units to experimental group (control vs. treatment)
Measure theoretically relevant variables at a later time point (posttest)
These designs are commonly assessed using difference scores, residualized change scores or Analysis of Covariance (ANCOVA)
In many areas of research, pretest-posttest control group designs also incorporate mediating variables (Mackinnon, 2008)
Combining commonly used Pretest-Posttest control group designs with Mediation
Difference Score (Bonate, 2002; Campbell & Kenny, 2002; Dwyer, 1983)
On average, how much did each group change from pretest to posttest?
∆y = posttest outcome – pretest outcome
∆M = posttest mediator – pretest mediator
Residualized Change Score
On average, how different are posttest scores given equal pretest scores?
RY = observed posttest outcome – predicted posttest outcome
RM = observed posttest meditator– predicted posttest mediator
Analysis of Covariance (ANCOVA) (Bonate, 2002; Campbell & Kenny, 1999)
Pretest scores for outcome and mediator used as covariates
Research Question
How do the three commonly used methods perform when assessing the mediated effect in a pretest posttest control group design?
Conclusions/Future Directions
Method
Advantages of assessing mediation in a pretest-posttest control group design:
Can help establish temporal order of variables (X, M, and Y) (MacKinnon, 2008)
Allows researchers to assess between and within group differences and control for time-invariant confounding (MacKinnon, 2008)
ANCOVA, Difference score, and Residualized change score methods have similar results across a variety of effect size combinations.
The power of these methods are generally comparable, but for small and moderate effect sizes ANCOVA has more power
Future Directions
Conduct simulation of wider range of conditions and compare power and bias of these methods to simple mediation model
Control for pretest measures using causal inference techniques used in the potential outcomes approach
To assess the performance of these methods in detecting the mediated effect, we conducted Monte Carlo simulation studies
For each condition, there were 1,000 replications for a sample size of N = 200
Total of 13 conditions with different combinations of effect sizes am2x, by2m2, and cy2x
Pretest measures correlation (ry1m1) = 0.5
Pretest mediator and posttest outcome lag (by2m1) = 0.5
Stability of mediator and outcome (Sm2m1, Sy2y1) = 0.7 e1 X M1 Y1 M2 Y2
ry1m1
Sm2m1
Sy2y1
by2m1
am2x
cy2x
by2m2 e2
References
Bonate, P. L. (2000). Analysis of pretest-posttest designs. CRC Press.
Campbell, D. T., & Kenny, D. A. (1999). A primer on regression artifacts. New York, NY, US: Guilford Press, New York, NY.
Dwyer, J. H. (1983). Statistical models for the social and behavioral sciences. New York: Oxford University Press.
MacKinnon, D. P. (2008). Introduction to statistical mediation analysis. New York, NY: Taylor & Francis Group/Lawrence Erlbaum Associates, New York, NY.
Mediated effect = am2x* by2m2
This work was supported in part by the National Institute of Drug Abuse (R01DA ).
Matthew J. Valente is the contact author for this research.
Presented at the 22nd Annual Meeting of the Society for Prevention Research, 2014