Using a Repeated Measures ANOVA to Analyze the Data from a Pretest- Posttest Design: A Potentially Confusing Task Schuyler Huck and Robert McLean

Overview  Pretest-Posttest Design  Repeated Measures ANOVA  Gain Scores  Problems with Repeated Measures ANOVA  Conservative F value  Interaction Effect versus Gain Scores  Post-Hoc Tests  Advantages of Gain Scores over Repeated Measures

SubjectPretestPost- test Experi- mental Group X 111 X 112 X 113 X 121 X 122 X 123 Control Group X 114 X 115 X 116 X 124 X 125 X 126  Repeated Measures ANOVA-  Effect of treatment  Effect of time  Interaction between treatment & time  Gain Scores  Difference between pretest and posttest scores for each person  Posttest-pretest  Use a one-way ANOVA, main effect of treatment

Problems with Repeated Measures ANOVA SubjectPretestPost- test Experi- mental Group X 111 X 112 X 113 X 121 X 122 X 123 Control Group X 114 X 115 X 116 X 124 X 125 X 126  F value is too small- pretest scores are included in effect of treatment, but no subject experienced treatment before the pretest scores. Treatment effects only influence posttest data.  Interaction effect is true main effect of treatment- the interaction examines the difference between groups depending upon pretest versus posttest scores.  Post Hoc Problems- Simple main effect tests run the risk of a “type IV error” and alpha values are controversial. Multiple comparison t-tests make sense only if gain score analyses was used.

Gain Scores vs Repeated Measures ANOVA  F ratios of Repeated Measures ANOVA are not useful  F main effect treatment- not an accurate estimate of treatment effect.  F interaction is equivalent to gain scores.  F main effect of time- no true experimental value.  Gain scores  Equivalent information as ANOVA, but without the confusion and controversy.  Principle of parsimony- simpler is better!