Chapter 9 Introduction to the Analysis of Variance Part 2: Oct. 24, 2013

Planned Comparisons When we reject null hypothesis in ANOVA, we conclude group means are not all the same But exactly which groups differ? Not known from overall F test Planned comparisons Allow us to determine which groups differ Often have a priori (theoretical) reason to predict that certain pairs of groups may differ, can test this

Procedure for planned comparisons: Estimate within-groups population variance – in same way as for overall F test Next, estimate between-groups population variance For overall F test, we used all groups means for this estimate, here… Use only the two means of interest (see formula next) Figure F in usual way – same as overall F test

Bonferroni correction Potential problem when making multiple planned comparisons: Want overall alpha = .05, but if doing 3 separate comparisons each at alpha = .05, then overall alpha = 3(.05) = .15 (which means a 15% Type 1 error rate) Not acceptable to increase alpha, need some control Use more stringent significance level for each separate comparison, not .05 Divide overall alpha level by # comparisons you plan to make (.05 / 3 = .017)  use .017 as alpha for each separate comparison and you keep overall alpha at .05

ANOVA Effect size In ANOVA, total effect size for the mean differences among all the groups is R2 Unlike t-test effect sizes, R2 cannot be negative

Effect size (cont.) R2 also known as η2 (eta squared) Ranges from 0-1 (make sure it’s pos.) Cohen’s standards for size of effect for R2 or η2 in ANOVA: small R2 = .01 medium R2 = .06 large R2 = or > .14

ANOVA in Research Articles F(3, 67) = 5.81, p < .01 F (df between groups, df within groups) = F obtained, significance level (here, signif using alpha of .01) Means given in a table or in the text Follow-up analyses Planned comparisons – report those F tests See SPSS example for APA format…

SPSS Example for 1-way ANOVA Harassment data set with school district employees “School” variable indicates work setting 1=elementary school 2=middle school 3=high school “Harassment in 1997” indicates har experiences from ’96-’97 Does the work setting influence harassment experiences?

In SPSS menus: Analyze  Compare Means  One-way ANOVA Then, “Dependent List” can indicate as many dependent variables as you’d like…here “Harassment in 1997” In “Factor” indicate the ‘grouping’ variable on which you’ll compare Harassment means… here “School”

Click the “Options” button at bottom, click the box for Descriptives under “Statistics”, hit continue… Click the “Post Hoc” button at bottom, click the box for “Bonferroni”, hit continue… (this will give you output for follow-up comparisons in case your overall ANOVA is signif  if it’s not, you’ll ignore these comparisons) Now hit “OK” to run the analysis

Output You’ll have 3 sections of output… The 1st reports the group harassment means for elementary, middle, and high school employees You’ll need to look back at this to help w/interpretation in case your ANOVA is signif! 2nd gives the overall ANOVA F test – for the null hypothesis of “no group differences” Notice the MSBetween and MS Within, then the F statistic is your F obtained value, next to that is the “Sig” value If “Sig” value is < .05 (or .01 – depends on alpha)  Reject Null and conclude there are significant group differences

Overall ANOVA test results Is there a significant difference in the group means? APA-formatted summary: Bonferroni post-hoc tests?

3rd section gives follow-up comparisons (Bonferroni – but remember to use .05/3 = .017 as your new comparison alpha level) Check each row for which pairs are being compared, then its “sig” value If “sig” < .017 (or whatever your new alpha is)  Reject null of equal group means; conclude those 2 group means differ Which schools significantly differ in harassment experiences? APA-formatted summary: