1. Chapter 10: Analysis of Variance (ANOVA)

2. t test --Uses the t statistic to compare 2 means One-Way ANOVA --Also know as the F test --Uses the F statistic to compare the difference(s) among two or more means --Interested in the effects of a single independent variable on a dependent variable Two-Way ANOVA --Also uses the F statistic to compare the differences among means --Interested in the effects of multiple independent variables on a dependent variable

3. ANOVA Difference from t-test: 1. t-test can only test two groups. 2. ANOVA analyzes the size of differences between the groups (variability) and explains where it comes from. Definitions: Factors: Independent Variables. Factors: Independent Variables. Example: Type of Therapy; gender Example: Type of Therapy; gender Levels of a Factor: Groups w/in IV. Levels of a Factor: Groups w/in IV. Example: Behavioral Therapy & Cognitive Therapy; male, female Example: Behavioral Therapy & Cognitive Therapy; male, female

4. ANOVA: When to use? More than one IV or factor More than one IV or factor EXAMPLE: EXAMPLE: Factor A: Type of pain reliever Factor A: Type of pain reliever Factor B: type of pain. Factor B: type of pain. More than two levels of IV. More than two levels of IV. EXAMPLE: 3 groups receive treatment. EXAMPLE: 3 groups receive treatment. Pain reliever = aspirin vs. tylenol vs. ibuporfen Pain reliever = aspirin vs. tylenol vs. ibuporfen Type of pain = headache vs. backache Type of pain = headache vs. backache

5. Analysis of VARIANCE Like the t-test, there are both between subjects ANOVA and repeated measures ANOVA. Easy way to think: ANOVA is used to compare two or more means Harder (more accurate) way to think: ANOVA is used to compare the variability within the groups to the variability between the groups

6. One-Way ANOVA One DV Expressed as the mean score of a continuous variable IV (also called factor) will have 2 or more levels - male/female - freshman, sophomore, junior, senior -treatment a, treatment b, control group

7. Appropriate DV’s (must be continuous, interval or ratio scale) Age, Height, Score on The BDI, Number of Drinks Consumed, Income, Quality of Life, Reaction Time, IQ… Appropriate IV’s (called factors in ANOVA) Categorical Gender (male vs. female) Ethnicity (Caucasian vs. African American vs. Latino vs Asian.) Treatment Group (CBT vs. Medication vs. Wait-list Control) Graded Manipulations (0 mg vs. 10 mg vs. 20 mg) Continuous Converted to Categorical Age (Adolescent, Young Adult, Adult, Older Adult) Income (low, medium, high) Reaction Time (slow, medium, fast)

8. Has this occurred to anyone… …why not just do multiple t test? Every time we do a test, we run the risk of committing a type I error (alpha level = likelihood of type 1 error). The more tests we run, the more likely an error becomes One F test is better than 3 t tests, because it controls the experiment-wise or family-wise error rate, or the cumulative probability of committing a type I error

9. Table 13-1 (p. 393) Hypothetical data from an experiment examining learning performance under three temperature conditions.*

10. Null Hypothesis H 0 : Group 1 = Group 2 = Group 3 (M 1 = M 2 = M 3 ) Any differences are due to sampling error Alternative Hypothesis states (H 1 ) that one or more of the groups will differ from one another: Group 1 ≠ Group 2(M 1 ≠ M 2 ) and/or Group 1 ≠ Group 3(M 1 ≠ M 3 ) and/or Group 2 ≠ Group 3 (M 2 ≠ M 3 )

11. The Logic of the ANOVA We are interested in looking at variance from different angles. (hint: think of variance as the difference between scores) 1. Between-Treatments Variance…the differences between sample means 2. Within-Treatment Variance…the differences within each sample

12. Figure 13-10 (p. 414) A visual representation of the between-treatments variability and the within-treatments variability that form the numerator and denominator, respectively, of the F-ratio. In (a), the difference between treatments is relatively large and easy to see. In (b), the same 4-point difference between treatments is relatively small and is overwhelmed by the within-treatments variability.

13. Figure 13-2 (p. 396) The ANOVA partitions, or analyzes, the total variability into two components: variance between treatments and variance within treatments.

14. Explanations for the Difference/Variance that Exists Between/Within Treatments TREATMENT EFFECT: What was manipulated between the groups. TREATMENT EFFECT: What was manipulated between the groups. Always different between groups. Always different between groups. Cannot influence within-treatment variance since all the subjects in a group are given the same treatment. This is a between treatment variance. Cannot influence within-treatment variance since all the subjects in a group are given the same treatment. This is a between treatment variance. CHANCE: Difference is simply due to chance. These are unplanned & unpredictable differenced that are not caused by any action on the part of the researcher. Can effect between & within variance. CHANCE: Difference is simply due to chance. These are unplanned & unpredictable differenced that are not caused by any action on the part of the researcher. Can effect between & within variance.

15. Sources of Variance Due to Chance (random error)  INDIVIDUAL DIFFERENCES: Variability between all participants (gender, age, education level, mood). People bring different experiences to your study.  EXPERIMENTAL ERROR: Inaccurate measurement of the DV, poor planning of the study. Maybe measured weight w/ a broken scale, or made errors in administration of intelligence test.

16. The F Ratio: The test statistic for ANOVA F = variance between treatments (treatment effects + error/chance) variance within treatments (differences due to error/chance)

17. Possible Outcomes 1. Treatment has no effect. F = treatment effects + differences due to chance difference due to chance F = 0 + differences due to chance difference due to chance Numerator (top) and denominator (bottom) cancel each other out, and so the F statistics = ~ 1

18. Possible Outcomes 1. Treatment does have an effect. F = treatment effects + differences due to chance difference due to chance F = positive number + differences due to chance difference due to chance Numerator (top) is larger than the denominator (bottom), and so the F statistics is substantially greater than 1

19. ANOVA Notation and Formulas: Hypothetical data from an experiment examining learning performance under three temperature conditions. N = total number of scores in entire study n = total number of scores in each level of IV (each condition) k = number of levels of IV T = total (sum) of scores in each group G = total (sum) of scores across all groups SS = sum of squared deviations from the mean NOTE HOW THERE IS VARIANCE WITHIN EACH CONDITION AND VARIANCE BETWEEN OR (ACROSS) CONDITIONS

20. Figure 13-3 (p. 399) The structure and sequence of calculations for the analysis of variance.

21. The Analysis of Variance Summary Table Sources ofSum of dfMean F VariationSquare Square Between groups30 215 11.28 Within groups16 121.33 Total4614 “ Sum of Square” is total variability “Mean Square” is Variance, or the average amount of variability Total Sum of Squares— How much Variability in the Data? The total sums of squares measures the total scatter of scores around the grand mean. The grand mean is the mean of all the subject's scores regardless of the group to which they belong. Between-Groups Sum of Squares— How do the various Group Means vary? The between-groups sum of squares measures the total scatter of the group means with respect to the grand mean. Within-group Sum of Squares— How do Scores within the Group Vary? The within-group sum of squares measures the scatter of scores within each group with respect to the mean of that particular group.

22. Computing ANOVAs

23. Computing Sum of Squares (SS) N = total number of scores in entire study n = total number of scores in each level of IV (each condition) k = number of levels of IV T = total (sum) of scores in each group G = total (sum) of scores across all groups SS = sum of squares SS Total ∑X 2 - (G 2 /N) = 106 – (30 2 /15) = 46 SS Within = SS 1 + SS 2 + SS 3 = 6 + 6 = 4 = 16 SS Between = SS Total – SS Within = 30

24. Computing SS Between ∑ (T 2 /n) – G 2 /N = 5 2 /5 + 20 2 /5 + 5 2 /5 – 30 2 /15 = 5 + 80 + 5 – 60 = 30

25. The Analysis of Variance Summary Table Sources ofSum of dfMean F VariationSquare Square Between groups30 2x X Within groups16 12x Total4614 Degrees of Freedom 1 st entry = Numerator df OR Between-Groups df = k – 1, the number of groups (levels) – 1 2 nd entry= Denominator df OR Within-Groups df = N – k, Sample size - # number of groups (levels) 3 rd entry= Total df = N - 1

26. The Analysis of Variance Summary Table Sources ofSum of dfMean F VariationSquare Square Between groups30 215 11.28 Within groups16 121.33 Total4614 Total Sum of Square = Between Groups SS + Within Group SS Mean square = Sum of Square / df Between Groups Mean Square= 30 / 2 = 15 Within Groups Mean Square = 16 / 12 = 1.33 F is a ratio of MS BG /MS WG F = 15 / 1.33 = 11.28

27. The distribution of F-ratios with df = 2, 12. Of all the values in the distribution, only 5% are larger than F = 3.88, and only 1% are larger than F = 6.93.

28. A portion of the F-distribution table. Entries in light type are critical values for the.05 level of significance, and bold type values are for the.01 level of significance. The critical values for df = 2, 12 have been highlighted (see text). (page 392)

29. Decision/Conclusions Critical F (Fcrit) = 3.88 Obtained F (Fobt) = 11.28 The F-ratio for these data is in the critical region. It is very unlikely (p <.01) that we would obtain a value this large if H 0 is true. Reject H 0 and conclude that there are differences among the means of the 3 populations. Room temperature has a significant impact on learning. The F-ratio for these data is in the critical region. It is very unlikely (p <.01) that we would obtain a value this large if H 0 is true. Reject H 0 and conclude that there are differences among the means of the 3 populations. Room temperature has a significant impact on learning.

30. Calculate Effect Size η 2 (Eta squared) η 2 = SS Between / SS Total = 30/46 =.65 or 65% variance η 2 = SS Between / SS Total = 30/46 =.65 or 65% variance Interpreting η 2.01 -.09 = small effect Interpreting η 2.01 -.09 = small effect.09 -.25 = medium effect >.25 = large effect

31. F table Because F-ratio is always computed from 2 variances, F values are always positive. Because F-ratio is always computed from 2 variances, F values are always positive. Like t-test, the more df = more reliable statistic. Like t-test, the more df = more reliable statistic.

32. F Test is an “omnibus test” Used to determine if any of the groups differ from one another Post-Hoc Testing Used to determine which groups differ from one another

33. Post-Hoc Testing There are many approaches to post-hoc testing General Approach: If the Omnibus F test is significant, than you look to see which means differ from one another Two specific approaches include the Tukey HSD and the Scheffe Test. Important to have planned (a-priori) comparisons based on theory or previous data

34. Some more sample problems…

35. Example #1 Sources ofSum ofdfMean F VariationSquare Square Between groups602 ???? ???? Within groups205??? Total807 Mean square = Sum of Square / df Between Groups = ??? Within groups = ??? F is a ratio of MS BG /MS WG F = ???

36. Example #1 Sources ofSum ofdfMean F VariationSquare Square Between groups602 30 7.50 Within groups2054 Total807 Mean square = Sum of Square / df Between Groups = 60 / 2 = 30 Within groups = 20 / 5 = 4 F is a ratio of MS BG /MS WG F = 30 / 4 = 7.50 Is this statistically significant at the.05 level? Is this statistically significant at the.01 level?

37. Example #1 Sources ofSum ofdfMean F VariationSquare Square Between groups602 30 7.50 Within groups2054 Total807 Critical value for df 2, 5 =5.79 for p<.05 & 12.06 for p<.01. What sort of decision can you make about the null hypothesis? You can reject the null hypothesis at the.05 level, since the F value is in the critical region. You would fail to reject the null hypothesis at the.01 level, since the F value is not in the critical region

38. Example #2 Sources ofSum ofdfMean F VariationSquare Square Between groups202 ?? ?? Within groups25025?? Total27027 Mean square = Sum of Square / df Between Groups = ?? Within groups = ?? F is a ratio of MS BG /MS WG F = ??

39. Example #2 Sources ofSum ofdfMean F VariationSquare Square Between groups202 10 1 Within groups2502510 Total27027 Mean square = Sum of Square / df Between Groups = 20 / 2 = 10 Within groups = 250 / 25 = 10 F is a ratio of MS BG /MS WG F = 10 / 10 = 1.00 What sort of decision should you make about the null hypothesis?

40. Example #2 Sources ofSum ofdfMean F VariationSquare Square Between groups202 10 1 Within groups2502510 Total27027 Critical value for df 2, 25 = 3.38 for p<.05, and 5.57 for p<.01 What sort of decision should you make about the null hypothesis? You fail to reject the null hypothesis, because it is not within the critical region.

41. The F ratio Higher F ratio means there is more variability BETWEEN the groups than there is WITHIN the groups. When the variability within the groups is greater than the variability between the groups, than you fail to reject the null hypothesis Which is the most striking difference? Study 1Study 2 MSDMSD Group 11011010 Group 21511510 Group 32012010 High FLow F Reject the NullFail to Reject the Null

42. Hypothesis Testing State Hypotheses State Hypotheses Select Alpha Select Alpha Select Test (ANOVA) Select Test (ANOVA) Calculate F-ratio value Calculate F-ratio value SS between, SS within, SS Total SS between, SS within, SS Total MS between, MS within MS between, MS within Locate Critical Value (F-Table) Locate Critical Value (F-Table) Compare values and draw conclusion. Compare values and draw conclusion. Compute Effect Size Compute Effect Size

43. ANOVA Practice Sample 1 Sample 2 Sample 3 066 485 059 144 026 T = 5; M = 1 T = 25; M = 5 T = 30; M = 6 SS = 12 SS = 20 SS = 14 N = 15 G = 60 ∑X 2 = 356

44. Step 1: State the Hypotheses and Select an Alpha Level Step 1: State the hypotheses & select the alpha level Ho = µ1 = µ2 = µ3 H1 = At least one condition is different. alpha =.05

45. Step 2: Locate the Critical Region Step 2: Locate the critical region for the F-ratio df total = N – 1 = 15-1 = 14 df between = k – 1 = 3 - 1 = 2 df within = N – k = 15 - 3 = 12 The F-ratio will have df = 2, 12 at p=.05. Critical F = 3.88

46. Step 3: Compute the Test Statistic It is best to compute all 3 Sum or squares (between, within, and total) and check that between and within add to the total It is best to compute all 3 Sum or squares (between, within, and total) and check that between and within add to the total SS Total ∑X 2 - (G2 /N) = 356 – (60 2 )/ 15 = 116 SS Within = ∑ SS = SS 1 + SS 2 + SS 3 = 12 + 20 + 14 = 46 SS Between ∑ (T 2 /n) – G 2 /N = 5 2 /5 + 25 2 /5 + 30 2 /5 – 60 2 /15 = 5 + 125 + 180 – 240 = 70 CHECK = SS W + SS B = SST (46 + 70 = 116)

47. Step 3: Compute the Test Statistic (continued) df total = 14; df between = 2; df within =12 MS between = SS between /df between = 70/2 = 35 MS within = SS within /df within = 46/12 = 3.83 F = MS between / MS within = 35/3.83 = 9.14

48. Step 4 Make Decision and State Conclusion Critical F (Fcrit) = 3.88 Obtained F (Fobt) = 9.14 The F-ratio for these data is in the critical region. It is very unlikely (p <.05) that will obtain a value this large if H 0 is true. Reject H 0 and conclude that there are differences among the means of the 3 populations. The F-ratio for these data is in the critical region. It is very unlikely (p <.05) that will obtain a value this large if H 0 is true. Reject H 0 and conclude that there are differences among the means of the 3 populations.

49. Step 5 Calculate Effect Size η 2 (Eta squared) η 2 = SS Between / SS Total = 70/116 =.60 or 60% variance η 2 = SS Between / SS Total = 70/116 =.60 or 60% variance Interpreting η 2.01 -.09 = small effect Interpreting η 2.01 -.09 = small effect.09 -.25 = medium effect >.25 = large effect