STAT 312 10.1 - Single-Factor ANOVA Chapter 10 - Analysis of Variance (ANOVA) Introduction 10.1 - Single-Factor ANOVA 10.2 - Multiple Comparisons in ANOVA 10.3 - More on Single-Factor ANOVA

1 2 k = H0: …etc… Idea: Test all possible pairwise comparisons, each via a two-sample t-test. Example : Suppose there are k = 5 treatment groups. There are such comparisons. PROBLEM???

SPURIOUS SIGNIFICANCE!!! Suppose we wish to make n independent comparisons, each at significance level . As the number of comparisons increases, so does the probability of rejection, just by chance! SPURIOUS SIGNIFICANCE!!! When n = 14, this probability is > 50%.

SPURIOUS SIGNIFICANCE!!! Suppose we wish to make n independent comparisons, each at significance level . As the number of comparisons increases, so does the probability of rejection, just by chance! SPURIOUS SIGNIFICANCE!!! When n = 14, this probability is > 50%. One remedy: Make each t-test comparison more conservative, i.e., harder to reject!

SPURIOUS SIGNIFICANCE!!!  p-value Suppose we wish to make n independent comparisons, each at significance level . As the number of comparisons increases, so does the probability of rejection, just by chance! SPURIOUS SIGNIFICANCE!!! When n = 14, this probability is > 50%. One remedy: Make each t-test comparison more conservative, i.e., harder to reject! * =  /n

p-value  * =  /n H0: = PROBLEM??? …etc… 1 2 k = H0: …etc… Idea: Test all possible pairwise comparisons, each via a two-sample t-test. Example : Suppose there are k = 5 treatment groups. There are such comparisons. PROBLEM??? * =  /n

* = .05 /10 = .005 H0: = PROBLEM??? …etc… 1 2 k = H0: …etc… Idea: Test all possible pairwise comparisons, each via a two-sample t-test. Example : Suppose there are k = 5 treatment groups. There are such comparisons. PROBLEM??? * = .05 /10 = .005

BONFERRONI CORRECTION 1 2 k = H0: …etc… Idea: Test all possible pairwise comparisons, each via a two-sample t-test. Example : Suppose there are k = 5 treatment groups. There are such comparisons. BONFERRONI CORRECTION Holm-Bonferroni Tukey’s Honest Significant Difference,… * = .05 /10 = .005

Analysis of Variance (ANOVA) Alternate method ~ Analysis of Variance (ANOVA) Main Idea: Among several (k  2) independent, equivariant, normally-distributed “treatment groups”… MODEL ASSUMPTIONS? 1 2 k = H0:

Analysis of Variance (ANOVA) Alternate method ~ Analysis of Variance (ANOVA) Main Idea: Among several (k  2) independent, equivariant, normally-distributed “treatment groups”… Equivariance can be tested via very similar “two variances” F-test in 6.2.2 (but this is very sensitive to normality assumption), or others. If violated, can extend Welch Test for two means. 1 2 k = H0:

Analysis of Variance (ANOVA) Alternate method ~ Analysis of Variance (ANOVA) Main Idea: Among several (k  2) independent, equivariant, normally-distributed “treatment groups”… Normality can be tested via usual methods. If violated, use nonparametric Kruskal-Wallis Test. 1 2 k = H0:

Analysis of Variance (ANOVA) Alternate method ~ Analysis of Variance (ANOVA) Main Idea: Among several (k  2) independent, equivariant, normally-distributed “treatment groups”… Extensions of ANOVA for data in matched “blocks” designs, repeated measures, multiple factor levels within groups, etc. 1 2 k = H0:

Analysis of Variance (ANOVA) Alternate method ~ Analysis of Variance (ANOVA) Main Idea: Among several (k  2) independent, equivariant, normally-distributed “treatment groups”… How to identify significant group(s)? Pairwise testing, with correction (e.g., Bonferroni) for spurious significance. Example: k = 5 groups result in 10 such tests, so let each α* = α / 10. 1 2 k = H0:

“spurious significance”

Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > concrete days cure 1 4 A 2 5 A 3 4 A 4 3 A ... 26 5 C 27 5 C

Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > boxplot(days ~ cure, data = concrete)

Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > boxplot(days ~ cure, data = concrete) Method A seems the fastest; B and C seem comparable, both taking a longer time.

Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 STRONGLY REJECTED > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > results = aov(days ~ cure, data = concrete) > summary(results) Df Sum Sq Mean Sq F value Pr(>F) cure 2 28.222 14.1111 11.906 0.0002559 *** Residuals 24 28.444 1.1852 --- Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

Pairwise comparisons using t tests with pooled SD Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 A B C STRONGLY REJECTED > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > results = aov(days ~ cure, data = concrete) Multiple Comparisons? > pairwise.t.test(days, cure, p.adjust="bonferroni") Pairwise comparisons using t tests with pooled SD data: days and cure A B B 0.00119 - C 0.00068 1.00000 P value adjustment method: bonferroni

Example Different methods of curing concrete are being compared. A total of 27 slabs are selected; 9 are randomly assigned to one of three curing methods. The number of days it takes to be fully cured is recorded: Cure A: 4 5 4 3 2 4 3 4 4 Cure B: 6 8 4 5 4 6 5 8 6 Cure C: 6 7 6 6 7 5 6 5 5 A B C STRONGLY REJECTED > days = c(4,5,4,3,2,4,3,4,4,6,8,4,5,4,6,5,8,6,6,7,6,6,7,5,6,5,5) > cure = c(rep("A",9), rep("B",9), rep("C",9)) > concrete = data.frame(days,cure) > results = aov(days ~ cure, data = concrete) Multiple Comparisons? Alt Method: Tukey’s Honest Significant Difference (HSD) > TukeyHSD(results, conf.level = 0.95) Tukey multiple comparisons of means 95% family-wise confidence level Fit: aov(formula = days ~ cure, data = concrete) $cure diff lwr upr p adj B-A 2.1111111 0.8295028 3.392719 0.0011107 C-A 2.2222222 0.9406139 3.503831 0.0006453 C-B 0.1111111 -1.1704972 1.392719 0.9745173