1. 26. Follow-up tests and two-way ANOVA
The Practice of Statistics in the Life Sciences
Third Edition
© 2014 W.H. Freeman and Company

2. Objectives (PSLS Chapter 26)
Follow-up tests and two-way ANOVA
Comparing several means
Tukey’s pairwise multiple comparisons
Contrasts
Two-way designs
The two-way ANOVA model
Interaction
Inference for two-way ANOVA

3. Comparing several means
When comparing >2 populations, the question is not only whether each population mean µi is different from the others, but also whether they are significantly different when taken as a group.
If we split a class of 200 into 20 random samples of 10 students and found the 20 mean heights, the 2 most extreme samples could very well be “significantly different” --- but that would be a Type 1 error.
We should ask instead if all 20 samples might have come from a single population or not.
There should only be one null hypothesis, stating that we have sampled many times from the same population.
It would be wrong to test a series of two-sample null hypotheses or to only test the two most extreme results.
If we set α = 5% and run multiple analyses, we can expect to incorrectly reject H0 (Type I error) about 5% of the time.

4. Handling multiple comparisons statistically
The first step in examining multiple populations statistically is to test for an overall statistical significance as evidence of any difference among the parameters we want to compare.
ANOVA F test (Chapter 24)
If that overall test showed statistical significance, then a detailed follow-up analysis can examine all pair-wise parameter comparisons to define which parameters differ from which and by how much.
 pairwise multiple comparisons

5. Tukey’s pairwise multiple comparisons
Pairwise multiple comparisons among k groups are variants of the two-sample t procedures. However,
they use the pooled standard deviation sp =√MSE
the pooled degrees of freedom DFE = N – k
and they compensate for the multiple comparisons.

6. Simultaneous confidence intervals
We can calculate simultaneous level C confidence intervals for all pairwise differences (µi – µj) between population means:
Table G contains values of m* for the 95% confidence level only .
m* depends on the confidence level C, the number of populations k, and the total number of observations N. Use technology or Table G.

7. Table G

8. Simultaneous tests of hypotheses
To carry out simultaneous Tukey tests of the hypotheses
H0: μi = μj
Ha: μi ≠ μj
for all pairs of population means, reject H0 for any pair whose Tukey level-C confidence interval does not contain 0.
These tests have an overall significance level no less than 1 − C. That is, 1 − C is the probability that, when all of the population means are equal, any of the tests incorrectly rejects its null hypothesis.

9. H0: µchow = µrestricted = µextended Ha: H0 is not true
Male lab rats were randomly assigned to 3 groups differing in access to food for 40 days. The rats were weighed (in grams) at the end.
Group 1: access to chow only.
Group 2: chow plus access to cafeteria food restricted to 1 hour every day.
Group 3: chow plus extended access to cafeteria food (all day long every day).
H0: µchow = µrestricted = µextended Ha: H0 is not true
One-way ANOVA: Chow, Restricted, Extended
Source DF SS MS F P
Factor
Error
Total
Pooled StDev = 54.42
With Table G, m* ≈ (df 3 and 47≈ 40),
Tukey 95% CI for µextended – µrestricted:
[Cafeteria food: bacon, sausage, cheesecake, pound cake, frosting and chocolate]
Another way to word the alternative hypothesis: Ha: at least one mean µ is different from at least one other mean µ
Because the interval computed contains 0, we fail to find significant evidence of a difference between the mean weights of male lab rats given extended and restricted access to cafeteria food.
N Mean
Chow Restricted Extended

10. Minitab
Tukey 95% Simultaneous Confidence Intervals
All Pairwise Comparisons
Individual confidence level = 98.05%
Chow subtracted from:
Lower Center Upper
Restricted ( *------)
Extended (------* )
Restricted subtracted from:
Lower Center Upper
Extended ( * )
Because the interval computed contains 0, we fail to find significant evidence of a difference between the mean weights with extended and restricted access to cafeteria food.
Notice that software gives the interval for µextended – µrestricted as –13.47 to 81.12, similar to our approximate results on the previous slide (–13.79 to 81.43) due to the conservative approximation of the value of m*.
 There are two significant differences at simultaneous significance level 5%:
μrestricted – μchow and μextended – μchow

11. Two-way designs
In a two-way design, 2 factors are studied in conjunction with the response variable. There is thus two ways of organizing the data, as shown in a two-way table.
Two-way table
(3  3 design to test the attractiveness of sticky boards to beetles)
When the response variable is quantitative, the data are analyzed with a two-way ANOVA procedure. A chi-square test is used instead if the response variable is categorical.
Color
Texture
Red
Green
Blue
Smooth 14
Grooves
Bumps

12. Advantages of a two-way ANOVA model
It is more efficient to study two factors at once than separately.
Smaller samples sizes per condition are needed because the samples for all levels of factor B contribute to sampling for factor A.
Including a second factor thought to influence the response variable helps reduce the residual variation in a model of the data.
In a one-way ANOVA for factor A, any effect of factor B is assigned to the residual (“error” term). In a two-way ANOVA, both factors contribute to the fit part of the model.
Interactions between factors can be investigated.
The two-way ANOVA breaks down the fit part of the model between two main effects and an interaction effect.

13. The two-way ANOVA model
We record a quantitative variable in a two-way design with R levels of the row factor and C levels of the column factor.
We have independent SRSs from each of R x C Normal populations. Sample sizes do not have to be identical (although some software only carry out the computations for equal sample sizes  “balanced design”).
All parameters are unknown. The population means may be different but all populations have the same standard deviation σ.
When the design is not balanced, most software will let you use a general linear model procedure, or GLM.

14. Main effects and interaction effect
Each factor is represented by a main effect: this is the impact on the response variable of varying levels of that factor, regardless of the other factor (i.e., pooling together the levels of the other factor). There are two main effects, one for each factor.
The interaction of both factors is also studied and is described by the interaction effect.
When there is no clear interaction, the main effects are enough to describe the data. In the presence of interaction, the main effects could mask what is really going on with the data.

15. Major types of two-way ANOVA outcomes
In a two-way design, statistical significance can be found for each factor, for the interaction effect, or for any combination of these.
Neither factor is significant
Only 1 factor is significant
Both factors are significant
No interaction
Interaction effect is significant
With or without significant interaction
Dependent var.
Levels of factor A B1 B2
Levels of factor B:
Dependent var.
Levels of factor A

16. Interaction
Two variables interact if a particular combination of variables leads to results that would not be anticipated on the basis of the main effects of those variables.
Drinking alcohol increases the chance of throat cancer. So does smoking. However, people who both drink and smoke have an even higher chance of getting throat cancer. The combination of smoking and drinking is particularly dangerous: these risk factors interact.
An interaction implies that the effect of one variable differs depending on the level of another variable.
The effect of smoking on the probability of getting throat cancer is greater for people who drink than for people who do not drink.

17. Interpreting plots Main effects, no interaction
Important interaction effect: the main effects may be misleading

18. Inference for two-way ANOVA
A one-way ANOVA tests the following model of your data:
Data (“total”) = fit (“groups”) + residual (“error”)
so that the sum of squares and degrees of freedom are:
SST = SSG + SSE
DFT = DFG + DFE
A two-way design breaks down the “fit” part of the model into more specific subcomponents, so that:
SST = SSR + SSC + SSRC + SSE
DFT = DFR + DFC + DFRC + DFE
where R and C are the 2 main effects from each of the 2 factors, and RC represents the interaction of factors R and C.

19. The two-way ANOVA table
Source of variation DF
Sum of squares SS
Mean square MS F
P-value
Factor R
DFR = r -1
SSR
SSR/DFR
MSR/MSE
for FR
Factor C
DFC = c - I
SSC
SSC/DFC
MSC/MSE
for FC
Interaction
DFRC = (r-1)(c-1)
SSRC
SSRC/DFRC
MSRC/MSE
for FRC
Error
DFE = N - rc
SSE
SSE/DFE
Total
DFT = N – 1 =DFR+DFC+DFRC+DFE
SST =SSR+SSC+SSRC+SSE
SST/DFT
Main effects: P-value for factor R, P-value for factor C.
Interaction: P-value for the interacting effect of R and C.
Error: It represents the variability in the measurements within the groups. MSE is an unbiased estimate of the population variance s2.

20. Nematodes and plant growth
Do nematodes affect plant growth? A botanist prepares 16 identical planting pots and adds different numbers of nematodes into the pots. Seedling growth (in mm) is recorded two weeks later. We analyzed these data with a one-way ANOVA in the previous chapter.
We also have data for another plant species. We can study the effect of nematode amounts (4 levels) on seedling growth for both plant species (2 levels, species A and B).
Plant species A was also grown with pesticide. We can analyze seedling growth for combinations of nematodes and pesticide conditions.

21. Nematode levels and plant species
Source SS df MS F-ratio P
NEMATODES
SPECIES
NEMATODES*SPECIES
Error
All plants suffer from the presence of nematodes (main effect P < 0.001), but plant A and plant B do not have significantly different growth (main effect P = 0.39).
The third plot shows that the effect of nematodes is less pronounced for plant A, shown in red (interaction effect P = 0.017).

22. Nematode levels and pesticide
Source SS df MS F-ratio P
NEMATODES
PESTICIDE
NEMATODES*PESTICIDE
Error
Both main effects are very significant (P < 0.001).
The interaction is significant (P = 0.001): We see from the third plot that the detrimental effect of nematodes is much more pronounced in pesticide-free pots, shown in red.

23. Dietary manipulations in fruit flies
An experiment assessed the percent of body fat in female fruit flies fed one of four diets enriched with yeast (a high-protein food). Wild-type fruit flies and mutants with a longer reproductive cycle were used.
Only one main effect is significant: yeast amount
(P < 0.001)
There is no significant interaction effect.

24. Better corn for heavier chicks?
A study compared the effect of 3 corn varieties (“norm,” “opaq” and “flou”) and 3 feed protein levels on the growth of chicks. Male chicks were randomly allotted to treatments at birth and their weight gain was recorded 21 days later.
The interaction effect is significant (P < 0.001), meaning that mean weight gain differs for different combinations of corn variety and protein level.
Both main effects are also significant (P ≤ 0.001).