1. General Linear Model 2
Intro to ANOVA

2. Questions
ANOVA makes assumptions about error for significance tests. What are the assumptions?
What might happen (why would it be a problem) if the assumption of {normality, equality of error, independence of error} turned out to be false?
What is an expected mean square? Why is it important?
Why do we use the F test to decide whether means are equal in ANOVA?

3. Questions (2) Correctly interpret ANOVA summary tables.
Find correct values of critical F from tabled values for a given test.
Suppose someone has worked out that a one-way ANOVA with 6 levels has a power of .80 for the overall F test. What does this mean?
Describe (make up) a concrete example of a one-way ANOVA where it makes sense to use an overall F test. Explain why ANOVA (not t, chi-square or something else) is the best method for the analysis.

4. New Distributions
So far, the normal (z) and its short, fat relative, the t distribution.
The normal has two children, chi-square ( ) and F.
Chi-square is made of the sum of v squared deviations from the unit normal. It essentially show the sampling distribution of the variance.
F is the ratio of two chi-squares.

5. ANOVA Assumptions
Recall we can partition total SS into between (treatment) and within (error) SS. No assumptions needed.
To conduct tests about population effects, have to make assumptions:
Within cells (treatments) error is normal.
Homogeneity of error variance.
Independent errors.

6. Assumptions
Normality – sampling distribution of means,variances; not bad if N is large; e.g. reaction time
Homogeneity – pooled estimate of population value. Where are means different? Assumed equal error for each. E.g., ceiling effects in training.
Independence – sampling distribution again; e.g., cheating on exam, nesting (schools, labs)

7. Mean Square Between Groups
Mean square = SS/df = = variance estimate.
MS between =
E(MS between) =
If there is no treatment effect, MS between = error variance.
If there is a treatment effect, MS between is bigger than error variance.
(J treatments)

8. Mean Square Within Groups
(N is total sample size and J is number of groups.)
MS within =
E(MS within) =
Expected mean square for error is Expected mean square for treatment is same plus treatment effect:
When there is no treatment effect, between and within estimate same thing.

9. Review
ANOVA makes assumptions about error for significance tests. What are the assumptions?
What might happen (why would it be a problem) if the assumption of {normality, equality of error, independence of error} turned out to be false?
What is an expected mean square? Why is it important?

10. The F Test (1) Suppose The null is equivalent to:
If the null is true, then
for all j
for some j
The ratio of the two variance estimates will be distributed as F with J-1 and N-J degrees of freedom.

11. The F Test (2)
This is a big deal because we can use variance estimates to test the hypothesis that any number of population means are equal. Equality of means is same as testing population treatment effect(s).
For a treatment effect to be detected, F must be larger than 1. F is one-tailed in the tables which show upper tail values of F given the two df.

12. F Table – Critical Values
Numerator df: dfB
dfW 1 2 3 4 5
5 5% 1%
6.61
16.3
5.79
13.3
5.41
12.1
5.19
11.4
5.05
11.0
10 5%
4.96
10.0
4.10
7.56
3.71
6.55
3.48
5.99
3.33
5.64
12 5%
4.75
9.33
3.89
6.94
3.49
5.95
3.26
3.11
5.06
14 5%
4.60
8.86
3.74
6.51
3.34
5.56
5.04
2.96
4.70

13. Review
Why do we use the F test to decide whether means are equal in ANOVA?
Suppose we have an ANOVA design with 3 cells and 5 people per cell. What is the critical value of F at alpha = .05?

14. Calculating F – 1 Way ANOVA
Sums of squares (squared deviations from the mean) tell the story of variance. The simple ANOVA designs have 3 sums of squares.
The total sum of squares comes from the distance of all the scores from the grand mean. This is the total; it’s all you have.
The within-group or within-cell sum of squares comes from the distance of the observations to the cell means. This indicates error.
The between-cells or between-groups sum of squares tells of the distance of the cell means from the grand mean. This indicates IV effects.

15. Computational Example: Caffeine on Test Scores
G1: Control
G2: Mild
G3: Jolt
Test Scores 75 80 70 77 82 72 79 84 74 81 86 76 83 88 78
Means
SDs (N-1)
3.16

16. Total Sum of Squares G1 75 79 16 Control 77 4 M=79 SD=3.16 81 83 G2 80
SD=3.16 81 83 G2 80 1
M=84 82 9 84 25 86 49 88 G3 70
M=74 72 74 76 78
Sum
370
Total Sum of Squares

17. Within Sum of Squares G1 75 79 16 Control 77 4 M=79 SD=3.16 81 83 G2
SD=3.16 81 83 G2 80 84
M=84 82 86 88 G3 70 74
M=74 72 76 78
Sum
120
Within Sum of Squares

18. Between Sum of Squares G1 79 Control M=79 SD=3.16 G2 84 25 M=84 G3 74
Control
M=79
SD=3.16 G2 84 25
M=84 G3 74
M=74
Sum
250
Between Sum of Squares

19. ANOVA Source (Summary) TableSS df MS F
Between Groups
250
J-1=
3-1=2
SS/df
250/2=
125 =MSB
F = MSB/MSW = 125/10
=12.5
Within Groups
120
N-J=
15-3=12
120/12 = 10 =
MSW
Total
370
N-1=
15-1=14

20. ANOVA Summary Calculate SS (total, between, within)
Each SS has associated df to calculate MS
F is ratio of MSb to MSw
Compare obtained F (12.5) to critical value (3.89). Significant if obtained F is larger than critical.
One-tailed test makes sense for F.

21. Review
Suppose we have 4 groups and 10 people per group. We find that SSB = 60 and SSW = 40. Construct an ANOVA summary table and test for significance of the overall effect.

22. ANOVA Descriptive Stats
Because SStot = SSb+SSw we can figure proportion of total variance due to treatment.
Proportion of total variance due to treatment is:
R2= SSb/SStot.
Varies from 0 (no effect) to 1 (no error).
Sample value is biased (too large).

23. Estimating Power
Power for what? For one-way ANOVA, power usually means for the overall F, i.e., at least 1 group mean is different from the others.
Howell uses noncentral F for sample size calculation.
Where k is the number of treatment goups; n is sample size per group. Variance of error is MSE in the population (variance of DV within cells). Mu(j) are treatment means; mu is grand mean.

24. SAS Power calculation
SAS will compute sample size requirements for a given scenario.
You input the expected means and a common (within cell) standard deviation, (along with alpha and desired power) and it will tell you the sample size you need.

25. SAS Input proc power ; onewayanova
run;
SAS Input
**********************************************************
* Power computation example from Howell, 2010, p. 350.
* Note the standard deviation is the square root of the
* provided MSE: sqrt(240.35) = ~ 15.5.
**********************************************************;
proc power ;
onewayanova
groupmeans = 34 | 50.8 | | 48.5 | 38.1
stddev = 15.5
alpha = 0.05
npergroup = .
power = .8;

26. SAS Output
The POWER Procedure Overall F Test for One-Way ANOVA Fixed Scenario Elements Method Exact Alpha 0.05 Group Means Standard Deviation 15.5 Nominal Power 0.8 Computed N Per Group Actual N Per Power Group

27. Review
Suppose someone has worked out that a one-way ANOVA with 6 levels has a power of .80 for the overall F test. What does this mean?
Describe (make up) a concrete example of a one-way ANOVA where it makes sense to use an overall F test. Explain why ANOVA (not t, chi-square or something else) is the best method for the analysis.