1. Lecture 11
Introduction to ANOVA

2. Experiment
First-born children tend to develop language skills faster than their younger siblings. One possible explanation for this phenomenon is that first-born have undivided attention from their parents. A researcher wanted to test this theory by comparing the language development of language across only children, twins and triplets. Davis (1973) predicted that the multiple birth children should have slower language development.

3. ANOVA
ANOVA or analysis of variance: a hypothesis testing procedure that is used to evaluate differences between 2 or more samples.
It is an omnibus test: permits the analysis of several variables at the same time
Nice because we have greater flexibility in designing our experiments. We can make multiple comparisons with one test.
single child vs. twins
single child vs. triplets
twins vs. triplet

4. ANOVA and research design
Independent Measures - 2 or more different samples
Repeated Measures - 2 or more measurements from the same sample. The same sample is tested across all the different treatment conditions.
Mixed Design

5. ANOVA and Research Design
Independent variable - Now we can test more than one independent variable
Say we wanted to look at language development across 2 independent variables (# of births at one time) and SES
Note: SES and births are quasi-independent variables: we are differentiating our groups by them, but can’t manipulate them.
Factors - The independent variables are called factors in ANOVA.
A study with only 1 IV is a single-factor design.
A study with more than 1 IV is a factorial design.
Levels - individual groups w/in a factor

6. Example How many factors are in this design? What are they?
What are the levels of each factor?
How many factors are in this design? What?
What are the levels of each factor?

7. ANOVA
Today we’ll just introduce ANOVA (although remember repeated measures and multiple factor designs are based on the statistics covered today)
Goal of ANOVA is to help us decide
(1) There are no differences between the populations (null hypothesis).
(2) The differences represent real differences between populations (alternative hypothesis).

8. Single-Factor Independent Measures ANOVA
Population 1
Treatment 1
Population 2
Treatment 2
Population 3
Treatment 3
Population 4
Treatment 4
μ = ?
μ = ?
μ = ?
μ = ?
Sample 1 30 45 40 42 35
Sample 2 50 52 54 56 60
Sample 3 42 43 45 47 52
Sample 4 31 42 45 34 33

9. Test Statistic For ANOVA: F Ratio
Reminder:
t = obtained difference between samples
difference expected by chance (error)
Structure for ANOVA is the same; test statistic is call the F-ratio:
F = variance (differences) between sample means
variance (differences) expected by chance
F - ratio is based on VARIANCE instead of sample mean DIFFERENCES
With more than 2 samples how would we calculate differences

10. Breaking it Down
s2 = 58.33
s2 = 2.33
Numerator - Can’t calculate mean difference score what would the diff. score bwtn 20, 30 and 35 be?
Variance simply gives us info about differences. The scores are different.
Set 1 has large differences btwn means and large variance
Set 2 has small differences btwn means and small variance
Denominator of t measures standard error or standard deviation expected by chance. Denominator of F simply squares that to variance.

11. ANOVA: Rationale ANOVA = analysis of variance
Analysis means break into parts
We are going to break the total variance into 2 parts.
Between - Treatments Variance = really measuring the differences between sample means
Within - Treatments Variance = measuring differences inside each treatment condition

12. A Closer Look… F = variance (differences) between sample means
variance (differences) expected by chance
Exp. 1
Exp. 2
Sample 1
Sample 2
Sample 3
Sample 1
Sample 2
Sample 3
s2 = MS = 225
s2 = MS = 4
Numerator = variance between samples.
Variance between samples is big for experiment 1 and small for experiment 2.
Denominator = variance within samples.
Measure variance within the group, so how much variability was there in the set of scores that made up Exp 1, sample 1 and so on.

13. Between Treatments Variance (MS) Numerator
Variance = Differences
Between treatments variance = how much difference between groups
(1) Differences between treatments due to chance.
Chance = unpredictable or unplanned differences
(1) Individual Differences
(2) Experimental Errors (measurement)
(2) Differences caused by the treatments

14. Within Treatments Variance (MS) Denominator
Within - treatments variance = inside each treatment condition each participant is treated the same. Researcher doesn’t do anything to cause individual differences.
But there is still variability between individuals
Chance

15. Total Variability Between-treatments Variance Within-treatments
Measures difference due to:
1. Chance
2. Treatment Effects
Measures difference due to:
1. Chance

16. Putting it Together: F - Ratio
Total Variability (F) = MS Between
MS Within OR
F = treatment + chance
chance
(1) Treatment has no effect = the numerator and denominator are only measuring chance, so ratio hovers near 1.
(2) Treatment has an effect ratio should be substantially > 1.
IMPORTANT NOTE: Since the denominator measures uncontrolled and unexplained variance it is called the ERROR TERM.

17. Putting it Together
What will happen to our F-ratio as the difference btwn treatment increases?
What will happen to our F-ratio as the difference within treatments increases?
What does an F-ratio of 1 mean?

18. ANOVA: Terms and Notation
Levels = # of individual treatment conditions that make up a factor (notated by k…for independent samples k also specifies the number of samples).
n = # of scores in each level or treatment (k).
N = total number of scores in the entire study (note that this is not the population just the n from all samples).
T = X for each treatment condition (total)
G = T for the entire study (grand total)

19. Let’s look with our Example
A single factor design - IV = # children in a single birthing experience
Levels: k = 3 ( # conditions that make up our factor)
n = 5 for each level (note this could be different across levels). (# of scores in each level)
N = 15 (total number of scores)
T1 = 40; T2 = 30; T3 = 20 (X for each treatment condition).
G = 90 (T for the entire study)

20. ANOVA Formulas Final Calculation is the F-ratio:
F = variance between treatments
variance within treatment
where s2 = SS / df
The entire process of ANOVA will take 9 calculations!
3 SS values: SS total; SS btwn; SS w/in
3 df values: df total; df btwn; df w/in
2 variances: MS btwn & MS w/in
F-ratio

21. Analysis of SS Reminder: SS = X2 - ((X)2 / N)
First we calculate the total and then each part; between and within
SStotal = X2 - (G2 / N) = (902 / 15) = 82
SSw/in = SSinside each treatment = = 42
SSbtwn = (T2 / n) - (G2 / N)
= ((402 / 5) + (302 / 5) + (202 / 5) )- (902 / 15)
= = 40
CHECK: SStotal = SSw/in + SSbtwn; 82 =
K = 3 X2 = 622
n = 5 each
N = 15
G = 90

22. Review: Breaking down SS
SStotal
X2 - (G2 / N)
SSw/in
SSinside each treatment
SSbtwn
(T2 / n) - (G2 / N)
Note: So far, we’ve completed 3 of the 9 calculations!

23. Analysis of Degrees of Freedom
Again, first we calculate the total and then each part; between and within.
Remember:
Each df is associated with a specific SS value.
Normally the value of SS is obtained by counting the number of items that were used to calculate SS and then subtract 1.
dftotal = N - 1 (this SS values measure the total variability) = = 14
dfwithin = (n-1) = dfeach treatment = = 12
or N - k = = 12
dfbetween = k - 1 (# T values or samples - 1) = = 2
CHECK: dftotal = dfwithin + dfbetween; 14 =
K = 3 X2 = 622
n = 5 each
N = 15
G = 90

24. Review: Breaking down df
dftotal
N - 1
dfw/in
N - k or dfeach treatment
dfbtwn
k - 1
Note: So far, we’ve completed 6 of the 9 calculations!

25. Analysis of MS and F - ratio
In ANOVA it is convention to use the term mean square or MS instead of variance or s2.
MS = variance = SS / df
MSbetween = SSbetween / dfbetween
= 40 / 2 = 20
MSwithin = SSwithin / dfwithin
= 42 / 12 = 3.5
F = MSbetween / MSwithin
= 20 / 3.5 = 5.7
So we’re greater than 1 which is better than chance, but so far we don’t know about significance

26. You Try It!!
The following data are the number of days it takes to adjust to a time zone change after a long flight. Use ANOVA to determine if jet lag varies for type of travel.
Westbound Eastbound Same Time Zone 2 1 3 4 6 4 8 5 7 1

27. Formulas SStotal = X2 - (G2 / N) SSw/in = SSinside each treatment
SSbtwn = (T2 / n) - (G2 / N)
dftotal = N - 1
dfwithin = (n-1) = dfeach treatment
dfbetween = k - 1
MSbetween = SSbetween / dfbetween
MSwithin = SSwithin / dfwithin
F = MSbetween / MSwithin

28. So, Now What? Hypothesis Testing!
(1) State the hypotheses: Note. The hypotheses are stated in terms of population parameters even though we use sample data to test them. Let’s use our example.
H0 = μ1 = μ2 = μ3 (no treatment effect)
H1 = At least one of the treatment means is different.
In general H1 states that the treatment treatment conditions are not all the same; so there is a real effect.
Note: we do not give any specific alternative hypothesis. This is because many different alternatives are possible and it would be tedious to list them all (& researcher typically have predictions).
e.g. H1 = μ1 = μ2 = μ3 or H1 = μ1 = μ3, but μ2 is different

29. Hypothesis Testing: Setting the critical region
(2) Set the critical region -
But first we need to know something about the distribution of F-ratios.
Because F -ratios are computed from 2 variances (the numerator and denominator), F values will always be positive numbers.
This comes from the fact that in order to calculate SS we have to square values, right?!
When H0 is true the numerator and denominator are both measuring the same variance (chance or error). They should be roughly the same size, so F-ratio should equal 1.
This means scores in our distribution (remember the distribution we test against is the null distribution) should pile up around 1.

30. Distribution of F-ratios
The distribution is cut off at 0.
The means pile up around 1 and then taper off to the right (positively skewed).
Like the t-distribution the exact F distribution depends on df.
Large df provides a more accurate estimate of population variance & F-ratios piled around 1.
Small df will result in a more spread out distribution
Large df
Small df

31. Hypothesis Testing: Setting the Critical Region
(2) Set the critical region. (In our example tailed.)
We need to know the degrees of freedom for our F-ratio.
df between (in our example dfbetween = 2)
df within (in our example dfwithin = 12)
Use the F-distribution table (pg. 693)
df values for the numerator are printed across the top
df values for the denominator are printed in the column on the left-hand side
Our example is said to have “degrees of freedom equal to 2 and 12”. So, critical value is 3.88
NOTE: With ANOVA is it okay to set the critical region after we have calculated our df values in Step 3 (collect data and compute test stat).

32. Hypothesis Testing: Calculate the Test Statistic
(3) Collect the data and calculate the test statistic.
We already did this:
Remember there are 9 calculations involved in computed an ANOVA
3 for SS
3 for df
2 for MS
1 for F-ratio

33. Hypothesis Testing: Make a Decision
The F-ratio we calculated was This is greater than our critical value of 3.88, so we can reject the null hypothesis.
There is a significant treatment effect. But we don’t know which of our means are different. We can only say that at least one difference exists

34. Let’s Do One: Hypothesis Testing with ANOVA
A psychologist would like to examine the relative effectiveness of 3 therapy techniques for treating mild phobias. A sample of N = 18 individuals who display a moderate fear of spiders is obtained. The dependent variable is a measure of reported fear of spiders after therapy.
NOTE: While you can do ANOVA with unequal sample sizes, which is what this example is going to show ANOVA is most accurate when sample sizes are equal. Therefore, when possible researchers should try to equate across samples. However, ANOVA will still be a valid test, especially when sample sizes are large and the discrepancy between sample sizes is not huge.

35. Let’s Do One: Hypothesis Testing with ANOVA
Therapy A
Therapy B
Therapy C 5 2 4 3 2 1 2 3
G = 39
X2 = 121
N = 18
T = 20
SS = 11.33
n = 6
T = 10
SS = 6
n = 5
T = 9
SS = 5.43
n = 7

36. Formulas SStotal = X2 - (G2 / N) SSw/in = SSinside each treatment
SSbtwn = (T2 / n) - (G2 / N)
dftotal = N - 1
dfwithin = (n-1) = dfeach treatment
dfbetween = k - 1
MSbetween = SSbetween / dfbetween
MSwithin = SSwithin / dfwithin
F = MSbetween / MSwithin

37. Effect Size in ANOVA
Simplest and most direct measure is r2, the percentage of variance accounted for by the treatment
r2 = SSbetween treatments
SSbetween treatments + SSwithin treatments (total)
r2 when computed for ANOVA is usually called 2
Compute the effect size for the previous example

38. In the Literature
The means and standard deviations are presented in Table 1. The analysis of variance (ANOVA) revealed a significant difference, F(2, 15) = 4.52, p < .05, 2 = .38

39. Post Hoc Tests
F is an omnibus test: it just says that the means differ, but not which ones. We have to do additional tests to determine.
When are post hoc tests done? As the name implies after an ANOVA
But only after a rejection of the null hypothesis.
Only if there are 3 more more treatments; k 3. If only 2 treatments we can just do a t-test.
Post hoc tests are going to let us go back through our data and compare individual treatments 2 at a time.
So in our original example we can compare: single v. twin, single v. triplets, twins v. triplets.

40. Post Hoc Tests Why can’t we do multiple t-tests?
Risk of Type I error accumulates!
If alpha = .05 then we have a 5% chance of a Type I error or 1 out of 20 chance.
So, for every 20 tests we perform we should expect 1 out of 20 to be a Type I error.
If we perform multiple t-tests we increase the likelihood of Type I error (probability doesn’t simply sum, but does increase).
Experimentwise alpha level: the overall possibility of committing a Type I error over a series of separate hypotheses tests.
Usually the experimentwise alpha level is quite a bit larger than the value of alpha used for any individual test.
Post hoc tests have been designed to control the experimentwise alpha level, so we control the number of Type I errors we are willing to make.

41. Tukey’s Honestly Significant Difference (HSD) Test
HSD - allows us to compute a single value that determines the minimum difference between any two treatments necessary for significance.
This difference is used to compare any 2 treatment conditions
Formula:
HSD = q MSwithin / n
Where q is found in table B.5 page 698.
Tukey assumes that n is the same size across samples (can’t use with ANOVA that uses diff. sample sizes)
Alpha should be the same as the alpha you set in original F-test.

42. Let’s Try HSD HSD = q MSwithin / n
Let’s use our original example with births and language:
n = 5
k = 3
MSwithin = 3.5
dfwithin = df (error term) = 12
M1 = 8, M2 = 6, M3 = 4
TEST EACH
Single v. Twins
Single v. Triplets
Twins v. Triplets

43. You Try One HSD = q MSwithin / n Use your flight data….
Westbound Eastbound Same Time Zone 2 1 3 4
M = 2.5 6 4 8 5 7
M = 6 1
M = .5
n = 6
K = 3

44. Scheffe Test Most Conservative Method:
Uses an F-ratio to test any 2 treatments
The numerator for MS between treatment is recalculated comparing only the 2 treatments you want to look at – denominator stays the same
The denominator for the F-ratio stays the same as overall ANOVA
Although you are only comparing 2 treatment it still using the value of k to calculate the between treatments df (as k increases our critical value increases; see the table)
Critical value for the F-ratio is going to be the same as the overall F-ratio critical.

45. Calculating the Sheffe
Rather than calculate for all possible combinations better to start with the largest difference and then test progressively smaller difference until we find one that is not significant
Single v. triplets
SSbtwn = (T2 / n) - (G2 / N) = 402 / / / 10 = 40
MSbetween = SSbetween / dfbetween = 40 / 2 = 20
F = MSbetween / MSwithin = 20 / 3.5 = 5.71
* Critical was Now do the next one.
k = 3 X2 = 622
n = 5 each N = 15
G = 90

46. Assumptions of Independent Measures ANOVA
The observations within each sample must be independent.
The populations from which the samples are selected must be normal.
The populations from which the samples are selected must have equal variances (homogeneity of variance)

47. F - Max Revisited
F - max is a test of homogeneity of variance (pg. 329).
Compute sample variance for each sample
S2 = SS / df
Select the largest and smallest of the sample variances and compute
F - max = s2 (largest) / s2 (smallest)
F - max assumes that each sample is the same size
Locate critical value in table B.3
K = # of separate samples
Df = n - 1 (because each sample should be the same size)
Look at appropriate alpha level
A large F-max value indicates a large diff. between s2 whereas a small F-max indicates a small diff. between s2.
F-max value needs to be SMALLER than critical value to assume that we have homogeneity of variance.

48. Do F-Max with our Flight Data
n = 6 in each sample
K = 3
Westbound Eastbound Same Time Zone 2 1 3 4
M = 2.5
SS = 5.5 6 4 8 5 7
M = 6
SS = 17 1
M = .5
SS = 1.5

49. Homework
You will be responsible on the exam for reading pg relationship between ANOVA and t tests (pg ).
Chapter 13
1, 3, 4, 5, 6, 8, 9, 12, 17, 18, 19