1. Analysis of Variance ANOVA
Dr. Richard Jackson
In this module we will take a look at another statistical procedure known as the analysis of variance. Quite often in the statistical vernacular it is simply referred to by its acronym ANOVA.
© Mercer University 2005
All Rights Reserved

2. Use of ANOVA Tests Difference in 2 or more Means F = t2 (2 groups)
See Table I K (K-1)/2 or 6 Possible Pairs
Multiple t tests Inappropriate
Ho : m1 = m2 = m3 = m4
Analysis of variance like the t test is a parametric statistic in that it utilizes means and standard deviations in its calculations. Analysis of variance is used to determine if there is a significant difference in two more more means. Generally speaking, if the analysis involves testing the difference in two means, the t test is used. Although the analysis of variance could be used. If only two groups or mean are involved, there is a mathematical relationship between the calculated t and the statistic that is calculated for ANOVA which is f. Specifically if there are two groups being investigated, the mathematical relationship between the statistic calculated and the ANOVA (f) is equal to t squared. Either analysis would lead to the same result. Take a look at the study reported in the supplemental materials you are provided with. It represents the study of data to determine if there is any difference in the testosterone levels among 4 different classifications of smokers. The means in the four groups are listed at the bottom. The analysis would involved in determining if there is any difference occurs in the mean testosterone levels among those 4 groups. A novice statistician may look at this study and think that well I will use the t test comparing non smokers to former smokers, non to light smokers, non to heavy, former to light, former to heavy, and light to heavy smokers. In other words, a total of 6 different t tests. However, when one does that, a egregious error has been made and what happens is, the p value that one quotes is actually a lot larger then it really is. So multiple t tests are inappropriate when you have more than 2 means. Instead analysis of variance is the appropriate statistic to use. The null hypothesis in this case is that the mean of the first group is equal to the mean of the second group is equaled to the mean of the third group which is equaled to the mean of the fourth group. Analysis of variance tells us whether we accept or reject this null hypothesis, like the t test. If we are led to accept the null hypothesis then all of the means are equal and the analysis is complete. However, if analysis of variance leads us to reject this null hypothesis, further analysis is necessary to determine where the difference or differences lie. There are several different pairs that may be investigated. The formula K times K minus 1 divided by 2 tells you the number of pairs of groups that you may compare. For example, in this study we have 4 different groups, so 4 times 4 minus 1 divided by 2 gives you 6. There are 6 possible pairs of means that may differ from the other. For example, non vs. former, non vs. light, non vs. heavy, former vs. light, former vs. heavy, and light vs. heavy. If our analysis of variance leads us to reject the null hypothesis then it means that at least one of those possible pairs of groups is significantly different one from the other. It could be as few as one or might be all 6, but when we are led to reject the null hypothesis with analysis of variance it means at least one of those pairs is significantly different one from the other. A later analysis will tell us which one or ones of those pairs is significantly different.

3. Rationale for Use Inflation of alpha (p-value)
As mentioned earlier the rationale for using the analysis of variance as opposed to multiple t tests is the inflation of the p value. In other words if we do multiple t tests and we state that our p value was a certain value, in actuality it is much higher then that which we state. So if one uses multiple t tests you are likely to be committing a type I error unknowingly. It is kind of like if I knew the probability of being hit by a car if I were to walk across I-85 blind folded was If I walked across I-85 six times the probability of me getting hit by a car is much higher then So the analogy is the same regarding multiple t tests. It inflates the probability of committing a type I error. Unfortunately, this error appears sometimes in the medical literature.

4. Requirements for ANOVA
Continuous Data
Normally Distributed
Non-Parametric Substitute: Kruschal-Wallis H
Like the t test, there are certain requirements associated with analysis of variance. The data must be continuous, normally distributed, and it must be interval or ratio. If the data are skewed or if the data are for example ordinal, then a non-parametric substitute must be used and it is non as the Kruschal-Wallis H. We are not going to the calculation or the use of this aged statistic, just understand if you see this statistic reported in the literature it means the data for an analysis probably would be skewed or ordinal data.

5. Sources of Variation (See Table I)
Between
Within
Total
Take a look at table 1. An analysis of variance involves in calculating three sources of variation or variation of three types. That is known as the between, within, and total variation. The actual calculation of these three items is beyond the scope of this course and in any even the computer would take care of the calculations for us anyway.

6. Between Variation (See Table I)
Variation of 4 Group Means about Grand Mean
0.69, 0.68, 0.57, 0.46, about 0.60
First of all the between variation. The between variation reflects the variation of the 4 groups: non, former, light, and heavy smokers about the grand mean. In other words, the variation of the 4 group means about the mean of all these subjects in the study or the grand mean. I can think you can see that the means 0.69, 0.68, 0.57, and 0.46 vary about the grand mean of 0.60.

7. Within Variation (See Table I)
Variation of each subject about its group mean
Example for Non-Smoking group: How much each subject varies from 0.69
Variation calculated from each group is then summed
Take a look at each individual group. For example, the first one, the non-smoking group. Each of those 10 subjects in that group varies about the mean of 0.69 in that group. In each of those 4 groups there is a variation in each of the 10 subjects about its respective group mean. If you determine the variation within each of these 4 groups for the 10 subjects in each one and sum them, that gives you the within variation of the entire study.

8. Total Variation (See Table I)
Variation of each subject (40) about grand mean of 0.60
Between Variation plus Within Variation equals Total Variation
The between variation, which was determined previously plus the within variation equals the total variation and the total variation is not only equal to the between and the within sum but it reflects the variation of all 40 subjects in the study about the grand mean of 0.60.

9. The ANOVA Summary Table (See Table II)
Source of Variation
Degrees of Freedom
Between K = 3
Within K(N-1) 4(10-1) = 36
Total KN-1 (4)(10)-1 = 39
Table II is a print out showing the computer analysis using the statistics of the data involving testosterone levels. Remember that the null hypothesis is N1 equals N2 equals N3 equals N4. At the top of the page you see an explanation of the three sources of variation: between, within, and total. The first amount of information is headed up by a column labeled DF which stands for degrees of freedom. Degrees of Freedom is specific for the between, within, and total variations and each one has a specific formula. For example, the between degrees of freedom is calculated by the formula K minus 1 where K is the number of groups. So the degrees of freedom for that is 4-1 which is 3. The within variation is K times the quantity N minus 1 where N is the number of subjects in this group, in each group or 10. So 4 times 10 minus 1 is 36. The total variation is KN minus 1 or 4 times 10 minus 1 which is 39. It may be seen that the DF between plus the DF within equals the total DF. You will be given all of these formulas on any exam that you might have.

10. The ANOVA Summary Table (See Table II)
SS “Sums of Squares”
SS is a measure of variability
Calculation Complicated
The next column is labeled SS and the SS stands for Sums of Squares. The sums of squares is the measure of variability which we just described. The calculation is very complicated and we will let the computer always do this for us.

11. The ANOVA Summary Table (See Table II)
MS is “Mean Square”
Obtained by dividing SS by df
The next column is labeled MS which stands for Mean Square. It two is a measure of variation. It is obtained by dividing the sums of squares by the DF for between and within variations. So the mean square between is equal to the sums of squares divided by 3. The mean square within is equal to the sums of squares within divided by 36

12. The ANOVA Summary Table (See Table II)
F is “Test Statistic”
F = MS Between / MS Within
When we get to the test statistic for the analysis of variance, it is referred to as the F value and the F is calculated by dividing the mean square between or by the mean square within or In this case we get an F value of 3.97 and just like the t that was calculated there is a corresponding p value that is associated with each F value that is calculated and just like the t test we accept or reject the null hypothesis based on the p value that is associated with our calculated value of F. The larger the value of F the smaller will be our p value. In this case we get a p value of Using our decision rule, if we had set our A priori max significance level of 0.05 we would then be led to reject the null hypothesis since this is less then 0.05.

13. F Statistic (3.97) p = 0.0152 (See Table II)
Tests Overall Ho
mn = mf = ml = mh
If p  0.05, Accept Ho: Analysis Complete
If p < 0.05, Reject Ho: Post Hoc Tests
If p < 0.05, at Least one Pair Different
The F test and corresponding p value tests the overall null hypothesis, which is that the mean of the non-smokers is equal to the formal smokers which is equal to the light smokers which is equal to the mean of the heavy smokers. If it is greater then 0.05 then we accept the null hypothesis. If it is less then we reject the null hypothesis. If we reject it then it means that one of the six possible means is significantly different one from the other. To determine which one or ones of those six possible pairs of means is different from one another, one performs what is known as a post Hoc test.

14. Post Hoc Tests (Compares Pairs of Means)
Bonferroni
Tukey
Scheffe
Newman – Kewls
LSD
HSD
Duncan
Dunnett
There are many different Post Hoc Tests that one may perform including the bonferroni, tukey, scheffe, newman-kewls, the lsd which stands for least significant difference, and hsd which stands for and this really is the truth, it stands for honestly significant different. There is also the duncan and dunnett test. There are minor reasons to choose one above the other. Just know that these are the various ones that are available to test the individual possible pairs of means to see where the significance lies following a significant F. Recalling that if our p value associated with our calculated F had come out to be greater then 0.05 then the study would have been over and we would have accepted the null hypothesis and conclude that there is no difference among those means. Since our p was less then 0.05, it means at least one of those pairs of means is significantly different from one or the other.

15. Post Hoc Tests These Null Hypotheses (See Table I)
mn = mf
mn = ml
mn = mh
mf = ml
mf – mh
m1 - mh
This slide reflects one of six possible pairs of means that are possible within this analysis and the post hoc test tests the six null hypotheses to see which one or ones that we should reject. In other words we are determining which one these pairs of means is significantly different one from the other.

16. Post Hoc Tests Performed by Statistix (See Table II)
Pairwise comparisons of Means
Group Sharing Common lines are equal
Mn = mf
Mn = ml
Mn  mh
Mf = ml
Mf  mh
Ml = mh
Take a look at table II towards the bottom and you will see each of the 4 groups means listed in descending order. To the right of that, under the heading homogenous groups there are in effect two different lines drawn vertically. They are broken apart in the computer print out, but I have connected them here so you can see them a little bit clearly. Those two lines will identify which groups are not significantly different one from the other. Those groups sharing either one of those common lines are not significantly different from each other. In other words, there is no significant difference between the former, non, and light smokers because they share that first common line. If you take a look at the right line, there is no significant difference between light and heavy smokers. If two means do not share a common line it means that they are not equal. So lets take a look at the six pair wise comparisons that we have. First, the means of the non and former smokers. They share a common line so they are equal. The mean of non vs. the light smokers, they share that first common line so they are equal. The non vs. heavy smokers do not share a common line so there is a significant difference there. In other words, they are unequal. The former vs. the light smokers share a common line so they are equal. The former vs. heavy smokers do not share a common line so there is a significant difference there and the mean of the former is not equal to the mean of the heavy smokers. Lastly, the mean of the light vs. the heavy smokers share a common line so there is no difference there. So in this case, we have performed an analysis of variance. We got a significant overall left and we performed a post hoc test which told us that two of the six possible pairs of comparisons are significantly different and the conclusion would be with regard to testosterone levels there is a significant difference between non and heavy smokers and a significant difference between former smokers and heavy smokers. The other possible comparisons are not statistically different one from the other.

17. Levels and Factors (See Table I)
“Factor” in Example is Smoking Status
“Levels” are the Four Types of the “Factor Investigated”
One Factor, Four Levels
In the vernacular of an analysis of variance several terms need to be defined. Among them being the terms factor and levels. In this example the factor involved is smoking status and there are four different levels of the factor of smoking status: non-smokers, former smokers, light smokers, and heavy smokers. So in this study we are talking about a single factor or one factor analysis of variance and there are four different levels of that factor that is being analyzed. This analysis because it is only involving one factor is known as a single factor analysis of variance.

18. Multi-Factor ANOVA Example Single-Factor ANOVA
Can Analyze More Than One Factor
Example Age (<40, 40)
Multi-Factor ANOVA Allows Analysis of “Interaction” of Factors
It is possible through analysis of variance to examine more than one factor at at time. For this example, in this study while it is a single factor of analysis of variance we could involve another factor such as age. In other words, men less then 40 and men greater then 40 yeas of age. In other words, we take a look at age as a factor in testosterone level as well as smoking status and the benefit of doing that is what is known as a multi factor analysis of variance in it that it allows researchers to determine if there is any interaction between the factors that may produce a significant difference. In other words, it may be the case where in non smokers under the age of 40 may have higher testosterone levels then heavy smokers over the age of 40. I think the interaction situation may be best explained or understood by means of a non clinical example. Suppose it takes 5 hours for a man to build a dog house. The assumption would be that if you had 2 men build the dog house, they could do it in 2.5 hours. But what if those men interacted with each other in an unfavorable way. It could be that one man building a dog house in 5 hours and we have 2 men building a dog house and it may take 6 hours. Or conversely we may have one man building a dog house in 5 hours and if you have 2 men that maybe work synergistic they may be able to build the dog house in 1 hour. So you see that there is an interaction and factors or levels of factors that may produce significant results. I guess from the clinical arena you may see this sometimes when one may be examining the incidence for example of cancer among oral contraceptive users and then you introduce another factor such as smoking and it helps you understand that maybe oral contraceptive users may have a greater incidence of a certain type of cancer but it may be made worse if that person is also a smoker as well. So to reiterate, multi factor analysis of variance beyond the scope of this course is however used in the literature on a limited extent and it does allow for the analysis of the interaction of factors.

19. Summary of ANOVA  2 Means
Following Significant Overall F use Post Hoc Tests
Post Hoc Tests Shows Where Differences Are
To summarize the analysis of variance, it is the statistic of choice when we want to analyze the difference in two or means following a significant overall F or F value that has a corresponding p value that is less then 0.05 wherein we would use a post hoc test. The post hoc test tells us where the significance lies or which pairs of means are significantly different one from the other.