Basics of ANOVA Why ANOVA Assumptions used in ANOVA Various forms of ANOVA Simple ANOVA tables Interpretation of values in the table Exercises

Why ANOVA If we have two samples then under mild conditions we can use t-test to test if difference between means is significant. When there are more than two sample then using t-test might become unreliable. ANalysis Of VAriances – ANOVA is designed to test differences between means in many sample cases. Examples of ANOVA: Suppose that we want test effect of various exercises on weight loss. We want to test 5 different exercises. We recruit 20 men and assign for each exercises four of them. After few weeks we record weight loss. Let us denote i=1,2,3,4,5 as exercise number and j=1,2,3,4 person’s number. Then Y ij is weight loss for jth person on the ith exercise programme. It is one-way balanced ANOVA. One way because we have only one category (exercise programme). Balanced because we have exactly same number of men on each exercise programme. Another example: Now we want to subdivide each exercises into 4 subcategories. For each subcategory of the exercise we recruit four men. We measure weight loss after few weeks. i – exercise category j – exercise subcategory k – kth men. Then Y ijk is weight loss for kth men in the jth subcategory of ith category. Number of observations is 5x4x4 = 80. It is two-fold nested ANOVA. We want to test: a) There is no significant differences between categories; b) there is no significant difference between different subcategories

Examples of ANOVA One more example: We have 5 categories of exercises and 4 categories of diets. We hire for each exercise and category 4 persons. There will be 5x4x4=80 men. It is two way crossed ANOVA. Two-way because we have categorised men in two ways: exercises and diets. This model is also balanced: we have exactly same number of men for each exercise-diet. i – exercise number j – diet number k – kth person Y ijk – kth person in the ith exercise and jth diet. In this case we can have two different type of hypothesis testing. Assume that mean for each exercise-diet combination is  ij. If we assume that model is additive, i.e. effect of exercise and diet adds up then we have:  ij =  i +  j.  i is the effect of ith exercise and  j the effect if diet. Then we want to test following hypotheses: a)  ij does not depend on exercise and b)  ij does not depend on diet. Sometimes we do not want to assume additivity. Then we want to test one more hypothesis: model is additive. If model is not additive then there might be some problems of interpretations with other hypotheses. In this case it might be useful to use transformation to make the model additive. Models used for ANOVA can be made more and more complicated. We can design three, four ways crossed models or nested models. We can combine nested and crossed models together. Number of possible ANOVA models is very large.

Assumptions ANOVA models are special cases of the linear models. We can write the model as: Where Y is the observation vector,  -is vector of the means composed of the treatement means and  is the error vector. Basic assumptions in ANOVA models are: 1.Expected values of the errors are 0 2.Variance of all errors are equal to each other 3.Errors are independent 4.Errors are normally distributed All ANOVA treatments are very sensitive to assumptions 1)-3). F-tests meant to be robust against the assumption 4). If assumptions 1)-3) are valid then 4) will always be valid at least asymptotically. I.e. for large number of the observations

ANOVA tables Standard ANOVA tables look like Where v 1,,,v p are values we want to test if they are 0. df is degrees of freedom corresponding to this value. SSh is sum of the squares corresponding to this value (h denotes hypothesis). F is F-value we want to test. Its degrees of freedom is (d i,d e ). Prob is corresponding probability. If probability is very low then we reject hypothesis that this value is 0. If the value for prob is high enough then we do not reject null-hypothesis that this value is 0. These values are calculated using likelihood ratio test. Let us say we want to test hypothesis: H 0: v i =0 vs H 1 :v i  0 Then we maximise likelihood under null hypothesis find corresponding variance then we maximise the likelihood under alternative hypothesis and find corresponding variance. Then we calculate sum of the squares for null and alternative hypothesis find F-statistics effectdfSShMSFprob v1v1 d1d1 SS 1 MS 1 =SS 1 /d 1 MS 1 /MS e pr 1... vpvp dpdp SS p MS p =SS p /d p MS p /MS e pr p errordede SS e MS e =SS e /d e totalNSS t

LR test for ANOVA Suppose variances are: Then mean sum of the squares for the null and alternative hypotheses as: Since first sum of squares is  2 with degrees of freedom dfh and the second sum of squares is  2 with degrees of freedom dfe and they are independent then their ratio has F-distribution with degrees of freedom (dfh,dfe). Degrees of freedom of hypothesis is found using number of elements in the category-1 in the simplest case. Using this type of ANOVA tables we can only tell if there is significant differences between means. It does not tell which one is significantly different. This ratio has F distribution if null-hypothesis is true. Otherwise it has non-central F-distribution.

Example: Two way ANOVA Let us consider an example taken from Box, Hunter and Hunter. Experiment was done on animals. Survival times of the animals for various poisons and treatment was tested. Table is: treatment A B C D poisons I II III

ANOVA table ANOVA table produced by R: Df Sum Sq Mean Sq F value Pr(>F) pois e-07 *** treat e-06 *** pois:treat Residuals Most important values are F and Pr(>F). In SPSS Pr is replaced by sig. In this table we have tests for pois. and treat. Moreover we have “interaction” between these two categories. Interaction means that it would be difficult to separate effects of these two categories. They should be considered simultaneously. Pr. for interaction is not very small and it is not large enough to discard interaction effects. In these situations transformation of the variables might help. Let us consider ANOVA table for the transformed observations. Let us use transformation 1/y. Now ANOVA table looks like: Df Sum Sq Mean Sq F value Pr(>F) pois e-13 *** treat e-09 *** pois:treat Residuals

ANOVA table According to this table Pr. corresponding to the interaction term is high. It means that interaction for the transformed variables is not significant. We could reject interaction terms. We can build the ANOVA table without the interactions. It will look like: Df Sum Sq Mean Sq F value Pr(>F) pois e-14 *** treat e-10 *** Residuals Now we can say that there is significant differences between poisons and treatments. Sometimes it is wise to use transformation to reduce effect of interactions. For this several different transformations (inverse, inverse square, log) could be used. For each of them ANOVA tables could be built. Then by inspection you can decide which transformation gives better results. Following argument could be used to justify transformation. If effects of two different categories is multiplicative then log of them will have additive effect. It is easier to interpret additive effects than others.