MANOVA Multivariate Analysis of Variance

One way Analysis of Variance (ANOVA) Comparing k Populations

The F test – for comparing k means Situation We have k normal populations Let  i and  denote the mean and standard deviation of population i. i = 1, 2, 3, … k. Note: we assume that the standard deviation for each population is the same.  1 =  2 = … =  k = 

We want to test against

Computing Formulae: Compute 1) 2) 3) 4) 5)

The data Assume we have collected data from each of k populations Let x i1, x i2, x i3, … denote the n i observations from population i. i = 1, 2, 3, … k.

Then 1) 2) 3)

Sourced.f.Sum of Squares Mean Square F-ratio Betweenk - 1SS Between MS Between MS B /MS W WithinN - kSS Within MS Within TotalN - 1SS Total Anova Table

Example In the following example we are comparing weight gains resulting from the following six diets 1.Diet 1 - High Protein, Beef 2.Diet 2 - High Protein, Cereal 3.Diet 3 - High Protein, Pork 4.Diet 4 - Low protein, Beef 5.Diet 5 - Low protein, Cereal 6.Diet 6 - Low protein, Pork

Thus Thus since F > we reject H 0

Sourced.f.Sum of Squares Mean Square F-ratio Between ** (p = ) Within Total Anova Table * - Significant at 0.05 (not 0.01) ** - Significant at 0.01

Equivalence of the F-test and the t-test when k = 2 the t-test

the F-test

Hence

Factorial Experiments Analysis of Variance

Dependent variable Y k Categorical independent variables A, B, C, … (the Factors) Let –a = the number of categories of A –b = the number of categories of B –c = the number of categories of C –etc.

The Completely Randomized Design We form the set of all treatment combinations – the set of all combinations of the k factors Total number of treatment combinations –t = abc…. In the completely randomized design n experimental units (test animals, test plots, etc. are randomly assigned to each treatment combination. –Total number of experimental units N = nt=nabc..

The treatment combinations can thought to be arranged in a k-dimensional rectangular block A 1 2 a B 12b

A B C

The Completely Randomized Design is called balanced If the number of observations per treatment combination is unequal the design is called unbalanced. (resulting mathematically more complex analysis and computations) If for some of the treatment combinations there are no observations the design is called incomplete. (In this case it may happen that some of the parameters - main effects and interactions - cannot be estimated.)

Example In this example we are examining the effect of We have n = 10 test animals randomly assigned to k = 6 diets The level of protein A (High or Low) and the source of protein B (Beef, Cereal, or Pork) on weight gains (grams) in rats.

The k = 6 diets are the 6 = 3×2 Level-Source combinations 1.High - Beef 2.High - Cereal 3.High - Pork 4.Low - Beef 5.Low - Cereal 6.Low - Pork

Table Gains in weight (grams) for rats under six diets differing in level of protein (High or Low) and s ource of protein (Beef, Cereal, or Pork) Level of ProteinHigh ProteinLow protein Source of ProteinBeefCerealPorkBeefCerealPork Diet Mean Std. Dev

Source of Protein Level of Protein BeefCerealPork High Low Treatment combinations Diet 1Diet 2Diet 3 Diet 4Diet 5Diet 6

Level of ProteinBeefCerealPorkOverall Low Source of Protein High Overall Summary Table of Means

Profiles of the response relative to a factor A graphical representation of the effect of a factor on a reponse variable (dependent variable)

Profile Y for A Y Levels of A a 123 … This could be for an individual case or averaged over a group of cases This could be for specific level of another factor or averaged levels of another factor

Profiles of Weight Gain for Source and Level of Protein

Example – Four factor experiment Four factors are studied for their effect on Y (luster of paint film). The four factors are: Two observations of film luster (Y) are taken for each treatment combination 1) Film Thickness - (1 or 2 mils) 2)Drying conditions (Regular or Special) 3)Length of wash (10,30,40 or 60 Minutes), and 4)Temperature of wash (92 ˚C or 100 ˚C)

The data is tabulated below: Regular DrySpecial Dry Minutes92  C100  C92  C100  C 1-mil Thickness mil Thickness

Definition: A factor is said to not affect the response if the profile of the factor is horizontal for all combinations of levels of the other factors: No change in the response when you change the levels of the factor (true for all combinations of levels of the other factors) Otherwise the factor is said to affect the response:

Profile Y for A – A affects the response Y Levels of A a 123 … Levels of B

Profile Y for A – no affect on the response Y Levels of A a 123 … Levels of B

Definition: Two (or more) factors are said to interact if changes in the response when you change the level of one factor depend on the level(s) of the other factor(s). Profiles of the factor for different levels of the other factor(s) are not parallel Otherwise the factors are said to be additive. Profiles of the factor for different levels of the other factor(s) are parallel.

Interacting factors A and B Y Levels of A a 123 … Levels of B

Additive factors A and B Y Levels of A a 123 … Levels of B

If two (or more) factors interact each factor effects the response. If two (or more) factors are additive it still remains to be determined if the factors affect the response In factorial experiments we are interested in determining –which factors effect the response and – which groups of factors interact.

The testing in factorial experiments 1.Test first the higher order interactions. 2.If an interaction is present there is no need to test lower order interactions or main effects involving those factors. All factors in the interaction affect the response and they interact 3.The testing continues with for lower order interactions and main effects for factors which have not yet been determined to affect the response.

Models for factorial Experiments

The Single Factor Experiment Situation We have t = a treatment combinations Let  i and  denote the mean and standard deviation of observations from treatment i. i = 1, 2, 3, … a. Note: we assume that the standard deviation for each population is the same.  1 =  2 = … =  a = 

The data Assume we have collected data for each of the a treatments Let y i1, y i2, y i3, …, y in denote the n observations for treatment i. i = 1, 2, 3, … a.

The model Note: where has N(0,  2 ) distribution (overall mean effect) (Effect of Factor A) Note:by their definition.

Model 1: y ij (i = 1, …, a; j = 1, …, n) are independent Normal with mean  i and variance  2. Model 2: where  ij (i = 1, …, a; j = 1, …, n) are independent Normal with mean 0 and variance  2. Model 3: where  ij (i = 1, …, a; j = 1, …, n) are independent Normal with mean 0 and variance  2 and

The Two Factor Experiment Situation We have t = ab treatment combinations Let  ij and  denote the mean and standard deviation of observations from the treatment combination when A = i and B = j. i = 1, 2, 3, … a, j = 1, 2, 3, … b.

The data Assume we have collected data (n observations) for each of the t = ab treatment combinations. Let y ij1, y ij2, y ij3, …, y ijn denote the n observations for treatment combination - A = i, B = j. i = 1, 2, 3, … a, j = 1, 2, 3, … b.

The model Note: where has N(0,  2 ) distribution and

The model Note: where has N(0,  2 ) distribution Note:by their definition.

Model : where  ijk (i = 1, …, a; j = 1, …, b ; k = 1, …, n) are independent Normal with mean 0 and variance  2 and Main effects Interaction Effect Mean Error

Maximum Likelihood Estimates where  ijk (i = 1, …, a; j = 1, …, b ; k = 1, …, n) are independent Normal with mean 0 and variance  2 and

This is not an unbiased estimator of  2 (usually the case when estimating variance.) The unbiased estimator results when we divide by ab(n -1) instead of abn

The unbiased estimator of  2 is where

Testing for Interaction: where We want to test: H 0 : (  ) ij = 0 for all i and j, against H A : (  ) ij ≠ 0 for at least one i and j. The test statistic

We reject H 0 : (  ) ij = 0 for all i and j, If

Testing for the Main Effect of A: where We want to test: H 0 :  i = 0 for all i, against H A :  i ≠ 0 for at least one i. The test statistic

We reject H 0 :  i = 0 for all i, If

Testing for the Main Effect of B: where We want to test: H 0 :  j = 0 for all j, against H A :  j ≠ 0 for at least one j. The test statistic

We reject H 0 :  j = 0 for all j, If

The ANOVA Table SourceS.S.d.f.MS =SS/dfF ASS A a - 1MS A MS A / MS Error BSS B b - 1MS B MS B / MS Error ABSS AB (a - 1)(b - 1)MS AB MS AB / MS Error ErrorSS Error ab(n - 1)MS Error TotalSS Total abn - 1

Computing Formulae

Manova Continued