Analysis of Variance Comparisons among multiple populations

More than two populations GroupABCD observations X A,1 X A,2 X A,3 X A,4 … X A,na X B,1 X B,2 X B,3 … X B,nb X C,1 X C,2 X C,3 X C,4 X C,5 … X C,nc X D,1 X D,2 X D,3 X D,4 … X D,nd H 0 : μ A =μ B =μ C =μ D Discuss in the Chapter 8 H0: μ A =μ B H0: μ B =μ C H0: μ C =μ D

Basic assumptions and the hypothesis testing logic The observed data are normally distributed with the same variance (although unknown) σ2. Derive two estimators for σ2 The first is always valid whether the hypothesis, H0: μA=μB=μC=μD, is true or not. The second one is usually greater than the real parameter σ2 when H0: μA=μB=μC=μD is not true. Compare these two estimators (2 nd /1 st ) through the sample. If the ratio is too large, then reject H 0

ANOVA testing ANalysis Of VAriance To test the considered hypothesis by analyzing the variance σ2 Search the proper estimators Decomposition of variance …

E.g., Decomposition of Syy, Syy=SSr+SSm X Y Yi (Yi- ) ( - ) 如果離差越大, 表示 Y^ 不太可能是水平線, 因 為若是水平線, 則差的 平方和將會很小 如果離差越小, 表示 迴歸線越接近真實 值, 預測得越準確 ! Y 的離差, 因給 定 sample 之後 固定不變

Review x 2 distribution X1, X2, X3,…Xn are independent random variables ~N(0,1) () ～ ～ N(0,1) ～

One-way ANOVA approach (i) # of groups # of obs. In each group Total d.f =# of group × obs. in each group The ith group mean Replace the group mean by sample mean ～ Total d.f =# of group ×( obs. in each group-1). Equal group size, n

One-way ANOVA approach (ii) By definition Called as “ within samples sum of squares ” SSw/σ2 ～ ∴ ∵ By definition Called as “ between samples sum of squares ” Group sample mean Total sample mean ∵ ∴ Replaced total mean by total sample mean X.. d.f.=# of group -1 Var(Xi.)=V ar(Xij)/n i,e., assume all Xi. population means are equal to μ in order to replace μ by total sample mean X.. SSb/σ2 ～

One-way ANOVA approach (iii) i.e., the numerator is sufficiently large, while the denominator is smaller =TS Or reject H0 when <α

Decomposition of Var(X ij ) (i) Xij A group B groupC group Total mean X.. A group mean B group mean X B. C group mean Total deviation Between deviation Within deviation If the group difference is smaller, the deviation from the center should be caused by the within randomness.

Decomposition of Var(X ij ) (ii) In usual, define SST as the total sum of squares =

If H0 is not accepted? Xi.~N(μi, σ2/n) Yi~N(μ., σ2/n) Set ∴ & ∵ Xi.=Yi+μi-μ. X..=Y. Within deviation =E[Yi]-E[Y.] =μ.-μ.=0

ANOVA table ∴ SS T= If p-value<α

The meaning of ANOVA table SourceSum of error squares Degree of freedom Mean of SSF Between SS b m-1 (# of group -1) MS b =SS b /(m-1) MS b /MS w Within SS w nm-m= m(n-1) MS w =SS w /(nm-m) total SS T nm-1 (# of observation -1) SST=SSb+SSw, 如果 SSb 越大， SSw 將越小，則在不變的組數 m 之下, MSb 將越大,MSw 越小, 於 是 F 值就越大，越可能 reject H0: 各組平均值無差異。也就是說觀察的變數 Xij 與 X 之總平均數 的差異，大部份肇因於 Xi. 類別平均數之間的差異。

Unbalanced case — unequal sample size within the groups Different group size n i conditional estimator of σ2 unconditional estimator of σ2

Unbalanced F-test for ANOVA A balanced design is suitable over an unbalanced one because of the insensitivity to slight departures from the assumption of equal population variances.

Two classification factors ABCD a X a,A,1 X a,A,2, …, X a,A,k X a,B,1 X a,B,2, …, X a,B,k X a,C,1 X a,C,2, …, X a,C,k X a,D,1 X a,D,2, …, X a,D,k b X b,A,1 X b,A,2 …, X b,A,k X b,B,1 X b,B,2 …, X b,B,k X b,C,1 X b,C,2 …, X b,C,k X b,D,1, X b,D,2 …, X b,Dk C X c,A,1, X c,A,2, …, X c,A,k X c,B,1, X c,B,2, …, X c,B,k X c,C,1, X c,C,2, …, X c,C,k X c,D,1, X c,D,2, …, X c,D,k Column factor Row factor

Two-way ANOVA approach (i) m types n types Review (α i =μ i -μ, the deviation from total μ) (∵(∵ ) Only one observation within each cell

Two-way ANOVA approach (ii) The cell mean of size k or other Supposed an additive model for cell mean, composed by a i and b j The ith row mean The jth column mean The total mean Average row factor Average column factor The ith row mean=the average column factor+ the specific ith row factor jj Deviation from average row factor, column factor &

Two-way ANOVA approach (iii)

Two-way ANOVA approach (iv) i.e., The expected value of specific ij cell could be decomposed into: Total mean + the ith deviation from average row factor (the ith row deviation from the total mean) + the jth deviation from average column factor (the jth column deviation from total mean) & Use the unbiased estimators to test the objective hypothesis The assumed two-way ANOVA model

Two-way ANOVA approach (v) Apply each unbiased estimator ? Reduced n-1 d.f. Reduced m-1 d.f. Reduced 1 d.f.

Two-way ANOVA approach (vi) If is true then Define the row sum of squares 2

Two way ANOVA table =m

Two-way ANOVA with interaction (i)

Two-way ANOVA with interaction (ii)

Two-way ANOVA with interaction (iii)

Two-way ANOVA with interaction (iv)

Two-way ANOVA with interaction (v)

Two-way ANOVA with interaction (vi)

Homework #3 Problem 5, 15, 19, 20, 25