ANOVA Determining Which Means Differ in Single Factor Models Determining Which Means Differ in Single Factor Models

Single Factor Models Review of Assumptions Recall that the problem solved by ANOVA is to determine if at least one of the true mean values of several different treatments differs from the others. For ANOVA we assumed: population normal 1.The distribution of the population for each treatment is normal. standard deviations equal 2.The standard deviations of each population, although unknown, are equal. randomindependent 3.Sampling is random and independent.

Determining Which Means Differ Basic Concept Suppose the result of performing a single factor ANOVA test is a low p-value, which indicates that at least one population mean does, in fact, differ from the others. The natural question is, “Which differ?” The answer is that we conclude that two population means differ if their two sample means differ by “a lot”. –The statistical question is, “What is ‘a lot’?”

Example The length of battery life for notebook computers is of concern to computer manufacturers. Toshiba is considering 5 different battery models (A, B, C, D, E) that have different costs. The question is, “Is there enough evidence to show that average battery life differs among battery types?”

Data A B C D E  x = Grand Mean  x =120

OUTPUT p-value = Can conclude differences

Motivation for The Fisher Procedure Fisher’s Procedure is a natural extension of the comparison of two population means when the unknown variances are assumed to be equal –Recall this is an assumption in single factor ANOVA Testing for the difference of two population means (with equal but unknown σ’s) has the form: H 0 : μ 1 – μ 2 = 0 H A : μ 1 – μ 2 ≠ 0 Reject H 0 (Accept H A ) if: What do we use for these two values?

Best Estimate for σ 2 and the Appropriate Degrees of Fredom Recall that when there were only 2 populations, the best estimate for σ 2 is s p 2 and the degrees of freedom is (n 1 -1) + (n 2 -1) or n 1 + n For ANOVA, using all the information from the k populations the best estimate for σ 2 is MSE and the degrees of freedom is DFE. Two populations With Equal VariancesANOVA Best estimate for σ 2 s p 2 MSE Degrees of Freedom n 1 + n 2 – 2 DFE

Two Types of Tests There are two types of tests that can be applied: 1.A test or a confidence interval for the difference in two particular means e.g. µ E and µ B 2.A set of tests which determine differences among all means. This is called a set of experiment wise (EW) tests. The approach is the same. Fisher LSD approach –We will illustrate an approach called the Fisher LSD approach. α will be different –Only the value used for α will be different.

Determining if μ i Differs From μ j Fisher’s LSD Approach H 0 : μ i – μ j = 0 H A : μ i – μ j ≠ 0 Reject H 0 (Accept H A ) if: That is, we conclude there is a differences between μ i and μ j if LSD LSD stands for “Least Significant Difference”

When Do We Conclude Two Treatment Means (µ i and µ j ) Differ? We conclude that two means differ, if their sample means,  x i and  x j, differ by “a lot”. “A lot” is LSD given by:

Confidence Intervals for the Difference in Two Population Means Confidence Intervals for the Difference in Two Population Means A confidence interval for μ i – μ j is found by: Confidence Interval for μ i – μ j

Equal vs. Unequal Sample Sizes If the sample sizes drawn from the various populations differ, then the denominator of the t-statistic will be different for each pairwise comparison. But if the sample sizes are equal (n 1 = n 2 = n 3 = ….), we can designate the equal sample size by N Then the t-test becomes: Reject H 0 (Accept H A ) if: LSD

LSD For Equal Sample Sizes

What Do We Use For α? Recall that α is In Hypothesis Tests: the probability of concluding that there is a difference when there is not. In Confidence Intervals: the probability the interval will not contain the true difference in mean values If doing a single comparison test or constructing a confidence interval, For an experimentwise comparison of all means, We will actually be conducting 10 t-tests: (1) μ E - μ D, (2) μ E - μ C, (3) μ E - μ B, (4) μ E - μ A, (5) μ D - μ C, (6) μ D - μ B, (7) μ D - μ A, (8) μ C – μ B, (9) μ C - μ A, (10) μ B - μ A select α as usual Use α EW

α EW = The probability of Making at least one Type I Error (making a Type I error) =α.Suppose each test has a probability of concluding that there is a difference when there is not (making a Type I error) = α. probability of not making a Type I error is 1-α. –Thus for each test, the probability of not making a Type I error is 1-α. (1- α) 10So the probability of not making any Type I errors on any of the 10 tests is: (1- α) 10 For α =.05, this is (.95) 10 =.5987 probability of making at least one Type I error in this experimentα EW.The probability of making at least one Type I error in this experiment, is denoted by α EW. α EW = = the probability over 40%!Here, α EW = = That is, the probability we make at least one mistake is now over 40%! lower α EWTo have a lower α EW, α for each test must be significantly reduced.

The Bonferroni Adjustment for α To make α EW reasonable, say.05, α for each test must be reduced. The Bonferroni Adjustment is as follows: NOTE: decreasing α, increases β, the probability of not concluding that there is a difference between two means when there really is. Thus, some researchers are reluctant to make α too small because this can result in very high β values. α for each Test For an experimentwise value, α EW, for each test use α = α EW /c c = number of tests For k treatments, c = k(k-1)/2

What Should α for Each Test Be? For α EW =.10 Number of Treatments, k α for each test For α EW =.05 Number of Treatments, k α for each test The required α values for the individual t-tests for α EW =.05 and α EW =.10 are:

LSD EW For Multiple Comparison Tests When doing the series of multiple comparison tests to determine which means differ, the test would be to conclude that µ i differs from µ j if : Where LSD EW is given by: When n i ≠ n j When n i = n j = N α for each test

Procedure for Testing Differences Among All Means We begin by calculating LSD EW which we have shown will not change from test to test if the sample sizes are the same from each sample. That is the situation in the battery example that we illustrate here. –A different LSD would have to be calculated for each comparison if the sample sizes are different.

Procedure (continued) Then construct a matrix as follows

Procedure (continued) Fill in mean of each treatment across the top row and down the left-most column; (in our example, X A = , X B = 90, X C = 100, X D = 140, X E = 150 )

Procedure (continued) For each cell below the main diagonal, compute the absolute value of difference of the means in the corresponding column and row

Procedure (continued) Compare each difference with LSD Ew ( in our case). If the difference between and > LSD Ew. we can conclude that there is difference in µ i and µ j.

Tests For the Battery Example For the battery example, 1.Which average battery lives can we conclude differ? 2.Give a 95% confidence interval for the difference in average battery lives between: C batteries and B batteries E batteries and B batteries Use LSD EW Multiple Comparisons Use LSD Individual Comparisons

Battery Example Calculations Battery Example Calculations Experimental error of  EW =.05 For k = 5 populations, α = α EW /10 =.05/10 =.005 From the Excel output:  x E = 150,  x D = 140,  x A = ,  x c = 100,  x B = 90 MSE = , DFE = 25, N = 6 from each population Use TINV(.005,25) to generate t.0025,25 =

Analysis of Which Means Differ We conclude that two population means differ if their sample means differ by more than LSD EW = Construct a matrix of differences, Compare with LSD EW

Conclusion of Comparisons

LSD For Confidence Intervals Confidence intervals for the difference between two mean values, i and j, are of the form: (Point Estimate) ± t  /2,DFE (Standard Error)  =.05 LSD (not LSD EW ) Confidence Interval for

LSD for Battery Example For the battery example:

The Confidence Intervals The Confidence Intervals 95% confidence interval for the difference in mean battery lives between batteries of type C and batteries of type B. 95% confidence interval for the difference in mean battery lives between batteries of type E and batteries of type B. Confidence Interval for μ C – μ B Confidence Interval for μ E – μ B

REVIEW The Fisher LSD Test –What to use for: Best Estimate of σ 2 = MSE Degrees of Freedom = DFE –Calculation of LSD Bonferroni Modification –Modify α so that α EW is reasonable –α = α EW /c, where the # of tests, c = k(k-1)/2 –Calculation of LSD EW Excel Calculations