Lecture 13 Multiple comparisons for one-way ANOVA (Chapter 15.7) Analysis of Variance Experimental Designs (Chapter 15.3)

15.7 Multiple Comparisons When the null hypothesis is rejected, it may be desirable to find which mean(s) is (are) different, and how they rank. Three statistical inference procedures, geared at doing this, are presented: Fisher’s least significant difference (LSD) method Bonferroni adjustment to Fisher’s LSD Tukey’s multiple comparison method

Example 15.1 Sample means: Does the quality strategy have a higher mean sales than the other two strategies? Do the quality and price strategies have a higher mean than the convenience strategy? Does the price strategy have a smaller mean sales than quality but a higher mean than convenience? • Pairwise comparison: Are two population means different?

Fisher Least Significant Different (LSD) Method This method builds on the equal variances t-test of the difference between two means. The test statistic is improved by using MSE rather than sp2. We conclude that mi and mj differ (at a% significance level if > LSD, where

Multiple Comparisons Problem A hypothetical study of the effect of birth control pills is done. Two groups of women (one taking birth controls, the other not) are followed and 20 variables are recorded for each subject such as blood pressure, psychological and medical problems. After the study, two-sample t-tests are performed for each variable and it is found that one null hypothesis is rejected. Women taking birth pills have higher incidences of depression at the 5% significance level (the p-value equals .02). Does this provide strong evidence that women taking birth control pills are more likely to be depressed?

Experimentwise Type I error rate (aE) versus Comparisonwise Type I error rate The comparisonwise Type I error rate is the probability of committing a Type I error for one pairwise comparison. The experimentwise Type I error rate ( ) is the probability of committing at least one Type I error when C tests are done and all null hypotheses are true. For a one-way ANOVA, there are k(k-1)/2 pairwise comparisons (k=number of populations) If the comparisons are not planned in advance and chosen after looking at the data, the experimentwise Type I error rate is the more appropriate one to look at.

Experimentwise Error Rate The expected number of Type I errors if C tests are done at significance level each is If C independent tests are done, aE = 1-(1 – a)C The Bonferroni adjustment determines the required Type I error probability per test (a) , to secure a pre-determined overall aE.

Bonferroni Adjustment Suppose we carry out C tests at significance level If the null hypothesis for each test is true, the probability that we will falsely reject at least one hypothesis is at most Thus, if we carry out C tests at significance level , the experimentwise Type I error rate is at most

Bonferroni Adjustment for ANOVA The procedure: Compute the number of pairwise comparisons (C) [all: C=k(k-1)/2], where k is the number of populations. Set a = aE/C, where aE is the true probability of making at least one Type I error (called experimentwise Type I error). We conclude that mi and mj differ at a/C% significance level (experimentwise error rate at most ) if

Fisher and Bonferroni Methods Example 15.1 - continued Rank the effectiveness of the marketing strategies (based on mean weekly sales). Use the Fisher’s method, and the Bonferroni adjustment method Solution (the Fisher’s method) The sample mean sales were 577.55, 653.0, 608.65. Then,

Fisher and Bonferroni Methods Solution (the Bonferroni adjustment) We calculate C=k(k-1)/2 to be 3(2)/2 = 3. We set a = .05/3 = .0167, thus t.0167/2, 60-3 = 2.467 (Excel). Again, the significant difference is between m1 and m2.

Tukey Multiple Comparisons The test procedure: Assumes equal number of obs. per populations. Find a critical number w as follows: k = the number of populations n =degrees of freedom = n - k ng = number of observations per population a = significance level qa(k,n) = a critical value obtained from the studentized range table (app. B17/18)

Tukey Multiple Comparisons Select a pair of means. Calculate the difference between the larger and the smaller mean. If there is sufficient evidence to conclude that mmax > mmin . Repeat this procedure for each pair of samples. Rank the means if possible. If the sample sizes are not extremely different, we can use the above procedure with ng calculated as the harmonic mean of the sample sizes.

Tukey Multiple Comparisons Example 15.1 - continued We had three populations (three marketing strategies). K = 3, Sample sizes were equal. n1 = n2 = n3 = 20, n = n-k = 60-3 = 57, MSE = 8894. Take q.05(3,60) from the table: 3.40. Population Sales - City 1 Sales - City 2 Sales - City 3 Mean 577.55 653 698.65 City 1 vs. City 2: 653 - 577.55 = 75.45 City 1 vs. City 3: 608.65 - 577.55 = 31.1 City 2 vs. City 3: 653 - 608.65 = 44.35

15.3 Analysis of Variance Experimental Designs Several elements may distinguish between one experimental design and another: The number of factors (1-way, 2-way, 3-way,… ANOVA). The number of factor levels. Independent samples vs. randomized blocks Fixed vs. random effects These concepts will be explained in this lecture.

Number of factors, levels Example: 15.1, modified Methods of marketing: price, convenience, quality => first factor with 3 levels Medium: advertise on TV vs. in newspapers => second factor with 2 levels This is a factorial experiment with two “crossed factors” if all 6 possibilities are sampled or experimented with. It will be analyzed with a “2-way ANOVA”. (The book got this term wrong.)

One - way ANOVA Single factor Two - way ANOVA Two factors Response Response Treatment 3 (level 1) Treatment 2 (level 2) Treatment 1 (level 3) Level 3 Level2 Factor A Level 1 Level2 Level 1 Factor B

Randomized blocks This is something between 1-way and 2-way ANOVA: a generalization of matched pairs when there are more than 2 levels. Groups of matched observations are collected in blocks, in order to remove the effects of unwanted variability. => We improve the chances of detecting the variability of interest. Blocks are like a second factor => 2-way ANOVA is used for analysis Ideally, assignment to levels within blocks is randomized, to permit causal inference.

Randomized blocks (cont.) Example: expand 13.03 Starting salaries of marketing and finance MBAs: add accounting MBAs to the investigation. If 3 independent samples of each specialty are collected (samples possibly of different sizes), we have a 1-way ANOVA situation with 3 levels. If GPA brackets are formed, and if one samples 3 MBAs per bracket, one from each specialty, then one has a blocked design. (Note: the 3 samples will be of equal size due to blocking.) Randomization is not possible here: one can’t assign each student to a specialty, and one doesn’t know the GPA beforehand for matching. => No causal inference.

Models of fixed and random effects Fixed effects If all possible levels of a factor are included in our analysis or the levels are chosen in a nonrandom way, we have a fixed effect ANOVA. The conclusion of a fixed effect ANOVA applies only to the levels studied. Random effects If the levels included in our analysis represent a random sample of all the possible levels, we have a random-effect ANOVA. The conclusion of the random-effect ANOVA applies to all the levels (not only those studied).

Models of fixed and random effects (cont.) Fixed and random effects - examples Fixed effects - The advertisement Example (15.1): All the levels of the marketing strategies considered were included. Inferences don’t apply to other possible strategies such as emphasizing nutritional value. Random effects - To determine if there is a difference in the production rate of 50 machines in a large factory, four machines are randomly selected and the number of units each produces per day for 10 days is recorded.

15.4 Randomized Blocks Analysis of Variance The purpose of designing a randomized block experiment is to reduce the within-treatments variation, thus increasing the relative amount of between treatment variation. This helps in detecting differences between the treatment means more easily.

Examples of Randomized Block Designs Factor Response Units Block Varieties of Corn Yield Plots of Land Adjoining plots Blood pressure Drugs Hypertension Patient Same age, sex, overall condition Management style Worker productivity Amount produced by worker Shifts

Randomized Blocks Block all the observations with some commonality across treatments Treatment 4 Treatment 3 Treatment 2 Treatment 1 Block3 Block2 Block 1

Randomized Blocks Block all the observations with some commonality across treatments

Partitioning the total variability The sum of square total is partitioned into three sources of variation Treatments Blocks Within samples (Error) Recall. For the independent samples design we have: SS(Total) = SST + SSE SS(Total) = SST + SSB + SSE Sum of square for treatments Sum of square for blocks Sum of square for error

Sums of Squares Decomposition = observation in ith block, jth treatment = mean of ith block = mean of jth treatment

Calculating the sums of squares Formulas for the calculation of the sums of squares SSB= SST =

Calculating the sums of squares Formulas for the calculation of the sums of squares SSB= SST =

Mean Squares To perform hypothesis tests for treatments and blocks we need Mean square for treatments Mean square for blocks Mean square for error

Test statistics for the randomized block design ANOVA Test statistic for treatments Test statistic for blocks df-T: k-1 df-B: b-1 df-E: n-k-b+1

The F test rejection regions Testing the mean responses for treatments F > Fa,k-1,n-k-b+1 Testing the mean response for blocks F> Fa,b-1,n-k-b+1

Randomized Blocks ANOVA - Example Are there differences in the effectiveness of cholesterol reduction drugs? To answer this question the following experiment was organized: 25 groups of men with high cholesterol were matched by age and weight. Each group consisted of 4 men. Each person in a group received a different drug. The cholesterol level reduction in two months was recorded. Can we infer from the data in Xm15-02 that there are differences in mean cholesterol reduction among the four drugs?

Randomized Blocks ANOVA - Example Solution Each drug can be considered a treatment. Each 4 records (per group) can be blocked, because they are matched by age and weight. This procedure eliminates the variability in cholesterol reduction related to different combinations of age and weight. This helps detect differences in the mean cholesterol reduction attributed to the different drugs.

Randomized Blocks ANOVA - Example Conclusion: At 5% significance level there is sufficient evidence to infer that the mean “cholesterol reduction” gained by at least two drugs are different. Treatments Blocks b-1 K-1 MST / MSE MSB / MSE