1. Linear Contrasts and Multiple Comparisons (§ 8.6)
One-way classified design AOV example.
Develop the concept of multiple comparisons and linear contrasts.
Multiple comparisons methods needed due to potentially large number of comparisons that may be made if Ho rejected in the one-way AOV test.
Fisher’s Protected LSD
Tukey’s W MCP
Studentized range distribution
Experimentwise error rate
Comparisonwise error rate
Student-Newman-Keuls procedure
Duncan’s new MRT
Waller-Duncan k-ratio MCP
Terms:
One-way classification
Linear Contrasts
Multiple comparisons
Data dredging
Mutually orthogonal contrasts
STA MCP

2. One-Way Layout Example
A study was performed to examine the effect of a new sleep inducing drug on a population of insomniacs. Three (3) treatments were used:
Standard Drug
New Drug
Placebo (as a control)
What is the role of the placebo in this study?
What is a control in an experimental study?
18 individuals were drawn (at random) from a list of known insomniacs maintained by local physicians. Each individual was randomly assigned to one of three groups. Each group was assigned a treatment. Neither the patient nor the physician knew, until the end of the study, which treatment they were on (double-blinded).
Why double-blind?
A proper experiment should be:
randomized, controlled, and double-blinded.
STA MCP

3. Average number of hours of sleep per night.
Response:
Average number of hours of sleep per night.
Placebo: 6.5, 5.7, 5.1, 3.8, 4.6, 5.1
Standard Drug: 8.4, 8.2, 8.8, 7.1, 7.2, 8.0
New Drug: 10.6, 6.6, 8.0, 8.0, 6.8, 6.6
yij = response for the j-th individual on the i-th treatment.
Hartley’s test for equal variances:
Fmax = 4.77 < Fmax_critical = 10.8
STA MCP

4. Excell Analysis Tool Output
What do we conclude here?
STA MCP

5. Linear Contrasts and Multiple Comparisons
If we reject H0 of no differences in treatment means in favor of HA, we conclude that at least one of the t population means differs from the other t-1.
Which means differ from each other?
Multiple comparison procedures have been developed to help determine which means are significantly different from each other.
Many different approaches - not all produce the same result.
Data dredging and data snooping - analyzing only those comparisons which look interesting after looking at the data – affects the error rate!
Problems with the confidence assumed for the comparisons:
1-a for a particular pre-specified comparison?
1-a for all unplanned comparisons as a group?
STA MCP

6. Where the ai are constants satisfying the constraint:
Linear Comparisons
Any linear comparison among t population means, m1, m2, ...., mt can be written as:
Where the ai are constants satisfying the constraint:
Example: To compare m1 to m2 we use the equation:
with coefficients
Note constraint is met!
STA MCP

7. Variance of a linear contrast:
A linear comparison estimated by using group means is called a linear contrast.
Variance of a linear contrast:
Test of significance
Ho: l = 0 vs. Ha: l  0
STA MCP

8. These two contrasts are said to be orthogonal if:
Orthogonal Contrasts
These two contrasts are said to be orthogonal if:
in which case l1 conveys no information about l2 and vice-versa.
A set of three or more contrasts are said to be mutually orthogonal if all pairs of linear contrasts are orthogonal.
STA MCP

9. Orthogonal Non-orthogonal
Compare average of drugs (2,3) to placebo (1).
Contrast drugs (2,3).
Orthogonal
Non-orthogonal
Contrast Standard drug (2) to placebo (1).
Contrast New drug (3) to placebo (1).
STA MCP

10. Drug Comparisons
STA MCP

11. Importance of Mutual Orthogonality
Assume t treatment groups, each group having n individuals (units).
t-1 mutually orthogonal contrasts can be formed from the t means. (Remember t-1 degrees of freedom.)
Treatment sums of squares (SSB) can be computed as the sum of the sums of squares associated with the t-1 orthogonal contrasts. (i.e. the treatment sums of squares can be partitioned into t-1 parts associated with t-1 mutually orthogonal contrasts).
t-1 independent pieces of information about the variability in the treatment means.
STA MCP

12. Example of Linear Contrasts
Objective: Test the wear quality of a new paint.
Treatments: Weather and wood combinations.
Treatment Code Combination
A m1 hardwood, dry climate
B m2 hardwood, wet climate
C m3 softwood, dry climate
D m4 softwood, wet climate
(Obvious) Questions:
Q1: Is the average life on hardwood the same as average life on softwood?
Q2: Is the average life in dry climate the same as average life in wet climate?
Q3: Does the difference in paint life between wet and dry climates depend upon whether the wood is hard or soft?
STA MCP

13. Q1
MSE= 5 t= 4 n t
-t= 8
Q1: Is the average life on hardwood the same as average life on softwood?
Comparison:
Estimated Contrast
Test H0: l1 = 0 versus HA: l1  0
What is MSl1 ?
Test Statistic:
Rejection Region: Reject H0 if
STA MCP

14. Conclusion: Since F=29. 4 > 5
Conclusion: Since F=29.4 > 5.32 we reject H0 and conclude that there is a significant difference in average life on hard versus soft woods.
STA MCP

15. Q2
MSE= 5 t= 4 n t
-t= 8
Q2: Is the average life in dry climate the same as average life in wet climate?
Comparison:
Estimated Contrast
Test H0: l2 = 0 versus HA: l2  0
Test Statistic:
Rejection Region: Reject H0 if
STA MCP

16. Conclusion: Since F=0. 6 < 5
Conclusion: Since F=0.6 < 5.32 we do not reject H0 and conclude that there is not a significant difference in average life in wet versus dry climates.
STA MCP

17. Q3
MSE= 5 t= 4 n t
-t= 8
Q3: Does the difference in paint life between wet and dry climates depend upon whether the wood is hard or soft?
Comparison:
Estimated Contrast
Test H0: l3 = 0 versus HA: l3  0
Test Statistic:
Rejection Region: Reject H0 if
STA MCP

18. Conclusion: Since F=0 < 5
Conclusion: Since F=0 < 5.32 we do not reject H0 and conclude that the difference between average paint life between wet and dry climates does not depend on wood type. Likewise, the difference between average paint life for the wood types does not depend on climate type (i.e. there is no interaction).
STA MCP

19. Mutual Orthogonality The three are mutually orthogonal.
Total Error SS = dferror x MSE = 8 x 5 = 40
SSl1 = MSl1 = 147
SSl2 = MSl2 = 3
SSl3 = MSl3 = 0
Treatment SS = 150
The three mutually orthogonal contrasts add up to the Treatment Sums of Squares.
STA MCP

20. Types of Error Rates
Compairsonwise Error Rate - the probability of making a Type I error in the comparison of two means. (what we have been discussing for all tests up to this point).
Experimentwise Error Rate - the probability of observing an experiment in which one or more of the pairwise comparisons are incorrectly declared significantly different. (Type I error.)
STA MCP

21. Error Rates: Problems
Suppose we make c mutually orthogonal (independent) comparisons, each with Type I comparisonwise error rate of a. The experimentwise error rate, e, is then:
(If the comparisons are not orthogonal, then the experimentwise error rate is smaller.)
Solution (Bonferroni): set e=0.05 and solve for . But there’s a problem…
E.g. if c=8, we get =0.0064!
STA MCP

22. Multiple Comparison Procedures
Terms:
If the multiple comparison procedure (MCP) requires a significant overall F test, then the procedure is labeled a “Protected” method.
Not all procedures produce the same results.
The major differences among all of the different MCPs is in the calculation of the “yardstick” used to determine if two means are significantly different. The yardstick can generically be referred to as the least significant difference. Any two means greater than this difference are declared significantly different.
Yardsticks are composed of a standard error term and a critical value from some tabulated statistic.
Some procedures have “fixed” yardsticks, some have “variable” yardsticks. The variable yardsticks will depend on how far apart two observed means are in a rank ordered list of the mean values.
Some procedures control Comparisonwise Error, other Experimentwise Error, and some attempt to control both.
STA MCP

23. Fisher’s Least Significant Difference - Protected
Mean of group i (mi) is significantly different from the mean of group j (mj) if
if all groups have same size n.
{tabled value}{standard error of difference}
Type I (comparisonwise) error rate = a
This procedure controls Comparisonwise Error. Experimentwise error control comes from requiring a significant overall F test prior to performing any means comparisons.
How well does it work?
STA MCP

24. Tukey’s W (Honestly Significant Difference) Procedure
Means are different if:
{Table 11 - critical values of the studentized range.}
Experimentwise error rate = a
This MCP controls experimentwise error rate! Comparisonwise error rates is thus very low.
How well does it work?
STA MCP

25. Student Newman Keul Procedure
Rank the t sample means from smallest to largest. For two means that are r “steps” apart in the ranked list, we declare the population means different if (modified Tukey’s MCP):
{Table 11 - critical values of the studentized range. Depends on which mean pair is being considered!}
varying yardstick
r=6
r=5
r=2
r=3
r=4
STA MCP

26. Duncan’s New Multiple Range Test
Neither an experimentwise or comparisonwise error rate control alone.
Based on a ranking of the observed means.
Introduces the concept of a “protection level” (1-a)r-1
Number of steps Protection Level Probability of
Apart, r (0.95)r-1 Falsely Rejecting H0
{Table A -11 in these notes}
STA MCP

27. Waller-Duncan k-ratio MCP (Protected)
A MCP that uses the sample data to help in determining whether we need to use a conservative rule (e.g. Tukey’s MCP) or a nonconservative rule (e.g. Fisher’s MCP).
tc is obtained from Table A-12 or A-13 (in these notes) and is based on:
k - the error weight ratio which designates the seriousness of a Type I error to a Type II error. (typical values 50, 100, 500.
df1 - the model degrees of freedom. (i.e. t-1).
df2 - the error degrees of freedom (i.e. t(n-1)).
F - the F statistics from the overall model effects test.
the assumption of equal group sample sizes.
STA MCP

28. Scheffé’s S Method For any linear contrast: Estimated by:
With estimated variance:
To test H0: l = 0 versus Ha: l ¹ 0
For a specified value of a, reject H0 if:
where:
Confidence interval:
STA MCP

29. Geometric Mean
If the sample sizes are not equal in all groups, the value of n in the previous equations is replaced with the geometric mean of the sample sizes:
E.g. Tukey’s procedure becomes:
STA MCP

30. Comparisonwise error rates for different MCP
STA MCP

31. Experimentwise error rates for different MCP
STA MCP