1. Multiple Comparison Procedures Comfort Ratings of 13 Fabric Types A.V. Cardello, C. Winterhalter, and H.G. Schultz (2003). "Predicting the Handle and Comfort of Military Clothing Fabrics from Sensory and Instrumental Data: Development and Application of New Psychophysical Methods," Textile Research Journal, Vol. 73, pp. 221-237.

2. Treatments Means and standard deviations of 45 comfort ratings for 13 military fabrics. Fabric types: 10R - 50/50 Nylon/combed cotton, ripstop poplin weave 11A - 50/50 Nylon/Polyester, oxford weave (Australian) 12T - 50/50 Nylon/cotton, twill weave 13P - 92/5/3 Nomex/Kevlar/P140, plain weave 14N - 100 Cotton (former flame retardant treated) 15B - 77/23 Cotton sheath/synthetic core, twill (UK) 16C - 100 combed cotton, ripstop poplin (former hot weather BDU) 17C - 65/35 Wool/Polyester, plain weave (Canada-unlaundered) 18L - 65/35 Wool/Polyester, plain weave (Canada-laundered) 19N - 92/5/3 Nomex/Kevlar/P140, oxford weave 20J - Carded cotton sheath/nylon core, plain weave (Canada) 124 - 100 Pima cotton ripstop poplin (experimental) 176 - 50/50 Nylon carded cotton ripstop poplin weave

3. Multiple Comparisons Individual & Combined Null Hypotheses (H 0  H 01 … H 0k ) Comparisonwise Error Rate  Pr(Reject H 0i |H 0i True) Experimentwise Error Rate  Pr(Reject any H 0i |All H 0i True) False Discovery Rate  E(# False Rejects/Total Rejections) Strong Familywise Error Rate  Pr(Any False Discoveries) Simultaneous Confidence Intervals  Pr(All Correct)=1-  Multiple Comparison Procedures control Type 1 Error Rate other than per comparison

4. Data and Analysis of Variance

5. Bonferroni Based Methods Construct P-values for all k test statistics Order P-values from smallest p (1) ≤ … ≤ p (k)  Bonferroni: Reject H 0(i) if p (i) ≤  k  Holm (Controls Strong FWER): Reject H 0(i) if p (j) ≤  k-j+1)  j ≤ i  False Discovery Rate: Reject H 0(i) if p (j) ≤ j  k for some j ≥ i (Assumes independent tests, not the case for this example) Example: Comparing all k=13(12)/2=78 pairs of fabrics

6. Fabric Example j=1,…,26

7. Fabric Example j=27,…,52

8. Fabric Example j=53,…,78

9. Scheffe’s Method for All Contrasts Can be used for any number of contrasts, even those suggested by data. Conservative (Wide CI’s, Low Power)

10. Example – Scheffe’s Method – All Pairwise Tests/CIs

11. Tukey’s Method for All Pairwise Comparisons Makes use of the Studentized Range Distribution Pr{(max(Y 1,…,Y n )-min(Y 1,…,Y n ))/S ≥ q( ,n, )} =   Y 1,…,Y n }  S  degrees of freedom for S

12. Tukey’s Method for All Pairwise Comparisons

13. Bonferroni’s Method for All Pairwise Comparisons Adjusts type I error rate for each test to  /(# of tests) Increases Confidence levels of CI’s to (1-(  /(# of CIs)))

14. Bonferroni’s Method for All Pairwise Comparisons

15. SNK Method for All Pairwise Comparisons Controls False Discovery Rate at  Uses Different Critical Values for different ranges of means

16. SNK Method for All Pairwise Comparisons

18. Duncan’s Method for All Pairwise Comparisons More powerful than SNK method, at cost of increasing  for longer stretches (does not control experimentwise error rate) Uses Different Critical Values for different ranges of means

19. Duncan’s Method for All Pairwise Comparisons

21. Fisher’s Protected LSD for All Pairwise Comparisons Controls Experimentwise Error Rate at  Only Conducted if F-test is significant (P-value ≤ 

22. Fisher’s Protected LSD for All Pairwise Comparisons

23. Multiple Comparisons with Best Treatment/Control Pr{subset of treatments contains the best} = 1- 

24. Multiple Comparisons with Best Treatment/Control Treatments 13P, 15B, and 11A all lie within 16.22 of the highest mean