1. Orthogonal Linear Contrasts This is a technique for partitioning ANOVA sum of squares into individual degrees of freedom

2. Definition Let x 1, x 2,..., x p denote p numerical quantities computed from the data. These could be statistics or the raw observations. A linear combination of x 1, x 2,..., x p is defined to be a quantity,L,computed in the following manner: L = c 1 x 1 + c 2 x 2 +... + c p x p where the coefficients c 1, c 2,..., c p are predetermined numerical values:

3. Definition Let  1,  2,...,  p denote p means and c 1, c 2,..., c p denote p coefficients such that: c 1 + c 2 +... + c p = 0, Then the linear combination L = c 1  1 + c 2  2 +... + c p  p is called a linear contrast of the p means  1,  2,...,  p.

4. Examples 1. 2. 3.L =  1 - 4  2 + 6  3 - 4  4 +  5 = (1)  1 + (-4)  2 + (6)  3 + (-4)  4 + (1)  5 A linear combination A linear contrast

5. Definition Let A = a 1  1 + a 2  2 +... + a p  p and B = b 1  1 + b 2  2 +... + b p  p be two linear contrasts of the p means  1,  2,...,  p. Then A and B are called Orthogonal Linear Contrasts if in addition to: a 1 + a 2 +... + a p = 0 and b 1 + b 2 +... + b p = 0, it is also true that: a 1 b 1 + a 2 b 2 +... + a p b p = 0.

6. Example Let Note:

7. Definition Let A = a 1  1 + a 2  2 +... + a p  p, B= b 1  1 + b 2  2 +... + b p  p,..., and L= l 1  1 + l 2  2 +... + l p  p be a set linear contrasts of the p means  1,  2,...,  p. Then the set is called a set of Mutually Orthogonal Linear Contrasts if each linear contrast in the set is orthogonal to any other linear contrast.

8. Theorem: The maximum number of linear contrasts in a set of Mutually Orthogonal Linear Contrasts of the quantities  1,  2,...,  p is p - 1. p - 1 is called the degrees of freedom (d.f.) for comparing quantities  1,  2,...,  p.

9. Comments 1.Linear contrasts are making comparisons amongst the p values  1,  2,...,  p 2.Orthogonal Linear Contrasts are making independent comparisons amongst the p values  1,  2,...,  p. 3.The number of independent comparisons amongst the p values  1,  2,...,  p is p – 1.

10. Definition Let denote a linear contrast of the p means Let where each mean,, is calculated from n observations.

11. Then the Sum of Squares for testing the Linear Contrast L, i.e. H 0 : L = 0 against H A : L ≠ 0 is defined to be:

12. the degrees of freedom (df) for testing the Linear Contrast L, is defined to be the F-ratio for testing the Linear Contrast L, is defined to be:

13. To test if a set of mutually orthogonal linear contrasts are zero: i.e. H 0 : L 1 = 0, L 2 = 0,..., L k = 0 then the Sum of Squares is: the degrees of freedom (df) is the F-ratio is:

14. Theorem: Let L 1, L 2,..., L p-1 denote p-1 mutually orthogonal Linear contrasts for comparing the p means. Then the Sum of Squares for comparing the p means based on p – 1 degrees of freedom, SS Between, satisfies:

15. Comment Defining a set of Orthogonal Linear Contrasts for comparing the p means allows the researcher to "break apart" the Sum of Squares for comparing the p means, SS Between, and make individual tests of each the Linear Contrast.

16. The Diet-Weight Gain example The sum of Squares for comparing the 6 means is given in the Anova Table:

17. Five mutually orthogonal contrasts are given below (together with a description of the purpose of these contrasts) : (A comparison of the High protein diets with Low protein diets) (A comparison of the Beef source of protein with the Pork source of protein)

18. (A comparison of the Meat (Beef - Pork) source of protein with the Cereal source of protein) (A comparison representing interaction between Level of protein and Source of protein for the Meat source of Protein) (A comparison representing interaction between Level of protein with the Cereal source of Protein)

19. Table of Coefficients Contrast diet 123456 L1L1 111 L2L2 10 10 L3L3 2 2 L4L4 10 01 L5L5 2 1-21 Note: L 4 = L 1 × L 2 and L 5 = L 1 × L 3 L 1 is the 1 df for the Level main effect L 2 and L 3 are the 2 df for the Source main effect L 4 and L 5 are the 2 df for the Source-Level interaction

20. The Anova Table for Testing these contrasts is given below: The Mutually Orthogonal contrasts that are eventually selected should be determine prior to observing the data and should be determined by the objectives of the experiment

21. Another Five mutually orthogonal contrasts are given below (together with a description of the purpose of these contrasts) : (A comparison of the Beef source of protein with the Pork source of protein) (A comparison of the Meat (Beef - Pork) source of protein with the Cereal source of protein)

22. (A comparison of the high and low protein diets for the Beef source of protein) (A comparison of the high and low protein diets for the Cereal source of protein) (A comparison of the high and low protein diets for the Pork source of protein)

23. Table of Coefficients Contrast diet 123456 L1L1 1010 L2L2 1-211 1 L3L3 10000 L4L4 0100 0 L5L5 00100 Note: L 1 and L 2 are the 2 df for the Source main effect L 3,L 4 and L 5 are the 3 df comparing the Level within the Source.

24. The Anova Table for Testing these contrasts is given below:

25. Techniques for constructing orthogonal linear contrasts

26. I.Comparing first k – 1 with k th Consider the p values – y 1, y 2, y 3,..., y p L 1 = 1 st vs 2 nd = y 1 - y 2 L 2 = 1 st, 2 nd vs 3 rd =½ (y 1 + y 2 ) – y 3 L 3 = 1 st, 2 nd, 3 rd vs 4 th = 1 / 3 (y 1 + y 2 + y 3 ) – y 4 etc

27. Helmert contrasts Contrastcoefficients L1L1 1000 L2L2 200 L3L3 30 L4L4 4 Contrastexplanation L1L1 2 nd versus 1 st L2L2 3 rd versus 1 st and 2 nd L3L3 4 th versus 1 st, 2 nd and 3 rd L4L4 5 th versus 1 st, 2 nd, 3 rd and 4 th

28. II.Comparing between Groups then within groups Consider the p = 10 values – y 1, y 2, y 3,..., y 10 Suppose these 10 values are grouped Group 1 y 1, y 2, y 3 Group 2 y 4, y 5, y 6, y 7 Group 3 y 8, y 9, y 10 Comparison of Groups (2 d.f.) L 1 = Group 1 vs Group 2 = 1 / 3 (y 1 + y 2 + y 3 ) - 1 / 4 (y 4 + y 5 + y 6 + y 7 ) L 2 = Group 1, Group 2 vs Group 2 = 1 / 7 (y 1 + y 2 + y 3 + y 4 + y 5 + y 6 + y 7 ) - 1 / 3 ( y 8 + y 9 + y 10 )

29. Comparison of within Groups Within Group 1 (2 d.f.) L 3 = 1 vs 2 = y 1 - y 2 L 4 = 1,2 vs 3= 1 / 2 (y 1 + y 2 ) - y 3 Within Group 2 (3 d.f.) L 5 = 4 vs 5 = y 4 – y 5 L 6 = 4, 5 vs 6= 1 / 2 (y 4 + y 5 ) – y 6 L 7 = 4, 5, 6 vs 7= 1 / 3 (y 4 + y 5 + y 6 ) –y 7 Within Group 3 (2 d.f.) L 8 = 8 vs 9 = y 8 – y 9 L 9 = 8, 9 vs 10= 1 / 2 (y 8 + y 9 ) –y 10

30. II.Comparisons when Grouping is done on two different ways Consider the p = ab values – y 11, y 12, y 13,..., y 1b, y 21, y 22, y 23,..., y 2b,..., y a1, y a2, y a3,..., y ab Column Groups 123...b Row Groups 1y 11 y 12 y 13...y1by1b 2y 21 y 22 y 23...y2by2b 3y 31 y 32 y 33...y3by3b aya1ya1 ya2ya2 ya3ya3 y ab

31. Comparison of Row Groups (a - 1 d.f.) R 1, R 2, R 3,..., R a -1 Comparison of Column Groups (b - 1 d.f.) C 1, C 2, C 3,..., C b -1 Interaction contrasts (a - 1) (b - 1) d.f. (RC) 11 = R 1 × C 1, (RC) 12 = R 1 × C 2,..., (RC) a - 1,b - 1 = R a - 1 × C b – 1 Comment: The coefficients of (RC) ij = R i × C j are found by multiplying the coefficients of R i with the coefficients of C j.

32. Example (a = 3, b = 4) 1234 1y 11 y 12 y 13 y 14 2y 21 y 22 y 23 y 24 3y 31 y 32 y 33 y 34

33. Orthogonal Contrasts y 11 y 12 y 13 y 14 y 21 y 22 y 23 y 24 y 31 y 32 y 33 y 34 R1R1 1111 0000 R2R2 11111111-2 C1C1 1001 001 00 C2C2 11-2011 011 0 C3C3 111-3111 111 (RC) 11 =R 1 ×C 1 100 1000000 (RC) 12 =R 1 ×C 2 11-20 200000 (RC) 13 =R 1 ×C 3 111-3 30000 (RC) 21 =R 2 ×C 1 1001 00-2200 (RC) 22 =R 2 ×C 2 11-2011 0 40 (RC) 23 =R 2 ×C 3 111-3111 -2 6

34. Orthogonal Linear Contrasts Polynomial Regression ≠

35. Let  1,  2,...,  p denote p means and consider the first differences  i =  i -  i-1 if  1 =  2 =... =  p then  i =  i -  i-1 = 0 If the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a straight line with non-zero slope then  i =  i -  i-1 ≠ 0 but equal.

36. Consider the 2 nd differences  2  i = (  i -  i-1 )-(  i -1 -  i-2 ) =  i - 2  i-1 +  i-2 If the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a straight line then  2  i =  i - 2  i-1 +  i-2 = 0 If the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a quadratic curve then  2  i =  i - 2  i-1 +  i-2 ≠ 0 but equal.

37. Consider the 3 rd differences  3  i =  i - 3  i-1 + 3  i-2 -  i-3 If the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a quadratic curve then  3  i =  i - 3  i-1 + 3  i-2 -  i-3 = 0 If the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a cubic curve then  3  i =  i - 3  i-1 + 3  i-2 -  i-3 ≠ 0 but equal.

38. Continuing, 4 th differences,  4  i will be non- zero but equal if the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a quartic curve (4 th degree). 5 th differences,  5  i will be non- zero but equal if the points (1,  1 ), (2,  2 ) … (p,  p ) lie on a quintic curve (5 th degree). etc.

39. Let L = a 2  2 + a 3  3 + … + a p  p Q 2 = b 3  2  3 + … + b p  2  p C = c 4  3  4 + … + c p  3  p Q 4 = d 5  4  5 + … + d p  4  p etc. Where a 2, …, a p, b 1, …, b p, c 1, … etc are chosen so that L, Q 2, C, Q 4, … etc are mutually orthogonal contrasts.

40. If the means are equal then L = Q 2 = C = Q 4 = … = 0. If the means are linear then L ≠ 0 but Q 2 = C = Q 4 = … = 0. If the means are quadratic then Q 2 ≠ 0 but C = Q 4, … = 0. If the means are cubic then C ≠ 0 but Q 4 = … = 0.

41. Orthogonal Linear Contrasts for Polynomial Regression

43. Example In this example we are measuring the “Life” of an electronic component and how it depends on the temperature on activation

44. The Anova Table SourceSSdfMSF Treat6604165.023.57 Linear187.501187.5026.79 Quadratic433.931433.9361.99 Cubic0.0010.000.00 Quartic38.57138.575.51 Error70107.00 Total73014 L = 25.00Q 2 = -45.00C = 0.00Q 4 = 30.00

45. The Anova Tables for Determining degree of polynomial Testing for effect of the factor

46. Testing for departure from Linear Q 2 + C + Q 4

47. Testing for departure from Quadratic C + Q 4

49. Multiple Testing Fisher’s L.S.D. (Least Significant Difference) Procedure Tukey’s Multiple comparison procedure Scheffe’s multiple comparison procedure

50. Multiple Testing – a Simple Example Suppose we are interested in testing to see if two parameters (  1 and  2 ) are equal to zero. There are two approaches 1.We could test each parameter separately a) H 0 :  1 = 0 against H A :  1 ≠ 0, then b)H 0 :  2 = 0 against H A :  2 ≠ 0 2.We could develop an overall test H 0 :  1 = 0,  2 = 0 against H A :  1 ≠ 0 or  2 ≠ 0

51. 1.To test each parameter separately a) then b) We might use the following test: then is chosen so that the probability of a Type I errorof each test is .

52. 2.To perform an overall test H 0 :  1 = 0,  2 = 0 against H A :  1 ≠ 0 or  2 ≠ 0 we might use the test is chosen so that the probability of a Type I error is .

60. Post-hoc Tests Multiple Comparison Tests

61. Post-hoc Tests Multiple Comparison Tests

62. Multiple Testing Fisher’s L.S.D. (Least Significant Difference) Procedure Tukey’s Multiple comparison procedure Scheffe’s multiple comparison procedure

63. Suppose we have p means An F-test has revealed that there are significant differences amongst the p means We want to perform an analysis to determine precisely where the differences exist.

64. Example One –way ANOVA The F test – for comparing k means Situation We have k normal populations Let  i and  denote the mean and standard deviation of population i. i = 1, 2, 3, … k. Note: we assume that the standard deviation for each population is the same.  1 =  2 = … =  k = 

65. We want to test against

66. Sourced.f.Sum of Squares Mean Square F-ratio Betweenk - 1SS Between MS Between MS B /MS W WithinN - kSS Within MS Within TotalN - 1SS Total Anova Table

67. Comments The F-test H 0 :  1 =  2 =  3 = … =  k against H A : at least one pair of means are different If H 0 is accepted we know that all means are equal (not significantly different) If H 0 is rejected we conclude that at least one pair of means is significantly different. The F – test gives no information to which pairs of means are different. One now can use two sample t tests to determine which pairs means are significantly different

68. Fishers LSD (least significant difference) procedure:

69. 1.Test H 0 :  1 =  2 =  3 = … =  k against H A : at least one pair of means are different, using the ANOVA F-test 2.If H 0 is accepted we know that all means are equal (not significantly different). Then stop in this case 3.If H 0 is rejected we conclude that at least one pair of means is significantly different, then follow this by using two sample t tests to determine which pairs means are significantly different

70. Tukey’s Multiple Comparison Test

71. Let Tukey's Critical Differences Two means are declared significant if they differ by more than this amount. denote the standard error of each = the tabled value for Tukey’s studentized range p = no. of means, = df for Error

72. Table: Critical values for Tukey’s studentized Range distribution

76. Scheffe’s Multiple Comparison Test

77. Scheffe's Critical Differences (for Linear contrasts) A linear contrast is declared significant if it exceeds this amount. = the tabled value for F distribution (p -1 = df for comparing p means, = df for Error)

78. Scheffe's Critical Differences (for comparing two means) Two means are declared significant if they differ by more than this amount.

79. Multiple Confidence Intervals Tukey’s Multiple confidence intervals Scheffe’s Multiple confidence intervals One-at-a-time confidence intervals

80. Comments Tukey’s Multiple confidence intervals Scheffe’s Multiple confidence intervals One-at-a-time confidence intervals The probability that each of these interval contains  i –  j is 1 –  The probability that all of these interval contains  i –  j is considerably lower than 1 –  These intervals can be computed not only for simple differences in means,  i –  j, but also any other linear contrast, c 1  1 + … + c k  k  The probability that all of these intervals contain its linear contrast is 1 – 

81. Example In the following example we are comparing weight gains resulting from the following six diets 1.Diet 1 - High Protein, Beef 2.Diet 2 - High Protein, Cereal 3.Diet 3 - High Protein, Pork 4.Diet 4 - Low protein, Beef 5.Diet 5 - Low protein, Cereal 6.Diet 6 - Low protein, Pork

83. Sourced.f.Sum of Squares Mean Square F-ratio Between54612.933922.5874.3 Within5411586.000214.556 (p = 0.0023) Total5916198.933 The Diet Example

84. k = 6, = 54(≈ 60), M.S.E = 214.556, n = 10 Tukey’s critical difference q 0.05 = 4.163 Tukey’s intervals

85. Scheffe’s critical difference F 0.05 = 2.368, 1 = k – 1= 5, 2 = 54 (≈ 60) Scheffe’s intervals

86. One-at-a-time critical difference t 0.025 = 2.00o, = 54 (≈ 60) One-at-a-time intervals

87. Multiple Confidence Intervals

88. Multiple comparisons for Factorial Designs In a balanced completely randomized design (CRD) it is appropriate to use Tukey’s or Scheffe’s procedure to compare the cell means associated with the interaction

89. Example – Four factor experiment Four factors are studied for their effect on Y (luster of paint film). The four factors are: Two observations of film luster (Y) are taken for each treatment combination 1) Film Thickness - (1 or 2 mils) 2)Drying conditions (Regular or Special) 3)Length of wash (10,30,40 or 60 Minutes), and 4)Temperature of wash (92 ˚C or 100 ˚C)

90. The data is tabulated below: Regular DrySpecial Dry Minutes92  C100  C92  C100  C 1-mil Thickness 203.43.419.614.52.13.817.213.4 304.14.117.517.04.04.613.514.3 404.94.217.615.25.13.316.017.8 605.04.920.917.18.34.317.513.9 2-mil Thickness 205.53.726.629.54.54.525.622.5 305.76.131.630.25.95.929.229.8 405.55.630.530.25.55.832.627.4 607.26.031.429.68.09.933.529.5

91. Since there is significant A(Temp) - B(drying) and A(Temp) – D(Thickness) interactions, it is appropriate to compare the 8 Temp×drying×Thickness cell means. Since length is significant and is additive with the other 3 factors it is approriate to compare the 4 cell means associated with this factor separately

92. Since there is significant A(Temp) - B(drying) and A(Temp) – D(Thickness) interactions, it is appropriate to compare the 8 Temp×drying×Thickness cell means. Since the main effect for Length is significant and is additive with the other 3 factors it is appropriate to compare the 4 cell means associated with this factor separately

95. Underlined groups that have no significant differences

96. There are many multiple (post hoc) comparison procedures 1.Tukey’s 2.Scheffe’, 3.Duncan’s Multiple Range 4.Neumann-Keuls etc Considerable controversy: “I have not included the multiple comparison methods of D.B. Duncan because I have been unable to understand their justification” H. Scheffe, Analysis of Variance