1. Statistical Analysis Professor Lynne Stokes Department of Statistical Science Lecture #19 Analysis of Designs with Random Factor Levels

2. Fermentation Process Experiment MGH Ex 10.17

3. Fermentation Process Experiment Proc GLM data=Ferment; class batch process; model response = batch process; random batch / test; lsmeans process / stderr pdiff; run;

4. Fermentation Process Experiment The GLM Procedure Dependent Variable: Response Sum of Source DF Squares Mean Square F Value Pr > F Model 7 389.0000000 55.5714286 1.73 0.1934 Error 12 386.0000000 32.1666667 Corrected Total 19 775.0000000 R-Square Coeff Var Root MSE Response Mean 0.501935 6.958977 5.671567 81.50000 Source DF Type I SS Mean Square F Value Pr > F Batch 4 324.0000000 81.0000000 2.52 0.0965 Process 3 65.0000000 21.6666667 0.67 0.5846 Source DF Type III SS Mean Square F Value Pr > F Batch 4 324.0000000 81.0000000 2.52 0.0965 Process 3 65.0000000 21.6666667 0.67 0.5846

5. Fermentation Process Experiment Mason, Gunst, & Hess: Exercise 10.17 The GLM Procedure Source Type III Expected Mean Square Batch Var(Error) + 4 Var(Batch) Process Var(Error) + Q(Process) The Random Statement Produces This Output

6. Fermentation Process Experiment Mason, Gunst, & Hess: Exercise 10.17 7 The GLM Procedure Tests of Hypotheses for Mixed Model Analysis of Variance Dependent Variable: Response Source DF Type III SS Mean Square F Value Pr > F Batch 4 324.000000 81.000000 2.52 0.0965 Process 3 65.000000 21.666667 0.67 0.5846 Error: MS(Error) 12 386.000000 32.166667 The Random Statement Produces This Output

7. Fermentation Process Experiment Mason, Gunst, & Hess: Exercise 10.17 8 Least Squares Means Response Standard LSMEAN Process LSMEAN Error Pr > |t| Number F1 79.0000000 2.5364017 <.0001 1 F2 82.0000000 2.5364017 <.0001 2 F3 84.0000000 2.5364017 <.0001 3 F4 81.0000000 2.5364017 <.0001 4 Least Squares Means for effect Process Pr > |t| for H0: LSMean(i)=LSMean(j) Dependent Variable: Response i/j 1 2 3 4 1 0.4193 0.1886 0.5874 2 0.4193 0.5874 0.7852 3 0.1886 0.5874 0.4193 4 0.5874 0.7852 0.4193 LSMEANS Standard Errors Only use the Fixed Effects Computing Formulas These are Incorrect (See Proc Mixed Results)

8. Estimation of Variance Components: Method of Moments Equate mean squares to their expected mean squares and solve Equate mean squares to their expected mean squares and solve Random Main Effects Model Method of Moments F Test: MS A / MS E

9. Fermentation Process Experiment Mason, Gunst, & Hess: Exercise 10.17 The GLM Procedure Source Type III Expected Mean Square Batch Var(Error) + 4 Var(Batch) Process Var(Error) + Q(Process)

10. Estimation of Variance Components: Method of Moments Equate mean squares to their expected mean squares and solve Equate mean squares to their expected mean squares and solve Three-Factor Random Effects Model Method of Moments F Test: MS ABC / MS E

11. Estimation of Variance Components Three-Factor Random Effects Model F Test: MS AB / MS ABC

12. Estimation of Variance Components Three-Factor Random Effects Model F Test: No Exact Test

13. Estimation of Variance Components Confidence Intervals 

14. Estimation of Variance Components Confidence Intervals 

15. Estimation of Variance Components Confidence Intervals

16. Testing Variance Components Three-Factor Random Effects Model F Test: No Exact Test

17. Satterthwaite’s Approximate F Statistic Assumptions MS 1, MS 2,..., MS k are Pairwise Independent ANOVA Mean Squares

18. Satterthwaite’s Approximate F Statistic Approximation L ~  L    L 

19. Satterthwaite’s Approximate F Statistic Solution

20. Satterthwaite’s Approximate F Statistic Application Select to be Independent of L Regardless of Ho Under Ho

21. Satterthwaite’s Approximate F Statistic

22. Approximation #1 M 1 & F Can Be Negative M 1 & F Can Be Negative

23. Satterthwaite’s Approximate F Statistic Approximation #2 M 2 & F Are Positive M 2 & F Are Positive

24. Random Effects Testing Three-Factor Random Effects Model Source Mean SquareExpected Mean Square AMS A  e 2 + r  abc  + cr  ab  + br  ag  + bcr  a  ABMS AB  e 2 + r  abc  + cr  ab  ABCMS ABC  e + r  abc  ErrorMS E  e 2 Effects Not Necessarily Tested Against Error Test Main Effects Even if Interactions are Significant May Not be an Exact Test (Mixed Effects Models) Effects Not Necessarily Tested Against Error Test Main Effects Even if Interactions are Significant May Not be an Exact Test (Mixed Effects Models)

25. Random Effects Testing Three-Factor Random Effects Model Source Mean SquareExpected Mean Square AMS A  e 2 + r  abc  + cr  ab  + br  ag  + bcr  a  ABMS AB  e 2 + r  abc  + cr  ab  ABCMS ABC  e + r  abc  ErrorMS E  e 2 Proc GLM: Random... / Test Produces Satterthwaite Approximate Test Statistics Fixed Effects Standard Errors May be Incorrect Proc GLM: Random... / Test Produces Satterthwaite Approximate Test Statistics Fixed Effects Standard Errors May be Incorrect

26. Restricted Maximum Likelihood

27. Proc Mixed Proc Mixed data=Ferment Cl; class batch process; model response = process; random batch ; lsmeans process / adjust=tukey pdiff; run;

28. Fermentation Process Experiment Mason, Gunst, & Hess: Exercise 10.17 8 The Mixed Procedure Covariance Parameter Estimates Cov Parm Estimate Alpha Lower Upper Batch 12.2083 0.05 2.8125 2023.05 Residual 32.1667 0.05 16.5405 87.6518 Balanced Design: Same as Method of Moments

29. Fermentation Process Experiment Type 3 Tests of Fixed Effects Num Den Effect DF DF F Value Pr > F Process 3 12 0.67 0.5846 Least Squares Means Standard Effect Process Estimate Error DF t Value Pr > |t| Process F1 79.0000 2.9791 12 26.52 <.0001 Process F2 82.0000 2.9791 12 27.53 <.0001 Process F3 84.0000 2.9791 12 28.20 <.0001 Process F4 81.0000 2.9791 12 27.19 <.0001 Correct Standard Errors

30. Fermentation Process Experiment Differences of Least Squares Means Standard Effect Process _Process Estimate Error DF t Value Pr > |t| Process F1 F2 -3.0000 3.5870 12 -0.84 0.4193 Process F1 F3 -5.0000 3.5870 12 -1.39 0.1886 Differences of Least Squares Means Effect Process _Process Adjustment Adj P Process F1 F2 Tukey-Kramer 0.8363 Process F1 F3 Tukey-Kramer 0.5261 Other Pairwise Comparisons on the Next Output Page Note: Standard Error of a Difference is Smaller than Random Effects Cancel in y i1 – y i2 (Pairwise Balance Needed)

31. Randomized Complete Block Designs Factorial Structure with Main Effect for Blocks Factorial Structure with Main Effect for Blocks Nothing New

32. Latin Square Designs Control Two Sources of Variability Restrictions Factor of Interest and Two blocking Factors Each at k Levels No Interactions Among the Experimental and Blocking Factors Experiment Size Latin Square : n = k 2 Complete Factorial : n = k 3 + r

33. Analysis of Latin Square Designs ith Row Block Effect Error Variation From All Sources Except Blocks & Factor Main Effects jth Column Block Effect kth Factor Level Effect Main Effects Analysis of Variance Model Main Effects Analysis of Variance Model

34. Balanced Incomplete Block Designs Used when blocks contain fewer experimental units than the number of unique factor-level combinations b blocks f factor-level combinations k < f experimental units per block No interactions with the design factor(s)

35. Asphalt-Pavement Rating Study Purpose Assess the Deterioration of Highway Pavement Response Rating: 0 = No pavement remaining 100 = excellent condition Design Factor 16 District Engineers (Random) Blocking Factor 16 Road Segments (Random)

36. Asphalt-Pavement Rating Study b = 16 Road Segments (Blocks) f = 16 Engineers (Factor-Level Combinations) k = 6 Engineers/Road Segment

37. Asphalt-Pavement Rating Study Design Engineer RoadSegmentRoadSegment

38. Analysis of Variance with Unbalanced Data Reduction in Error Sums of Squares R(M 1 | M 2 ) = SS E2 - SS E1 Error Sums of Squares Models 1 & 2 are Hierarchical Model 2 has a Subset of Model 1 Terms SS E2 SS E1 Testing Effects df = 2 - 1

39. Balanced Incomplete Block Design Model 1 Model 2 Model 3 Block Effect: R(M 1 | M 2 ) = SS E2 - SS E1 Factor Effect: R(M 1 | M 3 ) = SS E3 - SS E1 SAS PROC GLM Type I Sums of Squares Two Model Fits SAS PROC GLM Type I Sums of Squares Two Model Fits

40. Asphalt-Pavement Rating Study Sum of Source DF Squares Mean Square F Value Pr > F Model 30 13422.12500 447.40417 7.10 <.0001 Error 65 4098.83333 63.05897 Corrected Total 95 17520.95833 R-Square Coeff Var Root MSE Rating Mean 0.766061 12.93405 7.940968 61.39583 Source DF Type I SS Mean Square F Value Pr > F Road 15 11786.95833 785.79722 12.46 <.0001 Engineer 15 1635.16667 109.01111 1.73 0.0668 Source DF Type III SS Mean Square F Value Pr > F Road 15 11005.16667 733.67778 11.63 <.0001 Engineer 15 1635.16667 109.01111 1.73 0.0668

41. Asphalt-Pavement Rating Study Sum of Source DF Squares Mean Square F Value Pr > F Model 30 13422.12500 447.40417 7.10 <.0001 Error 65 4098.83333 63.05897 Corrected Total 95 17520.95833 R-Square Coeff Var Root MSE Rating Mean 0.766061 12.93405 7.940968 61.39583 Source DF Type I SS Mean Square F Value Pr > F Engineer 15 2416.95833 161.13056 2.56 0.0047 Road 15 11005.16667 733.67778 11.63 <.0001 Source DF Type III SS Mean Square F Value Pr > F Engineer 15 1635.16667 109.01111 1.73 0.0668 Road 15 11005.16667 733.67778 11.63 <.000

42. Asphalt-Pavement Rating Study Asphalt-Paving Rating Study Mason, Gunst, & Hess: Table 10.4 The GLM Procedure Source Type III Expected Mean Square Engineer Var(Error) + 5.3333 Var(Engineer) Road Var(Error) + 5.3333 Var(Road)

43. Asphalt-Pavement Rating Study The Mixed Procedure Convergence criteria met Covariance Parameter Estimates Cov Parm Estimate Alpha Lower Upper Road 121.96 0.05 63.5064 323.11 Engineer 7.9899 0.05 2.2767 222.91 Residual 63.4641 0.05 46.1892 92.6823

44. Allergic Reaction Study: Randomized Complete Block Design MGH Table 10.6 Not Additive Not Additive

45. Balanced Incomplete Block Designs Multiple Comparisons Use Adjusted Factor-Level Averages Multiple Comparisons Use Adjusted Factor-Level Averages Average of r Block Averages Containing Factor-Level i MGH Exhibit 10.5