1 Experimental Statistics - week 7 Chapter 15: Factorial Models (15.5) Chapter 17: Random Effects Models

2 Testing Procedure Revisted 2 factor CRD Design Step 1. Test for interaction. Step 2. (a) IF there IS NOT a significant interaction - test the main effects (b) IF there IS a significant interaction - compare a x b cell means (by hand) Main Idea: We are trying to determine whether the factors effect the response either individually or collectively.

3 Statistics 5372: Experimental Statistics Assignment Report Form Name: Data Set or Problem Description Key Results of the Analysis Conclusions in the Language of the Problem Appendices: A. Tables and Figures Cited in the Report B. SAS Log from the Final SAS Run Notes: 1. All assignments should be typed using a word processor according to the format above. 2. SAS output should consist only of tables and figures cited in the report. The report should refer to these tables and figures using numbers you assign, i.e. Table 1, etc. 3. The data should be listed somewhere in the report. (within SAS code is ok)

Auditory Visual 5 sec 10 sec 15 sec Warning Time

5 Note: For balanced designs, i.e. for STIMULUS data.228 = ( )/3 = ( )/2

Auditory Visual 5 sec 10 sec 15 sec Warning Time Now Consider:

7 Every Combination of the Factor Levels has an Equal Number of Repeats Sums of Squares –Uniquely Calculated »Usual Textbook Formulas Not Every Combination of the Factor Levels has an Equal Number of Repeats Sums of Squares –Not Uniquely Calculated »Usual Textbook Formulas Are Not Valid Balanced Experimental Designs Unbalanced Experimental Designs

8 - they typically use “Textbook Formulas” Many Software Programs Cannot Properly Calculate Sums of Squares for Unbalanced Designs SAS: - use Type III sums of squares -- analysis is closest to that for “Balanced Experiments” Unbalanced Experimental Designs - must Use Proc GLM, not Proc ANOVA - Type I and Type III sums-of-squares results will not generally agree

9 The GLM Procedure Dependent Variable: response Sum of Source DF Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total R-Square Coeff Var Root MSE response Mean Source DF Type I SS Mean Square F Value Pr > F type <.0001 time type*time Source DF Type III SS Mean Square F Value Pr > F type <.0001 time type*time Unbalanced Data -- GLM Output

10 Model for 3-factor Factorial Design where and also, the sum over any subscript of a 2 or 3 factor interaction is zero

11 Sum-of-Squares Breakdown (3-factor ANOVA)

12 3-Factor ANOVA Table (3-Factor Completely Randomized Design) Source SS df MS F Main Effects A SSA a  1 B SSB b  1 C SSC c  1 Interactions AB SSAB ( a  1)(b  1) AC SSAC ( a  1)(c  1) BC SSBC ( b  1)(c  1) ABC SSABC ( a  1)(b  1)(c  1) Error SSE abc(n  1) Total TSS abcn  See page 908

13 Popcorn Data Factors (A) Brand (3 brands) (B) Power of Microwave (500, 600 watts) (C) 4, 4.5 minutes n = 2 replications per cell Response variable -- % of kernels that popped

14 Popcorn Data

15 PROC GLM; class brand power time; MODEL percent=brand power time brand*power brand*time power*time brand*power*time; Title 'Popcorn Example -- 3-Factor ANOVA'; MEANS brand power time/LSD; RUN; SAS GLM Code – 3 Factor Model MODEL percent=brand power time brand*power brand*time power*time brand*power*time The Statement can be written as MODEL percent=brand | power | time;

16 The GLM Procedure Dependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE percent Mean Source DF Type I SS Mean Square F Value Pr > F brand power time brand*power brand*time power*time brand*power*time

17 Testing Procedure Testing Procedure 3 factor CRD Design Step 1. Test for 3rd order interaction. IF there IS a significant 3rd order interaction - compare cell means IF there IS NOT a significant 3rd order interaction - test 2nd order interactions IF there IS NOT a sig. 2nd order interaction - test the main effects IF there IS a significant 2rd order interaction - compare associated cell means In general -- test main effects only for variables not involved in a significant 2nd or 3rd order interaction

18 Examine brand x time cell means Examine Power main effect The GLM Procedure Dependent Variable: percent Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE percent Mean Source DF Type I SS Mean Square F Value Pr > F brand power time brand*power brand*time power*time brand*power*time

19

20 To complete the analysis: 1. The F-test for Power was not significant (.2443) 2. Compare the 6 cell means plotted in interaction plot using procedure analogous to the one used for pilot plant data. PROC SORT data=one;BY brand time; PROC MEANS mean std data=one;BY brand time; OUTPUT OUT=cells MEAN=percent; Title 'Brand x Time Cell Means for Popcorn Data'; RUN; Obs brand time _TYPE_ _FREQ_ percent LSD =

21 Popcorn Data = = cell mean for Brand 1 and Time 4.5

22 Models with Random Effects Fixed-Effects Models -- the models we’ve studied to this point -- factor levels have been specifically selected - investigator is interested in testing effects of these specific levels on the response variable Examples: -- CAR data - interested in performance of these 5 gasolines -- Pilot Plant data - interested in the specific temperatures (160 o and 180 o ) and catalysts (C1 and C2)

23 Random-Effect Factor -- the factor has a large number of possible levels -- the levels used in the analysis are a random sample from the population of all possible levels - investigator wants to draw conclusions about the population from which these levels were chosen (not the specific levels themselves)

24 Fixed Effects vs Random Effects This determination affects - the model - the hypothesis tested - the conclusions drawn - the F-tests involved (sometimes)

25 1-Factor Random Effects Model Assumptions:

26 Hypotheses: H o :     H a :     H o says (considering the variability of the y ij ’s) : - the component of the variance due to “Factor” has zero variance -- i.e. no factor “level-to-level” variation - all of the variability observed is just unexplained subject-to-subject variation -- at least none is explained by variation due to the factor

27 DATA one; INPUT operator output; DATALINES; ; PROC GLM; CLASS operator; MODEL output=operator; RANDOM operator; TITLE ‘Operator Data: One Factor Random Effects Model'; RUN; These are data from an experiment studying the effect of four operators (chosen randomly) on the output of a particular machine. t = n =

28 The GLM Procedure Dependent Variable: output Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE output Mean Source DF Type I SS Mean Square F Value Pr > F operator The GLM Procedure Source Type III Expected Mean Square operator Var(Error) + 4 Var(operator) One Factor Random effects Model

29 We reject H o :     (p =.0002) and we conclude that there is variability due to operator Conclusion: Note: Multiple comparisons are not used in random effects analyses -- we are interested in whether there is variability due to operator - not interested in which operators performed better, etc. (they were randomly chosen)

30 Rationale for F-test and critical region: RECALL: 1-Factor (Fixed-Effects) ANOVA Table (page 389) estimates + constant × - if no factor effects, we expect F ≈ 1; - if factor effects, we expect F > 1

31 Expected Mean Squares for 1-Factor ANOVA’s (p.979) EMS Source SS df MS Fixed Effects Random Effects Treatments SST t  1 MST Error SSE t(n  1) MSE Total TSS tn  Rationale for Test Statistic and Critical Region is the Same: Fixed or Random

32 DATA one; INPUT operator output; DATALINES; ; PROC GLM; CLASS operator; MODEL output=operator; RANDOM operator; TITLE ‘Operator Data: One Factor Random Effects Model'; RUN; These are data from an experiment studying the effect of four operators (chosen randomly) on the output of a particular machine.

33 The GLM Procedure Dependent Variable: output Sum of Source DF Squares Mean Square F Value Pr > F Model Error Corrected Total R-Square Coeff Var Root MSE output Mean Source DF Type I SS Mean Square F Value Pr > F operator The GLM Procedure Source Type III Expected Mean Square operator Var(Error) + 4 Var(operator) One Factor Random effects Model

34 Estimating Variance Components Solving for     we get: so, we estimate    by Also,

35 For OPERATOR Data,

36 RECALL: 2-Factor Fixed-Effects Model where

37 Expected Mean Squares for Fixed Effects 2-Factor ANOVA with Fixed Effects : A B AB Error Expected MSF-test MSA/MSE MSB/MSE MSAB/MSE

38 2-Factor Random Effects Model Assumptions: Sum-of-Squares obtained as in Fixed-Effects case

39 Expected Mean Squares for Random Effects 2-Factor ANOVA with Random Effects : A B AB Error Expected MS

40 To Test: we use F = Note: Test each of these 3 hypotheses (no matter whether H o :      is rejected)

41 2-Factor Random Effects ANOVA Table Source SS df MS F Main Effects A SSA a  1 B SSB b  1 Interaction AB SSAB ( a  1)(b  1) Error SSE ab(n  1) Total TSS abn 

42 Estimating Variance Components 2-Factor Random Effects Model (note error on page 986)

43 DATA one; INPUT operator filter loss; DATALINES; ; PROC GLM; CLASS operator filter; MODEL loss=operator filter operator*filter; TITLE ‘2-Factor Random Effects Model'; RANDOM operator filter operator*filter/test; RUN; Operator Filter Filtration Process: Response - % material lost through filtration A – Operator (randomly selected) ( a = ) B – Filter (randomly selected) ( b = ) n =

44 2-Factor Random Effects Model General Linear Models Procedure Dependent Variable: LOSS Sum of Mean Source DF Squares Square F Value Pr > F Model Error Corrected Total R-Square C.V. Root MSE LOSS Mean Source DF Type III SS Mean Square F Value Pr > F OPERATOR FILTER OPERATOR*FILTER Source Type III Expected Mean Square OPERATOR Var(Error) + 3 Var(OPERATOR*FILTER) + 9 Var(OPERATOR) FILTER Var(Error) + 3 Var(OPERATOR*FILTER) + 12 Var(FILTER) OPERATOR*FILTER Var(Error) + 3 Var(OPERATOR*FILTER) SAS Random-Effects Output (Filtration Data)

45 Tests of Hypotheses for Random Model Analysis of Variance Dependent Variable: LOSS Source: OPERATOR Error: MS(OPERATOR*FILTER) Denominator Denominator DF Type III MS DF MS F Value Pr > F Source: FILTER Error: MS(OPERATOR*FILTER) Denominator Denominator DF Type III MS DF MS F Value Pr > F Source: OPERATOR*FILTER Error: MS(Error) Denominator Denominator DF Type III MS DF MS F Value Pr > F SAS Random-Effects Output – continued “../test” option