1. 1 Experimental Statistics - week 14 Multiple Regression – miscellaneous topics

2. 2 Polynomial Regression: - we looked at this briefly in Lab - basically a multiple regression where the independent variables are powers of a single independent variable - use SAS to compute the independent variables x 2, x 3, …, x p

3. 3 Outlier Detection - there are tests for outliers - throwing away outliers should technically be done only when there is evidence that the values “do not belong”

4. 4 Use of Dummy Variables in Regression

5. 5 Example 6.1, Text page 268-269 Does a drug retains its potency after 1 year of storage? 2 groups: 1) fresh product 2) product stored for 1 year n = 10 observations from each group -- indep. samples) Fresh Stored 10.2 9.8 10.5 9.6.. Variable measured is potency reading Question: How would you compare groups?

6. 6 1-Factor ANOVA Model where    mean of fresh product    mean of 1-year old product We want to test:

7. 7 data ott269; input type$ y; datalines; F 10.2 F 10.5 F 10.3 F 10.8 F 9.8... S 9.6 S 9.8 S 9.9 ; proc glm; class type; model y=type; means type/lsd; title 'ANOVA -- Potency Data - page 269 (t-test)'; run;

8. 8 ANOVA -- Potency Data - page 269 (t-test) The GLM Procedure Class Level Information Class Levels Values type 2 F S The GLM Procedure Dependent Variable: y Sum of Source DF Squares Mean Square F Value Pr > F Model 1 1.45800000 1.45800000 17.95 0.0005 Error 18 1.46200000 0.08122222 Corrected Total 19 2.92000000 R-Square Coeff Var Root MSE potency Mean 0.499315 2.821734 0.284995 10.10000 Source DF Type I SS Mean Square F Value Pr > F type 1 1.45800000 1.45800000 17.95 0.0005 Source DF Type III SS Mean Square F Value Pr > F type 1 1.45800000 1.45800000 17.95 0.0005

9. 9 Since p =.0005 we reject and conclude that storage time does make a difference. t Tests (LSD) for y NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 18 Error Mean Square 0.081222 Critical Value of t 2.10092 Least Significant Difference 0.2678 Means with the same letter are not significantly different. t Grouping Mean N type A 10.3700 10 F B 9.8300 10 S Fresh product has higher potency on average. Also – estimated difference in means = 10.37 – 9.83 =.54

10. 10 quantitative Regression analysis – requires the independent variables to be quantitative Let’s consider recoding the group membership variable (i.e. F and S) into the numeric scores: 0 = fresh 1 = stored one year and running a regression analysis with this new “dummy” variable as a “quantitative” independent variable - let’s call the “dummy” variable x. Regression Model

11. 11 data ott269; input x y; datalines; 0 10.2 0 10.5 0 10.3 0 10.8 0 9.8... 1 9.6 1 9.8 1 9.9 ; proc reg; model y=x; title ‘Regression Analysis -- Potency Data - page 269'; run;

12. 12 The REG Procedure Dependent Variable: y Number of Observations Read 20 Number of Observations Used 20 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 1 1.45800 1.45800 17.95 0.0005 Error 18 1.46200 0.08122 Corrected Total 19 2.92000 Root MSE 0.28500 R-Square 0.4993 Dependent Mean 10.10000 Adj R-Sq 0.4715 Coeff Var 2.82173 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 10.37000 0.09012 115.06 <.0001 x 1 -0.54000 0.12745 -4.24 0.0005 Regression Equation:

13. 13 Note: the regression model On the basis of this model:

14. 14 Dummy Variables with More than 2 Groups Example: Balloon Data - 4 groups

15. 15 1122.4 2324.6 3120.3 4419.8 5324.3 6222.2 7228.5 8225.7 9320.2 10119.6 11228.8 12424.0 13417.1 14419.3 15324.2 16115.8 17218.3 18117.5 19418.7 20322.9 21116.3 22414.0 23416.6 24218.1 25218.9 26416.0 27220.1 28322.5 29316.0 30119.3 31115.9 32320.3 Balloon Data Col. 1-2 - observation number Col. 3 - color (1=pink, 2=yellow, 3=orange, 4=blue) Col. 4-7 - inflation time in seconds “Research Question”: Is the average time required to inflate the balloons the same for each color? Recall:

16. 16 GLM Procedure ANOVA --- Balloon Data Dependent Variable: time Sum of Source DF Squares Mean Square F Value Pr > F Model 3 126.1512500 42.0504167 3.85 0.0200 Error 28 305.6475000 10.9159821 Corrected Total 31 431.7987500 R-Square Coeff Var Root MSE time Mean 0.292153 16.31069 3.303934 20.25625 Source DF Type I SS Mean Square F Value Pr > F color 3 126.1512500 42.0504167 3.85 0.0200 Analysis using 1-factor ANOVA Model with 4 Groups Grouping Mean N color A 22.575 8 2(yellow) A A 21.875 8 3(orange) B 18.388 8 1(pink) B B 18.188 8 4(blue) LSD Results

17. 17 Dummy Variables For 4 groups -- 3 dummy variables needed.

18. 18 11 000 22.4 23 010 24.6 31 000 20.3 44 001 19.8 53 010 24.3 62 100 22.2 72 100 28.5 82 100 25.7 93 010 20.2 101 000 19.6 112 100 28.8 124 001 24.0 134 001 17.1 144 001 19.3 153 010 24.2 161 000 15.8 172 100 18.3 181 000 17.5 194 001 18.7 203 010 22.9 211 000 16.3 224 001 14.0 234 001 16.6 242 100 18.1 252 100 18.9 264 001 16.0 272 100 20.1 283 010 22.5 293 010 16.0 301 000 19.3 311 000 15.9 323 010 20.3 Col. 1-2 - observation number Col. 3 - color (1=pink, 2=yellow, 3=orange, 4=blue) Col. 5 X1 Col. 6 X2 Col. 7 X3 Col. 9-12 - inflation time in seconds Balloon Data Set with Dummy Variables:

19. 19 ANOVA --- Balloon Data using Dummy Variables The REG Procedure Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 3 126.15125 42.05042 3.85 0.0200 Error 28 305.64750 10.91598 Corrected Total 31 431.79875 Root MSE 3.30393 R-Square 0.2922 Dependent Mean 20.25625 Adj R-Sq 0.2163 Coeff Var 16.31069 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 18.38750 1.16812 15.74 <.0001 x1 1 4.18750 1.65197 2.53 0.0171 x2 1 3.48750 1.65197 2.11 0.0438 x3 1 -0.20000 1.65197 -0.12 0.9045

20. 20 According to the Model:

21. 21 Problem 1

22. 22 Multiple Comparisons for Fixed Effect (Inspection Level) -- Use MSAB in place of MSE where ▪ N denotes the # of observations involved in the computation of a marginal mean ▪ v denotes the df associated with AB interaction Recall: Mixed Model

23. 23 When comparing means using a multiple comparison procedure (i.e. LSD, Bonferroni, etc.) use the MS used in the denominator of the associated F-test SAS always gives multiple comparison results using MSE General Rule Note:

24. 24 PROC GLM; class group ewe week; TITLE 'Ewe Study'; model milk=group ewe(group) week group*week; random ewe(group)/test; means group week/lsd; output out=newe r=resmilk; RUN; Ewe Data – problem 1 The GLM Procedure Dependent Variable: milk Sum of Source DF Squares Mean Square F Value Pr > F Model 23 388848.1481 16906.4412 8.54 <.0001 Error 30 59411.1111 1980.3704 Corrected Total 53 448259.2593 R-Square Coeff Var Root MSE milk Mean 0.867463 21.57157 44.50135 206.2963 Source DF Type I SS Mean Square F Value Pr > F group 2 256803.7037 128401.8519 64.84 <.0001 ewe(group) 6 114788.8889 19131.4815 9.66 <.0001 week 5 2970.3704 594.0741 0.30 0.9090 group*week 10 14285.1852 1428.5185 0.72 0.6983

25. 25 Ewe Study The GLM Procedure Source Type III Expected Mean Square group Var(Error) + 6 Var(ewe(group)) + Q(group,group*week) ewe(group) Var(Error) + 6 Var(ewe(group)) week Var(Error) + Q(week,group*week) group*week Var(Error) + Q(group*week) Ewe Study The GLM Procedure Tests of Hypotheses for Mixed Model Analysis of Variance Dependent Variable: milk Source DF Type III SS Mean Square F Value Pr > F * group 2 256804 128402 6.71 0.0295 Error 6 114789 19131 Error: MS(ewe(group)) * This test assumes one or more other fixed effects are zero. Source DF Type III SS Mean Square F Value Pr > F ewe(group) 6 114789 19131 9.66 <.0001 * week 5 2970.370370 594.074074 0.30 0.9090 group*week 10 14285 1428.518519 0.72 0.6983 Error: MS(Error) 30 59411 1980.370370 * This test assumes one or more other fixed effects are zero.

26. 26 Ewe Study The GLM Procedure Source Type III Expected Mean Square group Var(Error) + 6 Var(ewe(group)) + Q(group,group*week) ewe(group) Var(Error) + 6 Var(ewe(group)) week Var(Error) + Q(week,group*week) group*week Var(Error) + Q(group*week) Ewe Study The GLM Procedure Tests of Hypotheses for Mixed Model Analysis of Variance Dependent Variable: milk Source DF Type III SS Mean Square F Value Pr > F * group 2 256804 128402 6.71 0.0295 Error 6 114789 19131 Error: MS(ewe(group)) * This test assumes one or more other fixed effects are zero. Source DF Type III SS Mean Square F Value Pr > F ewe(group) 6 114789 19131 9.66 <.0001 * week 5 2970.370370 594.074074 0.30 0.9090 group*week 10 14285 1428.518519 0.72 0.6983 Error: MS(Error) 30 59411 1980.370370 * This test assumes one or more other fixed effects are zero.

27. 27 t Tests (LSD) for Group Differences NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 30 Error Mean Square 1980.37 Critical Value of t 2.04227 Least Significant Difference 30.295 Means with the same letter are not significantly different. t Grouping Mean N group A 291.67 18 1 B 204.44 18 2 C 122.78 18 3

28. 28 t Tests (LSD) for Group Differences NOTE: This test controls the Type I comparisonwise error rate, not the experimentwise error rate. Alpha 0.05 Error Degrees of Freedom 30 Error Mean Square 1980.37 Critical Value of t 2.04227 Least Significant Difference 30.295 Means with the same letter are not significantly different. t Grouping Mean N group A 291.67 18 1 B 204.44 18 2 C 122.78 18 3 “Error” Degrees of Freedom = “Error” Mean Square = Critical Value of t = Least Significant Difference = Corrected:

29. 29 residuals milk Ewe Data interaction plot – Milk Production by Week Why non-normal?

30. 30 Ewe Data – Box Plots

31. 31 Problem 2

32. 32 Outliers Removed Kidney Data Original Model Log Model

33. 33 Kidney Data Original ModelLog Model R 2 =.855R 2 =.866

34. 34 Kidney Data Original ModelOutliers Removed R 2 =.855R 2 =.871

35. 35 Kidney Data Log Model R 2 =.866 Log Model – Outliers Removed R 2 =.901

36. 36 Problem 3

37. 37 Original Variables Log Survival vs Other Original Variables Survival Data

38. 38 Original Variables Survival vs Square of Independent Variables Survival Data

39. 39 Dependent Variable: Log(Survival) Number in Adjusted Model R-Square R-Square Variables in Model 4 0.7122 0.7229 clot prog enzyme liver 3 0.7115 0.7196 clot prog enzyme 5 0.7104 0.7239 clot prog enzyme liver age 4 0.7098 0.7207 clot prog enzyme age 3 0.6781 0.6871 prog enzyme liver 4 0.6758 0.6879 prog enzyme liver age 2 0.6412 0.6479 prog enzyme 3 0.6388 0.6489 prog enzyme age 4 0.5357 0.5531 clot enzyme liver age Dependent Variable: Survival Number in Adjusted Model R-Square R-Square Variables in Model 4 0.6999 0.7112 clot prog enzyme liver 5 0.6970 0.7112 clot prog enzyme liver age 3 0.6908 0.6995 clot prog enzyme 4 0.6878 0.6995 clot prog enzyme age 3 0.6520 0.6618 prog enzyme liver 4 0.6487 0.6618 prog enzyme liver age 2 0.5750 0.5829 prog enzyme 3 0.5709 0.5829 prog enzyme age

40. 40 Survival Data – Log(Survival) Model without Age

41. 41 Grades “Conditional” – under assumption of good performance on next Thursday’s lab Final Exam -- optional (scheduled for 8:00 AM – 11:00 AM Friday, May 6) -- “in class” exam -- will be averaged in equally with the other 2 exams to comprise 75% of grade - can raise or lower final grade From Syllabus GRADE COMPUTATION: Exam Grades (75%) Daily Assignments (25%)

42. 42 W e showed that 1-factor ANOVA can be run using regression analysis with dummy variables. Question: What’s the benefit? Answer: Dummy variables can be mixed in with regular quantitative variables to give a combination of regression and ANOVA analyses. Dummy Variables

43. 43 For 4 groups -- 3 dummy variables needed. Dummy Variables for 4 Groups: 0, 0, 0 → group 1 1, 0, 0 → group 2 0, 1, 0 → group 3 0, 0, 1 → group 4

44. 44 Dummy Variables for 4 Groups:

45. 45 1122.4 2324.6 3120.3 4419.8 5324.3 6222.2 7228.5 8225.7 9320.2 10119.6 11228.8 12424.0 13417.1 14419.3 15324.2 16115.8 17218.3 18117.5 19418.7 20322.9 21116.3 22414.0 23416.6 24218.1 25218.9 26416.0 27220.1 28322.5 29316.0 30119.3 31115.9 32320.3 Balloon Data Col. 1-2 - observation number Col. 3 - color (1=pink, 2=yellow, 3=orange, 4=blue) Col. 4-7 - inflation time in seconds “Research Question”: Is the average time required to inflate the balloons the same for each color?

46. 46 GLM Procedure ANOVA --- Balloon Data Dependent Variable: time Sum of Source DF Squares Mean Square F Value Pr > F Model 3 126.1512500 42.0504167 3.85 0.0200 Error 28 305.6475000 10.9159821 Corrected Total 31 431.7987500 R-Square Coeff Var Root MSE time Mean 0.292153 16.31069 3.303934 20.25625 Source DF Type I SS Mean Square F Value Pr > F color 3 126.1512500 42.0504167 3.85 0.0200 Analysis using 1-factor ANOVA Model with 4 Groups Grouping Mean N color A 22.575 8 2(yellow) A A 21.875 8 3(orange) B 18.388 8 1(pink) B B 18.188 8 4(blue) LSD Results

47. 47 11 000 22.4 23 010 24.6 31 000 20.3 44 001 19.8 53 010 24.3 62 100 22.2 72 100 28.5 82 100 25.7 93 010 20.2 101 000 19.6 112 100 28.8 124 001 24.0 134 001 17.1 144 001 19.3 153 010 24.2 161 000 15.8 172 100 18.3 181 000 17.5 194 001 18.7 203 010 22.9 211 000 16.3 224 001 14.0 234 001 16.6 242 100 18.1 252 100 18.9 264 001 16.0 272 100 20.1 283 010 22.5 293 010 16.0 301 000 19.3 311 000 15.9 323 010 20.3 Col. 1-2 - observation number Col. 3 - color (1=pink, 2=yellow, 3=orange, 4=blue) Col. 5 X1 Col. 6 X2 Col. 7 X3 Col. 9-12 - inflation time in seconds Balloon Data Set with Dummy Variables:

48. 48 ANOVA --- Balloon Data using Dummy Variables The REG Procedure Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 3 126.15125 42.05042 3.85 0.0200 Error 28 305.64750 10.91598 Corrected Total 31 431.79875 Root MSE 3.30393 R-Square 0.2922 Dependent Mean 20.25625 Adj R-Sq 0.2163 Coeff Var 16.31069 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 18.38750 1.16812 15.74 <.0001 x1 1 4.18750 1.65197 2.53 0.0171 x2 1 3.48750 1.65197 2.11 0.0438 x3 1 -0.20000 1.65197 -0.12 0.9045

49. 49 Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 18.38750 1.16812 15.74 <.0001 x1 1 4.18750 1.65197 2.53 0.0171 x2 1 3.48750 1.65197 2.11 0.0438 x3 1 -0.20000 1.65197 -0.12 0.9045 (i.e. “pink” ≠ “yellow”) i.e. conclude “pink” ≠ “orange” i.e. we cannot conclude “pink” and “blue” are different

50. 50 Dummy Variables for 4 Groups:

51. 51 Grouping Mean N color A 22.575 8 2(yellow) A A 21.875 8 3(orange) B 18.388 8 1(pink) B B 18.188 8 4(blue) LSD Results

52. 52 Recall: There was an issue with order in which balloons were inflated - lab assistant “improved” - we tried to account for this by randomizing run order run order inflation time

53. 53 Put “run order” in the model. PROC REG; MODEL y=x1 x2 x3 id; TITLE 'ANOVA --- Balloon Data using Dummy Variables and Run Order'; RUN; t-tests in a MLR model test the effects of individual independent variables while all other independent variables stay constant - in this example, we can test for color effects while “adjusting for” or taking out the effect of run order Another Strategy: Recall:

54. 54 Sum of Mean Source DF Squares Square F Value Pr > F Model 4 249.57490 62.39373 9.24 <.0001 Error 27 182.22385 6.74903 Corrected Total 31 431.79875 Root MSE 2.59789 R-Square 0.5780 Dependent Mean 20.25625 Adj R-Sq 0.5155 Coeff Var 12.82513 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 21.85333 1.22494 17.84 <.0001 x1 1 4.05420 1.29932 3.12 0.0043 x2 1 3.75410 1.30044 2.89 0.0076 x3 1 -0.12002 1.29908 -0.09 0.9271 id 1 -0.21328 0.04987 -4.28 0.0002 PROC REG ANOVA Table – Balloon Data We can see that: - pink is still significantly different from yellow and orange and not significantly different from blue - there is a significant “run order” effect

55. 55 ANOVA --- Balloon Data with No Randomization The GLM Procedure Dependent Variable: time Sum of Source DF Squares Mean Square F Value Pr > F Model 3 157.9609375 52.6536458 5.37 0.0048 Error 28 274.4987500 9.8035268 Corrected Total 31 432.4596875 R-Square Coeff Var Root MSE time Mean 0.365262 15.46440 3.131058 20.24688 Source DF Type I SS Mean Square F Value Pr > F color 3 157.9609375 52.6536458 5.37 0.0048 t Tests (LSD) for time Alpha 0.05 Error Degrees of Freedom 28 Error Mean Square 9.803527 Critical Value of t 2.04841 Least Significant Difference 3.2068 Means with the same letter are not significantly different. t Grouping Mean N color A 23.475 8 1 A B A 21.088 8 2 B B C 18.625 8 4 C C 17.800 8 3

56. 56 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 3 157.96094 52.65365 5.37 0.0048 Error 28 274.49875 9.80353 Corrected Total 31 432.45969 Root MSE 3.13106 R-Square 0.3653 Dependent Mean 20.24688 Adj R-Sq 0.2973 Coeff Var 15.46440 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 23.47500 1.10700 21.21 <.0001 x1 1 -2.38750 1.56553 -1.53 0.1385 x2 1 -5.67500 1.56553 -3.62 0.0011 x3 1 -4.85000 1.56553 -3.10 0.0044 Non-randomized Balloon Data

57. 57 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 4 158.41667 39.60417 3.90 0.0126 Error 27 274.04302 10.14974 Corrected Total 31 432.45969 Root MSE 3.18587 R-Square 0.3663 Dependent Mean 20.24688 Adj R-Sq 0.2724 Coeff Var 15.73510 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 23.70938 1.57865 15.02 <.0001 x1 1 -1.97083 2.53061 -0.78 0.4429 x2 1 -4.84167 4.24308 -1.14 0.2639 x3 1 -3.60000 6.11036 -0.59 0.5606 id 1 -0.05208 0.24579 -0.21 0.8338 Non-randomized Balloon Data

58. 58 DATA survival; INPUT clot prog enzyme liver age gender alch1 alch2 survival; DATALINES; 6.7 62812.5950010695 5.159661.7039000403 7.457832.1655000710 6.573412.0148000349 7.8651154.30450012343 5.838721.4265110348. ; PROC reg; MODEL survival=clot prog enzyme liver age/selection=adjrsq; output out=new r=ressurv p=predsurv; RUN; Survival Data PROC reg; MODEL lgsurv=clot prog enzyme liver age/selection=adjrsq; output out=new r=ressvlg p=predsvlg; RUN; Gender: 0=male, 1=female Alcohol Use alch1 alch2 None 0 0 Moderate 1 0 Heavy 0 1

59. 59 Dependent Variable: survival Number in Adjusted Model R-Square R-Square Variables in Model 6 0.7611 0.7745 clot prog enzyme liver alch1 alch2 5 0.7606 0.7718 clot prog enzyme liver alch2 7 0.7592 0.7749 clot prog enzyme liver age alch1 alch2 7 0.7591 0.7748 clot prog enzyme liver gender alch1 alch2 6 0.7587 0.7723 clot prog enzyme liver age alch2 6 0.7587 0.7722 clot prog enzyme liver gender alch2 8 0.7571 0.7753 clot prog enzyme liver age gender alch1 alch2 7 0.7568 0.7727 clot prog enzyme liver age gender alch2 5 0.7416 0.7536 clot prog enzyme alch1 alch2 Adjusted R-Square Selection Method Dependent Variable: log(survival) Number in Adjusted Model R-Square R-Square Variables in Model 6 0.7649 0.7781 clot prog enzyme liver gender alch2 7 0.7634 0.7789 clot prog enzyme liver gender alch1 alch2 5 0.7628 0.7738 clot prog enzyme liver alch2 7 0.7627 0.7782 clot prog enzyme liver age gender alch2 6 0.7614 0.7747 clot prog enzyme liver alch1 alch2 8 0.7612 0.7790 clot prog enzyme liver age gender alch1 alch2 6 0.7605 0.7740 clot prog enzyme liver age alch2 7 0.7591 0.7749 clot prog enzyme liver age alch1 alch2

60. 60 Dependent Variable: lgsurv Number of Observations Read 108 Number of Observations Used 108 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model 5 19.79454 3.95891 63.66 <.0001 Error 102 6.34335 0.06219 Corrected Total 107 26.13789 Root MSE 0.24938 R-Square 0.7573 Dependent Mean 6.36909 Adj R-Sq 0.7454 Coeff Var 3.91545 Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept 1 4.31842 0.12650 34.14 <.0001 prog 1 0.01180 0.00157 7.51 <.0001 enzyme 1 0.01216 0.00135 9.02 <.0001 liver 1 0.13055 0.03001 4.35 <.0001 gender 1 0.05179 0.04935 1.05 0.2964 alch2 1 0.32690 0.06095 5.36 <.0001

61. 61 None: (0,0) mean survival = 640.5 Moderate: (1,0) mean survival = 608.4 Severe: (0,1) mean survival = 815.2 Alcohol Use