1. Chapter 10: Inferential for Regression http://jonfwilkins.blogspot.com/2011_08_01_archive.html 1

2. 10.1: Simple Linear Regression 10.2: More Detail about Simple Linear Regression Goals Describe the simple linear regression model (review – Ch. 2). Be able to perform the method (with the output from software packages, Lab 8). Use diagnostic plots to check the assumptions. Be able to perform inference on the slope (Confidence interval and hypothesis test). Be able to determine if there is an association between the response and explanatory variables. Be able to perform a hypothesis test using the correlation coefficient. Be able to state the similarities and differences between a confidence interval for a mean response and a prediction interval and in which situations each would be used (if there is time) 2

3. Conditions for Linear Regression We have n (x,y) pairs. For any fixed x, y ~ N(  y,  ) Each y i is independent of the other y j ’s.  y =  0 +  1 x 3

4. Model for Linear Regression 4 y i =  0 +  1 x +  i Data = Fit + Error

5. Linear Regression 5 ŷ = b 0 + b 1 x ŷ is an unbiased estimator for  y b 0 is an unbiased estimator for  0 b 1 is an unbiased estimator for  1

6. Linear Regression b 0 = ȳ - b 1 x̄ 6

7. Other SS and df 7

8. ANOVA table for Linear Regression SourcedfSSMS Model (Regression) 1Σ(ŷ i - ȳ) 2 Errorn – 2Σ(y i - ŷ i ) 2 Totaln - 1Σ(y i - ȳ) 2 8

9. Conditions for Linear Regression SRS Observations are independent of each other. The relationship is linear in the population. The response, y, is normally distribution around the population regression line. The standard deviation of the response is constant. Important plots: – Scatter plot – Residual plot – Histogram/Normal quantile plot of the residuals. 9

10. Residual Plots 10

11. Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 11

12. Example: Linear Regression 1 (cont.) x:132.0129.0120.0113.2105.092.084.0 y:46.048.051.052.154.052.059.0 x:83.288.459.080.081.571.069.2 y:58.761.664.061.454.658.858.0 12

13. Example: SLR 1 - Scatterplot 13

14. Example: SLR 1 – Residual Plot 14

15. Example: SLR 1 – Normality 15

16. Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 16

17. Example: SLR 1 – Fitted Line x: 132.0129.0120.0113.2105.092.084.0 y: 46.048.051.052.154.052.059.0 x: 83.288.459.080.081.571.069.2 y: 58.761.664.061.454.658.858.0 r = -0.88925s x = 22.8755s y = 5.3864 y̅ = 55.657x̅ = 93.393 17

18. Example: SLR – fitted line 18

19. Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 19

20. Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 20

21. Example: SLR - s x: 132.0129.0120.0113.2105.092.084.0 y: 46.048.051.052.154.052.059.0 x: 83.288.459.080.081.571.069.2 y: 58.761.664.061.454.658.858.0 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429 21

22. Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 22

23. Example: SLR 1 x: 132.0129.0120.0113.2105.092.084.0 y: 46.048.051.052.154.052.059.0 x: 83.288.459.080.081.571.069.2 y: 58.761.664.061.454.658.858.0 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429 23

24. Confidence Interval Point estimates – b 0 is an unbiased estimator for  0 – b 1 is an unbiased estimator for  1 Assumptions – SRS – linearity – Constant standard deviation of residuals – Normality If y is normal, then both b 0 and b 1 are normal If y is not normal, there is still CLT 24

25. Standard deviation for b 1 25

26. Confidence Interval for  1 26

27. Example: SLR 1 - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. e)What is the 95% Confidence Interval for the population slope? f)Is the model useful (that is, is there a useful linear relationship between iodine value and cetane number)? 27

28. Example: SLR 1 x: 132.0129.0120.0113.2105.092.084.0 y: 46.048.051.052.154.052.059.0 x: 83.288.459.080.081.571.069.2 y: 58.761.664.061.454.658.858.0 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429 b 1 = -0.209 S xx = 6802.77 28

29. Example: SLR 1 – CI. We are 95% confidence that the population slope is between -0.277 and -0.141. 29

30. Example: SLR – fitted line 30

31. LR Hypothesis Test: Summary Alternative Hypothesis P-Value Upper-tailed H a :  1 > Δ P(T ≥ t) Lower-tailed H a :  1 < Δ P(T ≤ t) two-sided H a :  1 ≠ Δ 2P(T ≥ |t|) 31

32. Example: SLR 1 - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. e)What is the 95% Confidence Interval for the population slope? f)Is the model useful (that is, is there a useful linear relationship between iodine value and cetane number)? 32

33. Example: SLR 1 - HT The data does provide strong support (P = 2.13 x 10 -5 ) to the claim that there is a linear relationship between iodine value and cetane number. 33

34. ANOVA table for Linear Regression SourcedfSSMS Model (Regression) 1Σ(ŷ i - ȳ) 2 Errorn – 2Σ(y i - ŷ i ) 2 Totaln - 1Σ(y i - ȳ) 2 34

35. LR Hypothesis Test: Summary 35

36. Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. g)Perform the hypothesis test using the F test statistic. h)Perform the hypothesis using the population correlation coefficient 36

37. Example: LR – Inference - ANOVA Parameter Estimates VariableDF Parameter Estimate Standard Error t ValuePr > |t| Intercept175.212432.9836325.21<.0001 iodine1-0.209390.03109-6.73<.0001 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429 37

38. Example: LR – Inference (cont) The data does provide strong support (P = 2.09 x 10 -5 ) to the claim that there is a linear relationship between iodine value and cetane number. 38

39. Inference for Correlation: Assumptions (x,y) are independent (x,y) is normal Linear relationship between x and y Constant variance for the residuals. 39

40. LR Hypothesis Test: Summary Alternative Hypothesis P-Value Upper-tailed H a :  > Δ P(T ≥ t) Lower-tailed H a :  < Δ P(T ≤ t) two-sided H a :  ≠ Δ 2P(T ≥ |t|) 40

41. Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. g)Perform the hypothesis test using the F test statistic. h)Perform the hypothesis using the population correlation coefficient. 41

42. Example: LR – Inference - ANOVA Parameter Estimates VariableDF Parameter Estimate Standard Error t ValuePr > |t| Intercept175.212432.9836325.21<.0001 iodine1-0.209390.03109-6.73<.0001 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429 42

43. Example: LR - Inference The data does provide strong support (P = 2.12 x 10 -5 ) to the claim that there is a linear relationship between iodine value and cetane number. 43

44. SE µ̂h 44

45. Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. i)What is the 95% confidence interval for the cetane number with the iodine value is 100. j)Predict the cetane number for the next sample of biofuel that contains an iodine value of 100. 45

46. Example: LR – Inference Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429  ̂ = 54.313 S xx = 6802.77 x̅ = 93.393 46

47. Example: SLR (cont) We are 95% confident that the population mean cetane number is between 52.754 and 55.872 when the iodine value is 100. 47

48. SE ŷ 48

49. Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. i)What is the 95% confidence interval for the cetane number with the iodine value is 100. j)Predict the cetane number for the next sample of biofuel that contains an iodine value of 100. 49

50. Example: LR – Inference Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model1298.25443 45.35<.0001 Error1278.919866.57665 Corrected Total 13377.17429  ̂ = 54.313 S xx = 6802.77 x̅ = 93.393 50

51. Example: SLR (cont) We are 95% confident that the next cetane number is between 48.512 and 60.114 when the iodine value is 100. Mean response: (52.754, 55.872) Prediction interval: (48.512. 60.114) 51

52. CI for mean response Prediction interval 52

53. Example: Confidence/Prediction Band 53

54. Multiple Regression: Examples 1 1)A portrait studio operates in cities of medium size and specializes in portraits of children. They want to open a store in a other similar community, but want to be able to predict sales. 2)So that only students that succeed are accepted into college, the registrar’s office wants to be able to predict GPA from entering high school students. 3)A researcher studied the effects of the charge rate and the temperature on the life of a new type of power cell in a preliminary small-scale experiment. 54

55. Multiple Regression: Examples 2 4)An experiment was run to investigate the yield of tomato plants as a function of the amount of water levels. A series of plots were randomized to different water levels and at the end of the season, the yield of the plants was determined. 5)Fernandez-Juricic et al. (2003) examined the effect of human disturbance on the nesting of house sparrows (Passer domesticus). They counted breeding sparrows per hectare in 18 parks in Madrid, Spain, and also counted the number of people per minute walking through each park (both measurement variables). 55