Chapter 10: Inferential for Regression 1

10.1: Simple Linear Regression 10.2: More Detail about Simple Linear Regression Goals Describe the simple linear regression model (review – Ch. 2). Be able to perform the method (with the output from software packages, Lab 8). Use diagnostic plots to check the assumptions. Be able to perform inference on the slope (Confidence interval and hypothesis test). Be able to determine if there is an association between the response and explanatory variables. Be able to perform a hypothesis test using the correlation coefficient. Be able to state the similarities and differences between a confidence interval for a mean response and a prediction interval and in which situations each would be used (if there is time) 2

Conditions for Linear Regression We have n (x,y) pairs. For any fixed x, y ~ N(  y,  ) Each y i is independent of the other y j ’s.  y =  0 +  1 x 3

Model for Linear Regression 4 y i =  0 +  1 x +  i Data = Fit + Error

Linear Regression 5 ŷ = b 0 + b 1 x ŷ is an unbiased estimator for  y b 0 is an unbiased estimator for  0 b 1 is an unbiased estimator for  1

Linear Regression b 0 = ȳ - b 1 x̄ 6

Other SS and df 7

ANOVA table for Linear Regression SourcedfSSMS Model (Regression) 1Σ(ŷ i - ȳ) 2 Errorn – 2Σ(y i - ŷ i ) 2 Totaln - 1Σ(y i - ȳ) 2 8

Conditions for Linear Regression SRS Observations are independent of each other. The relationship is linear in the population. The response, y, is normally distribution around the population regression line. The standard deviation of the response is constant. Important plots: – Scatter plot – Residual plot – Histogram/Normal quantile plot of the residuals. 9

Residual Plots 10

Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 11

Example: Linear Regression 1 (cont.) x: y: x: y:

Example: SLR 1 - Scatterplot 13

Example: SLR 1 – Residual Plot 14

Example: SLR 1 – Normality 15

Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 16

Example: SLR 1 – Fitted Line x: y: x: y: r = s x = s y = y̅ = x̅ =

Example: SLR – fitted line 18

Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 19

Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 20

Example: SLR - s x: y: x: y: Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

Example: Linear Regression 1 The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. a)Verify the assumptions required for linear regression. b)Determine the equation of the fitted line. c)What is a point estimate of the true average cetane number whose iodine value is 100? d)Estimate the value of σ. e)What proportion of the observed variation in cetane number that can be attributed to the iodine value? 22

Example: SLR 1 x: y: x: y: Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

Confidence Interval Point estimates – b 0 is an unbiased estimator for  0 – b 1 is an unbiased estimator for  1 Assumptions – SRS – linearity – Constant standard deviation of residuals – Normality If y is normal, then both b 0 and b 1 are normal If y is not normal, there is still CLT 24

Standard deviation for b 1 25

Confidence Interval for  1 26

Example: SLR 1 - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. e)What is the 95% Confidence Interval for the population slope? f)Is the model useful (that is, is there a useful linear relationship between iodine value and cetane number)? 27

Example: SLR 1 x: y: x: y: Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total b 1 = S xx =

Example: SLR 1 – CI. We are 95% confidence that the population slope is between and

Example: SLR – fitted line 30

LR Hypothesis Test: Summary Alternative Hypothesis P-Value Upper-tailed H a :  1 > Δ P(T ≥ t) Lower-tailed H a :  1 < Δ P(T ≤ t) two-sided H a :  1 ≠ Δ 2P(T ≥ |t|) 31

Example: SLR 1 - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. e)What is the 95% Confidence Interval for the population slope? f)Is the model useful (that is, is there a useful linear relationship between iodine value and cetane number)? 32

Example: SLR 1 - HT The data does provide strong support (P = 2.13 x ) to the claim that there is a linear relationship between iodine value and cetane number. 33

ANOVA table for Linear Regression SourcedfSSMS Model (Regression) 1Σ(ŷ i - ȳ) 2 Errorn – 2Σ(y i - ŷ i ) 2 Totaln - 1Σ(y i - ȳ) 2 34

LR Hypothesis Test: Summary 35

Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. g)Perform the hypothesis test using the F test statistic. h)Perform the hypothesis using the population correlation coefficient 36

Example: LR – Inference - ANOVA Parameter Estimates VariableDF Parameter Estimate Standard Error t ValuePr > |t| Intercept <.0001 iodine <.0001 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

Example: LR – Inference (cont) The data does provide strong support (P = 2.09 x ) to the claim that there is a linear relationship between iodine value and cetane number. 38

Inference for Correlation: Assumptions (x,y) are independent (x,y) is normal Linear relationship between x and y Constant variance for the residuals. 39

LR Hypothesis Test: Summary Alternative Hypothesis P-Value Upper-tailed H a :  > Δ P(T ≥ t) Lower-tailed H a :  < Δ P(T ≤ t) two-sided H a :  ≠ Δ 2P(T ≥ |t|) 40

Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. g)Perform the hypothesis test using the F test statistic. h)Perform the hypothesis using the population correlation coefficient. 41

Example: LR – Inference - ANOVA Parameter Estimates VariableDF Parameter Estimate Standard Error t ValuePr > |t| Intercept <.0001 iodine <.0001 Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total

Example: LR - Inference The data does provide strong support (P = 2.12 x ) to the claim that there is a linear relationship between iodine value and cetane number. 43

SE µ̂h 44

Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. i)What is the 95% confidence interval for the cetane number with the iodine value is 100. j)Predict the cetane number for the next sample of biofuel that contains an iodine value of

Example: LR – Inference Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total  ̂ = S xx = x̅ =

Example: SLR (cont) We are 95% confident that the population mean cetane number is between and when the iodine value is

SE ŷ 48

Example: LR - Inference The cetane number is a critical property in specifying the ignition quality of a fuel used in a diesel engine. Determination of this number for a biodiesel fuel is expensive and time-consuming. Therefore a way of predicting this number is wanted. The data on the next slide is x = iodine value (g) and y = cetane number for a sample of 14 biofuels. The iodine value is the amount of iodine necessary to saturate a sample of 100g of oil. i)What is the 95% confidence interval for the cetane number with the iodine value is 100. j)Predict the cetane number for the next sample of biofuel that contains an iodine value of

Example: LR – Inference Analysis of Variance SourceDFSum of Squares Mean Square F Value Pr > F Model <.0001 Error Corrected Total  ̂ = S xx = x̅ =

Example: SLR (cont) We are 95% confident that the next cetane number is between and when the iodine value is 100. Mean response: (52.754, ) Prediction interval: ( ) 51

CI for mean response Prediction interval 52

Example: Confidence/Prediction Band 53

Multiple Regression: Examples 1 1)A portrait studio operates in cities of medium size and specializes in portraits of children. They want to open a store in a other similar community, but want to be able to predict sales. 2)So that only students that succeed are accepted into college, the registrar’s office wants to be able to predict GPA from entering high school students. 3)A researcher studied the effects of the charge rate and the temperature on the life of a new type of power cell in a preliminary small-scale experiment. 54

Multiple Regression: Examples 2 4)An experiment was run to investigate the yield of tomato plants as a function of the amount of water levels. A series of plots were randomized to different water levels and at the end of the season, the yield of the plants was determined. 5)Fernandez-Juricic et al. (2003) examined the effect of human disturbance on the nesting of house sparrows (Passer domesticus). They counted breeding sparrows per hectare in 18 parks in Madrid, Spain, and also counted the number of people per minute walking through each park (both measurement variables). 55