2. Simple Linear Regression Analysis
Chapter Eleven
Simple Linear Regression Analysis
McGraw-Hill/Irwin
Copyright © 2004 by The McGraw-Hill Companies, Inc. All rights reserved.

3. Simple Linear Regression
11.1 The Simple Linear Regression Model
11.2 The Least Squares Point Estimates
11.3 Model Assumptions, Mean Squared Error, Std. Error
11.4 Testing Significance of Slope and y-Intercept
11.5 Confidence Intervals and Prediction Intervals
11.6 The Coefficient of Determination and Correlation
11.7 An F Test for the Simple Linear Regression Model
*11.8 Checking Regression Assumptions by Residuals
*11.9 Some Shortcut Formulas

4. 11.1 The Simple Linear Regression Model
y|x = b0 + b1x + e is the mean value of the dependent variable y when the value of the independent variable is x.
b0 is the y-intercept, the mean of y when x is 0.
b1 is the slope, the change in the mean of y per unit change in x.
e is an error term that describes the effect on y of all factors other than x.

5. The Simple Linear Regression Model Illustrated

6. 11.2 The Least Squares Point Estimates
Estimation/Prediction Equation:
Least squares point estimate of the slope 1
Least squares point estimate of the y-intercept 0

7. Example: The Least Squares Point Estimates
Prediction (x = 40)
Slope b1
y-Intercept b0

8. 11.3 The Regression Model Assumptions
Assumptions about the model error terms, ’s
Mean Zero The mean of the error terms is equal to 0.
Constant Variance The variance of the error terms s2 is, the same for all values of x.
Normality The error terms follow a normal distribution for all values of x.
Independence The values of the error terms are statistically independent of each other.

9. Regression Model Assumptions Illustrated

10. Mean Square Error and Standard Error
Sum of Squared Errors
Mean Square Error, point estimate of residual variance s2
Standard Error, point estimate of residual standard deviation s
Example The Fuel Consumption Case

11. 11.4 Significance Test and Estimation for Slope
If the regression assumptions hold, we can reject H0: 1 = 0 at the  level of significance (probability of Type I error equal to ) if and only if the appropriate rejection point condition holds or, equivalently, if the corresponding p-value is less than .
Alternative
Reject H0 if:
p-Value
Test Statistic
100(1-)% Confidence Interval for 1
t, t/2 and p-values are based on n – 2 degrees of freedom.

12. Significance Test and Estimation for y-Intercept
If the regression assumptions hold, we can reject H0: 0 = 0 at the  level of significance (probability of Type I error equal to ) if and only if the appropriate rejection point condition holds or, equivalently, if the corresponding p-value is less than .
Alternative
Reject H0 if:
p-Value
Test Statistic
100(1-)% Conf Interval for 0
t, t/2 and p-values are based on n – 2 degrees of freedom.

13. Example: Inferences About Slope and y-Intercept
Tests
Intervals
Example 11.7
The Fuel Consumption Case Excel Output

14. 11.5 Confidence and Prediction Intervals
Prediction (x = x0)
Distance Value
If the regression assumptions hold,
100(1 - a)% confidence interval for the mean value of y, my|xo
100(1 - a)% prediction interval for an individual value of y
ta/2 is based on n-2 degrees of freedom

15. Example: Confidence and Prediction Intervals
Example The Fuel Consumption Case
Minitab Output (predicted FuelCons when Temp, x = 40)
Predicted Values
Fit StDev Fit % CI % PI
( , ) ( , )

16. 11.6 The Simple Coefficient of Determination
The simple coefficient of determination r2 is
r2 is the proportion of the total variation in y explained by the simple linear regression model

17. The Simple Correlation Coefficient
The simple correlation coefficient measures the strength of the linear relationship between y and x and is denoted by r.
Where, b1 is the slope of the least squares line.
Example 11.15
Fuel Consumption
Excel Output

18. Different Values of the Correlation Coefficient

19. 11.7 F Test for Simple Linear Regression Model
To test H0: 1= 0 versus Ha: 1 0 at the  level of significance
Test Statistic:
Reject H0 if
F(model) > Fa or
p-value < a
Fa is based on 1 numerator and n-2 denominator degrees of freedom.

20. Example: F Test for Simple Linear Regression
Example The Fuel Consumption Case Excel Output
F-test at  = 0.05 level of significance
Test Statistic:
Reject H0 at  level of significance, since
Fa is based on 1 numerator and 6 denominator degrees of freedom.

21. *11.8 Checking the Regression Assumptions by Residual Analysis
For an observed value of y, the residual is
where the predicted value of y is calculated as
If the regression assumptions hold, the residuals should look like a random sample from a normal distribution with mean 0 and variance 2.
Residual Plots
Residuals versus independent variables
Residuals versus predicted y’s
Residuals in time order (if the response is a time series)
Histogram of residuals
Normal plot of the residuals

22. Checking the Constant Variance Assumption
Example 11.18: The QHIC Case
Plot: Residual versus x and predicted responses

23. Checking the Normality Assumption
Example 11.18: The QHIC Case
Plots: Histogram and Normal Plot of Residuals

24. Checking the Independence Assumption
Plots: Residuals versus Fits (to check for functional form, not shown)
Residuals versus Time Order

25. Combination Residual Plots
Example 11.18: The QHIC Case Minitab Output
Plots: Histogram and Normal Plot of Residuals, Residuals versus Order (I Chart), Residuals versus Fit.

26. *11.9 Some Shortcut Formulas
where

27. Simple Linear Regression
Summary:
11.1 The Simple Linear Regression Model
11.2 The Least Squares Point Estimates
11.3 Model Assumptions, Mean Squared Error, Std. Error
11.4 Testing Significance of Slope and y-Intercept
11.5 Confidence Intervals and Prediction Intervals
11.6 The Coefficient of Determination and Correlation
11.7 An F Test for the Simple Linear Regression Model
*11.8 Checking Regression Assumptions by Residuals
*11.9 Some Shortcut Formulas