1. Prepared by Lee Revere and John Large
Chapter 4
Regression
Models
Prepared by Lee Revere and John Large
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-1

2. Learning Objectives Students will be able to:
Identify variables and use them in a regression model.
Develop simple linear regression equations from sample data and interpret the slope and intercept.
Compute the coefficient of determination and the coefficient of correlation and interpret their meanings.
Interpret the F-test in a linear regression model.
List the assumptions used in regression and use residual plots to identify problems.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-2

3. Learning Objectives (continued)
Students will be able to:
Develop a multiple regression model and use it to predict.
Use dummy variables to model categorical data.
Determine which variables should be included in a multiple regression model.
Transform a nonlinear function into a linear one for use in regression.
Understand and avoid common mistakes made in the use of regression analysis.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-3

4. Chapter Outline 4.1 Introduction 4.2 Scatter Diagrams
4.3 Simple Linear Regression
4.4 Measuring the Fit of a Regression Model
4.5 Using Computer Software for Regression
4.6 Assumptions of the Regression Model
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-4

5. Chapter Outline (continued)
4.7 Testing the Model for Significance
4.8 Multiple Regression Analysis
4.9 Binary or Dummy Variables
Model Building
Nonlinear Regression
Cautions and Pitfalls in Regression Analysis
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-5

6. Introduction Regression analysis is a very
valuable tool for today’s manager.
Regression is used to:
understand the relationship between variables.
predict the value of one variable based on another variable.
Cost estimation models are a good example.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-6

7. Introduction (continued)
A regression model is comprised of a dependent, or response, variable and an independent, or predictor, variable.
Dependent Variable = Independent Variable(s)
Prediction Relationship
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-7

8. Scatter Diagram A scatter diagram is used to
graphically investigate the
relationship between the dependent
and independent variables.
Plot the dependent variable on the Y axis.
Plot the independent variable on the X axis.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-8

9. Triple A Construction Example
Triple A Construction Company renovates
old homes in Albany. They have found that
its dollar volume of renovation work is
dependent on the Albany area payroll.
Triple A Sales
($100,000’s)
Local Payroll
($100,000,000’s) 6 3 8 4 9 5
4.5 2
9.5
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-9

10. Triple A Construction Example (continued)
Scatter Diagram
Dependent Variable
Independent Variable
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-10

11. Simple Linear Regression
Regression models are used to test if a relationship exists between variables; that is, to use one variable to predict another. However, there is some random error that cannot be predicted.
Y = 0 + 1X + error
Where, Y = dependent variable (response)
X = independent variable (predictor / explanatory)
0 = intercept (value of Y when X = 0)
1 = slope of the regression line
Error = random error
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-11

12. Simple Linear Regression (continued)
Sample data are used to estimate the true values for the intercept and slope.
Y = b + b X
Where,
Y = predicted value of Y
The difference between the actual value of Y and the predicted value (using sample data) is known as the error.
Error = (actual value) – (predicted value)
e = Y - Y
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-12 9

13. Least Squares Regression
Least squares regression minimizes the sum of the squared errors.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-13

14. Least Squares Regression Equations
Least squares regression equations are:
Y = b + b X
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-14 10

15. Calculating the Regression Line: Triple A Construction
Sales (Y)
Payroll (X)
(X - X)
(X-X)(Y-Y) 6 3 1 8 4 9 5
4.5 2
9.5
2.5 2
Summations for each column:
Y = 42/6 = X = 24/6 = 4
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-15

16. Calculating the Regression Line (continued)
Calculating the required parameters:
b = (X-X)(Y-Y)
(X-X)
b = Y – b X = 7 – (1.25)(4) = 2
So,
Y = X ∑ 1
= = 1.25 2 ∑ o
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-16

17. Using Regression Line Y = 2 + 1.25 X
If the payroll estimations for next
year were $600 million, what is
the predicted value of Triple A’s
sales?
Y = X
Sales = (payroll)
So,
Next year sales = (6) = 9.5
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-17

18. Measuring the Fit of the Regression Model
To understand how well the model predicts the response variable, we evaluate the following:
The variability in the Y variable SST – Total variability about the mean
SSE – Variability about the regression line SSR – Variability that is explained
Coefficient of Determination
r2 - Proportion of explained variation
Correlation Coefficient
r – Strength of the relationship between Y and X variables
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-18

19. Measuring the Fit of the Regression Model
Errors (deviations) may be positive or negative. Summing the errors would be misleading, thus we square the terms prior to summing.
Sum of Squares Total (SST) measures the total variable in Y.
Sum of the Squared Error (SSE) is less than the SST because the regression line reduced the variability.
Sum of Squares due to Regression (SSR) indicated how much of the total variability is explained by the regression model. 2
SST = (Y-Y) ∑
SSE = e = (Y-Y) ∑
SSR = (Y-Y) ∑ 2
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-19

20. Measuring the Fit of the Regression Model (continued)
For Triple A Construction:
SST = (Y-Y) 2 ∑
= 22.5
SSE = e = (Y-Y) ∑
= 6.875
SSR = (Y-Y) ∑ 2 =
Note:
SST = SSR + SSE
Explained
Variability
Unexplained
Variability
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-20

21. Coefficient of Determination
The coefficient of determination (r2 ) is the proportion of the variability in Y that is explained by the regression equation.
r2 = SSR = 1 – SSE
SST SST
For Triple A Construction:
r2 = =
22.5
69% of the variability in sales is explained
by the regression based on payroll.
Note: 0 < r2 < 1
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-21

22. Correlation Coefficient
The correlation coefficient (r) measures the strength of the linear relationship.
For Triple A Construction, r =
Note: -1 < r < 1
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-22

23. Correlation Coefficient (continued)
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-23

24. Computer Software for Regression
In Excel, use Tools/
Data Analysis. This
is an ‘add-in’ option.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-24

25. Computer Software for Regression (continued)
After selecting the regression option, this will appear
X and Y ranges
Specify labels if included in range
Output area
Scatter diagram
output
Residual (error) output
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-25

26. Computer Software for Regression (continued)
A scatter diagram will be given.
Multiple r is correlation
coefficient (r)
High r (close to 1) 2
Regression coefficients
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-26

27. Assumptions of the Regression Model
We make certain assumptions about the errors in a regression model which allow for statistical testing.
Assumptions:
Errors are independent.
Errors are normally distributed.
Errors have a mean of zero.
Errors have a constant variance.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-27

28. Residual Analysis
Residual analyses (plots) will highlight glaring violations of the assumptions.
Healthy Residual Plot – no violations X
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-28

29. Residual Analysis: Nonlinear Violation
Nonlinear Residual Plot –violation X
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-29 18

30. Residual Analysis: Nonconstant Error
Nonconstant Error Residual Plot –violation X
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-30 19

31. Estimating the Variance
The mean squared error (MSE) is the estimate of the error variance of the regression equation.
s2 = MSE = SSE/(n-k-1)
Where,
n = number of observations in the sample
k = number of independent variables
For Triple A Construction, s = 2
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-31

32. Estimating the Variance (continued)
The standard deviation of the regression is used in many statistical tests about the regression model.
s = MSE
For Triple A Construction, s = 1.31
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-32

33. Testing the Model for Significance: F-test
An F-test is used to statistically
test the null hypothesis that there
is no linear relationship between
the X and Y variables (i.e. β = 0).
If the significance level for the F
test is low, we reject H0 and conclude
there is a linear relationship. 1
F = MSR / MSE
where, MSR = SSR k
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-33

34. Testing the Model for Significance: F-test
For Triple A Construction:
MSR = = 1
F = =
1.7188
The significance level for F = is , indicating we reject Ho and conclude a linear relationship exists between sales and payroll.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-34

35. Testing the Model for Significance: R2
r2 is the best measure of the
strength of the prediction
relationship between the X and Y
variables.
Values closer to 1 indicate a strong prediction relationship.
Good regression models have significant F-test and high r2 values.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-35

36. Testing the Model for Significance: Coefficient Hypotheses
Statistical tests of significance
can be performed on the coefficients.
The null hypothesis is that the
coefficient of X (i.e., the slope of the
line) is 0.
P values are the observed significance level and can be used to test the null hypothesis.
For a simple linear regression the test of the regression coefficients gives the same information as the F-test.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-36

37. ANOVA Tables
When developing a regression model, an ANOVA table is computing by most statistical software. The general form of the ANOVA table is helpful for understanding the interrelatedness of error terms. DF SS MS F
Significance
Regression k
SSR
MSR
MSR/MSE
P-value
Residual
n-k-1
SSE
MSE
Total
n-1
SST
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-37

38. Multiple Regression Y = b + b X + b X +…+ b X
Multiple regression models are similar to simple linear regression models except they include more than one X variable.
Y = b + b X + b X +…+ b X
n n
slope
Independent variables
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-38

39. Multiple Regression: Wilson Realty Example
Wilson Realty wants to develop a model to determine the suggested listing price for a house based on size and age.
Price
Sq. Feet
Age
Condition
35000
1926 30
Good
47000
2069 40
Excellent
49900
1720
55000
1396 15
58900
1706 32
Mint
60000
1847 38
67000
1950 27
70000
2323
78500
2285 26
79000
3752 35
87500
2300 18
93000
2525 17
95000
3800
97000
1740 12
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-39

40. Wilson Realty Example (continued)
67% of the variation in sales price is explained by size and age.
Ho: No linear
relationship
is rejected
Y = (size) – (age)
Ho: β1 = 0 is rejected
Ho: β2 = 0 is rejected
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-40

41. Wilson Realty Example (continued)
Wilson Realty has found a linear relationship between price and size and age. The coefficient for size indicates each additional square foot increases the value by $21.91, while each additional year in age decreases the value by $
Ŷ = (size) – (age)
For a 1900 square foot house that is 10 years old, the following prediction can be made:
$87,951 = 21.91(1900) (10)
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-41

42. Binary Variables
Binary (or dummy) variables are special variables that are created for qualitative data.
A dummy variable is assigned a value of 1 if a particular condition is met and a value of 0 otherwise.
The number of dummy variables must equal one less than the number of categories of the qualitative variable.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-42

43. Wilson Realty Example: Binary Variables
Return to Wilson Realty, and let’s evaluate how to use property condition in the regression model. There are three categories: Mint, Excellent, and Good.
X = 1 if the house is in excellent condition
= 0 otherwise
X = 1 if the house is in mint condition
= 0 otherwise
Note: If both X and X = 0 then the house is in good condition 3 4 4 3
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-43

44. Wilson Realty: Binary Variables (continued)
What can you say about the new model?
Y = (size) – (age) +
(if mint) (if excellent)
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-44

45. Model Building
The best model is a statistically significant model with a high r2
and a few variables.
As more variables are added to the model, the r2 usually increases .
The adjusted r2 takes into account the number of independent variables in the model.
Note: When variables are added to the model, the value of r2 can never decrease; however, the adjusted r2 may decrease.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-45

46. Model Building (continued)
Collinearity or multicollinearity exists when an independent variable is correlated with one or more independent variable(s).
Collinearity and multicollinearity create problems in the coefficients.
The overall model prediction is still good; however interpretation of the individual variable coefficients is questionable.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-46

47. Nonlinear Regression
Nonlinear relationships may exist between variables, thereby requiring a transformation of one or more variables to achieve linearity.
Transformations may be used to turn a nonlinear model into a linear model.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-47

48. Automobile Example: Nonlinear Regression
Engineers at Colonel Motors want to use regression analysis to improve fuel efficiency. They are studying the impact of weight on miles per gallon (MPG).
MPG
Weight 12
4.58 20
3.18 13
4.66 23
2.68 15
4.02 24
2.65 18
2.53 33
1.70 19
3.09 36
1.95
3.11 42
1.92
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-48

49. Automobile Example (continued)
Perhaps a nonlinear relationship exists?
Linear regression line
Nonlinear regression line
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-49

50. Automobile Example (continued)
Linear regression model:
MPG = 47.8 – 8.2 (weight)
F significance =
r2 =
Nonlinear (transformed variable) regression model
MPG = 79.8 – 30.2(weigth) (weight) F significance =
R2 = 2
Which model is best? What are the difficulties with interpreting the individual coefficients?
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-50

51. Cautions and Pitfalls
If the assumptions are not met, the statistical test may not be valid.
Correlation does not mean causation.
Multicollinearity causes problems with coefficient interpretation.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-51

52. Cautions and Pitfalls (continued)
Prediction beyond the range of X values in the sample can be misleading, including interpretation of the intercept (X = 0).
A linear regression model may not be the best model, even in the presence of a significant F test.
A statistically significant relationship does not mean practical value.
To accompany Quantitative Analysis for Management, 9e
by Render/Stair/Hanna
4-52