Prepared by Lee Revere and John Large Chapter 4 Regression Models Prepared by Lee Revere and John Large To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-1

Learning Objectives Students will be able to: Identify variables and use them in a regression model. Develop simple linear regression equations from sample data and interpret the slope and intercept. Compute the coefficient of determination and the coefficient of correlation and interpret their meanings. Interpret the F-test in a linear regression model. List the assumptions used in regression and use residual plots to identify problems. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-2

Learning Objectives (continued) Students will be able to: Develop a multiple regression model and use it to predict. Use dummy variables to model categorical data. Determine which variables should be included in a multiple regression model. Transform a nonlinear function into a linear one for use in regression. Understand and avoid common mistakes made in the use of regression analysis. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-3

Chapter Outline 4.1 Introduction 4.2 Scatter Diagrams 4.3 Simple Linear Regression 4.4 Measuring the Fit of a Regression Model 4.5 Using Computer Software for Regression 4.6 Assumptions of the Regression Model To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-4

Chapter Outline (continued) 4.7 Testing the Model for Significance 4.8 Multiple Regression Analysis 4.9 Binary or Dummy Variables 4.10 Model Building 4.11 Nonlinear Regression 4.12 Cautions and Pitfalls in Regression Analysis To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-5

Introduction Regression analysis is a very valuable tool for today’s manager. Regression is used to: understand the relationship between variables. predict the value of one variable based on another variable. Cost estimation models are a good example. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-6

Introduction (continued) A regression model is comprised of a dependent, or response, variable and an independent, or predictor, variable. Dependent Variable = Independent Variable(s) Prediction Relationship To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-7

Scatter Diagram A scatter diagram is used to graphically investigate the relationship between the dependent and independent variables. Plot the dependent variable on the Y axis. Plot the independent variable on the X axis. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-8

Triple A Construction Example Triple A Construction Company renovates old homes in Albany. They have found that its dollar volume of renovation work is dependent on the Albany area payroll. Triple A Sales ($100,000’s) Local Payroll ($100,000,000’s) 6 3 8 4 9 5 4.5 2 9.5 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-9

Triple A Construction Example (continued) Scatter Diagram Dependent Variable Independent Variable To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-10

Simple Linear Regression Regression models are used to test if a relationship exists between variables; that is, to use one variable to predict another. However, there is some random error that cannot be predicted. Y = 0 + 1X + error Where, Y = dependent variable (response) X = independent variable (predictor / explanatory) 0 = intercept (value of Y when X = 0) 1 = slope of the regression line Error = random error To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-11

Simple Linear Regression (continued) Sample data are used to estimate the true values for the intercept and slope. Y = b + b X Where, Y = predicted value of Y 0 1 The difference between the actual value of Y and the predicted value (using sample data) is known as the error. Error = (actual value) – (predicted value) e = Y - Y To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-12 9

Least Squares Regression Least squares regression minimizes the sum of the squared errors. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-13

Least Squares Regression Equations Least squares regression equations are: Y = b + b X 0 1 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-14 10

Calculating the Regression Line: Triple A Construction Sales (Y) Payroll (X) (X - X) (X-X)(Y-Y) 6 3 1 8 4 9 5 4.5 2 9.5 2.5 2 Summations for each column: 42 24 10 12.5 Y = 42/6 = 7 X = 24/6 = 4 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-15

Calculating the Regression Line (continued) Calculating the required parameters: b = (X-X)(Y-Y) 12.5 (X-X) 10 b = Y – b X = 7 – (1.25)(4) = 2 So, Y = 2 + 1.25 X ∑ 1 = = 1.25 2 ∑ o 1 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-16

Using Regression Line Y = 2 + 1.25 X If the payroll estimations for next year were $600 million, what is the predicted value of Triple A’s sales? Y = 2 + 1.25 X Sales = 2 + 1.25 (payroll) So, Next year sales = 2 + 1.25 (6) = 9.5 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-17

Measuring the Fit of the Regression Model To understand how well the model predicts the response variable, we evaluate the following: The variability in the Y variable SST – Total variability about the mean SSE – Variability about the regression line SSR – Variability that is explained Coefficient of Determination r2 - Proportion of explained variation Correlation Coefficient r – Strength of the relationship between Y and X variables To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-18

Measuring the Fit of the Regression Model Errors (deviations) may be positive or negative. Summing the errors would be misleading, thus we square the terms prior to summing. Sum of Squares Total (SST) measures the total variable in Y. Sum of the Squared Error (SSE) is less than the SST because the regression line reduced the variability. Sum of Squares due to Regression (SSR) indicated how much of the total variability is explained by the regression model. 2 SST = (Y-Y) ∑ SSE = e = (Y-Y) 2 2 ∑ SSR = (Y-Y) ∑ 2 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-19

Measuring the Fit of the Regression Model (continued) For Triple A Construction: SST = (Y-Y) 2 ∑ = 22.5 SSE = e = (Y-Y) 2 2 ∑ = 6.875 SSR = (Y-Y) ∑ 2 = 15.625 Note: SST = SSR + SSE Explained Variability Unexplained Variability To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-20

Coefficient of Determination The coefficient of determination (r2 ) is the proportion of the variability in Y that is explained by the regression equation. r2 = SSR = 1 – SSE SST SST For Triple A Construction: r2 = 15.625 = 0.6944 22.5 69% of the variability in sales is explained by the regression based on payroll. Note: 0 < r2 < 1 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-21

Correlation Coefficient The correlation coefficient (r) measures the strength of the linear relationship. For Triple A Construction, r = 0.8333 Note: -1 < r < 1 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-22

Correlation Coefficient (continued) To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-23

Computer Software for Regression In Excel, use Tools/ Data Analysis. This is an ‘add-in’ option. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-24

Computer Software for Regression (continued) After selecting the regression option, this will appear X and Y ranges Specify labels if included in range Output area Scatter diagram output Residual (error) output To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-25

Computer Software for Regression (continued) A scatter diagram will be given. Multiple r is correlation coefficient (r) High r (close to 1) 2 Regression coefficients To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-26

Assumptions of the Regression Model We make certain assumptions about the errors in a regression model which allow for statistical testing. Assumptions: Errors are independent. Errors are normally distributed. Errors have a mean of zero. Errors have a constant variance. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-27

Residual Analysis Residual analyses (plots) will highlight glaring violations of the assumptions. Healthy Residual Plot – no violations X To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-28

Residual Analysis: Nonlinear Violation Nonlinear Residual Plot –violation X To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-29 18

Residual Analysis: Nonconstant Error Nonconstant Error Residual Plot –violation X To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-30 19

Estimating the Variance The mean squared error (MSE) is the estimate of the error variance of the regression equation. s2 = MSE = SSE/(n-k-1) Where, n = number of observations in the sample k = number of independent variables For Triple A Construction, s = 1.7188 2 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-31

Estimating the Variance (continued) The standard deviation of the regression is used in many statistical tests about the regression model. s = MSE For Triple A Construction, s = 1.31 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-32

Testing the Model for Significance: F-test An F-test is used to statistically test the null hypothesis that there is no linear relationship between the X and Y variables (i.e. β = 0). If the significance level for the F test is low, we reject H0 and conclude there is a linear relationship. 1 F = MSR / MSE where, MSR = SSR k To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-33

Testing the Model for Significance: F-test For Triple A Construction: MSR = 15.625 = 15.625 1 F = 15.625 = 9.0909 1.7188 The significance level for F = 9.0909 is 0.0394, indicating we reject Ho and conclude a linear relationship exists between sales and payroll. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-34

Testing the Model for Significance: R2 r2 is the best measure of the strength of the prediction relationship between the X and Y variables. Values closer to 1 indicate a strong prediction relationship. Good regression models have significant F-test and high r2 values. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-35

Testing the Model for Significance: Coefficient Hypotheses Statistical tests of significance can be performed on the coefficients. The null hypothesis is that the coefficient of X (i.e., the slope of the line) is 0. P values are the observed significance level and can be used to test the null hypothesis. For a simple linear regression the test of the regression coefficients gives the same information as the F-test. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-36

ANOVA Tables When developing a regression model, an ANOVA table is computing by most statistical software. The general form of the ANOVA table is helpful for understanding the interrelatedness of error terms. DF SS MS F Significance Regression k SSR MSR MSR/MSE P-value Residual n-k-1 SSE MSE Total n-1 SST To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-37

Multiple Regression Y = b + b X + b X +…+ b X Multiple regression models are similar to simple linear regression models except they include more than one X variable. Y = b + b X + b X +…+ b X 0 1 1 2 2 n n slope Independent variables To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-38

Multiple Regression: Wilson Realty Example Wilson Realty wants to develop a model to determine the suggested listing price for a house based on size and age. Price Sq. Feet Age Condition 35000 1926 30 Good 47000 2069 40 Excellent 49900 1720 55000 1396 15 58900 1706 32 Mint 60000 1847 38 67000 1950 27 70000 2323 78500 2285 26 79000 3752 35 87500 2300 18 93000 2525 17 95000 3800 97000 1740 12 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-39

Wilson Realty Example (continued) 67% of the variation in sales price is explained by size and age. Ho: No linear relationship is rejected Y = 60815.45 + 21.91(size) – 1449.34 (age) Ho: β1 = 0 is rejected Ho: β2 = 0 is rejected To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-40

Wilson Realty Example (continued) Wilson Realty has found a linear relationship between price and size and age. The coefficient for size indicates each additional square foot increases the value by $21.91, while each additional year in age decreases the value by $1449.34. Ŷ = 60815.45 + 21.91(size) – 1449.34 (age) For a 1900 square foot house that is 10 years old, the following prediction can be made: $87,951 = 21.91(1900) + 1449.34(10) To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-41

Binary Variables Binary (or dummy) variables are special variables that are created for qualitative data. A dummy variable is assigned a value of 1 if a particular condition is met and a value of 0 otherwise. The number of dummy variables must equal one less than the number of categories of the qualitative variable. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-42

Wilson Realty Example: Binary Variables Return to Wilson Realty, and let’s evaluate how to use property condition in the regression model. There are three categories: Mint, Excellent, and Good. X = 1 if the house is in excellent condition = 0 otherwise X = 1 if the house is in mint condition = 0 otherwise Note: If both X and X = 0 then the house is in good condition 3 4 4 3 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-43

Wilson Realty: Binary Variables (continued) What can you say about the new model? Y = 48329.23 + 28.21 (size) – 1981.41(age) + 23684.62 (if mint) + 16581.32 (if excellent) To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-44

Model Building The best model is a statistically significant model with a high r2 and a few variables. As more variables are added to the model, the r2 usually increases . The adjusted r2 takes into account the number of independent variables in the model. Note: When variables are added to the model, the value of r2 can never decrease; however, the adjusted r2 may decrease. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-45

Model Building (continued) Collinearity or multicollinearity exists when an independent variable is correlated with one or more independent variable(s). Collinearity and multicollinearity create problems in the coefficients. The overall model prediction is still good; however interpretation of the individual variable coefficients is questionable. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-46

Nonlinear Regression Nonlinear relationships may exist between variables, thereby requiring a transformation of one or more variables to achieve linearity. Transformations may be used to turn a nonlinear model into a linear model. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-47

Automobile Example: Nonlinear Regression Engineers at Colonel Motors want to use regression analysis to improve fuel efficiency. They are studying the impact of weight on miles per gallon (MPG). MPG Weight 12 4.58 20 3.18 13 4.66 23 2.68 15 4.02 24 2.65 18 2.53 33 1.70 19 3.09 36 1.95 3.11 42 1.92 To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-48

Automobile Example (continued) Perhaps a nonlinear relationship exists? Linear regression line Nonlinear regression line To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-49

Automobile Example (continued) Linear regression model: MPG = 47.8 – 8.2 (weight) F significance = 0.0003 r2 = 0.7446 Nonlinear (transformed variable) regression model MPG = 79.8 – 30.2(weigth) + 3.4 (weight) F significance = 0.0002 R2 = 0.8478 2 Which model is best? What are the difficulties with interpreting the individual coefficients? To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-50

Cautions and Pitfalls If the assumptions are not met, the statistical test may not be valid. Correlation does not mean causation. Multicollinearity causes problems with coefficient interpretation. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-51

Cautions and Pitfalls (continued) Prediction beyond the range of X values in the sample can be misleading, including interpretation of the intercept (X = 0). A linear regression model may not be the best model, even in the presence of a significant F test. A statistically significant relationship does not mean practical value. To accompany Quantitative Analysis for Management, 9e by Render/Stair/Hanna 4-52