1. 1 Multiple Regression Chapter 17

2. 2 Introduction In this chapter we extend the simple linear regression model, and allow for any number of independent variables. We expect to build a model that fits the data better than the simple linear regression model.

3. 3 Weight Calories consumed Introduction We all believe that weight is affected by the amount of calories consumed. Yet, the actual effect is different from one individual to another. Therefore, a simple linear relationship leaves much unexplained error.

4. 4 Weight Calories consumed Introduction Click to to continue In an attempt to reduce the unexplained errors, we’ll add a second explanatory (independent) variable

5. 5 Weight Calories consumed Weight =  0 +  1 Calories +  2 Height +  Height Introduction If we believe a person’s height explains his/her weight too, we can add this variable to our model. The resulting Multiple regression model is shown:

6. 6 We shall use computer printout to –Assess the model How well it fits the data Is it useful Are any required conditions violated? –Employ the model Interpreting the coefficients Making predictions using the prediction equation Estimating the expected value of the dependent variable Introduction

7. 7 Dependent variableIndependent variables Random error variable 17.1 Model and Required Conditions Coefficients We allow k independent variables to potentially explain the dependent variable y =  0 +  1 x 1 +  2 x 2 + …+  k x k + 

8. 8 The error  is normally distributed. The mean is equal to zero and the standard deviation is constant (    for all values of y. The errors are independent. Model Assumptions – Required conditions for 

9. 9 –If the model assessment indicates good fit to the data, use it to interpret the coefficients and generate predictions. – Assess the model fit using statistics obtained from the sample. –Diagnose violations of required conditions. Try to remedy problems when identified. 17.2 Estimating the Coefficients and Assessing the Model The procedure used to perform regression analysis: – Obtain the model coefficients and statistics using a statistical software.

10. 10 Example 1 Where to locate a new motor inn? –La Quinta Motor Inns is planning an expansion. –Management wishes to predict which sites are likely to be profitable. –Several areas where predictors of profitability can be identified are: Competition Market awareness Demand generators Demographics Physical quality Estimating the Coefficients and Assessing the Model, Example

11. 11 Profitability Competition Market awareness CustomersCommunity Physical Operating Margin RoomsNearestOffice space Enrollment IncomeDistance Distance to downtown. Median household income. Distance to the nearest La Quinta inn. Number of hotels/motels rooms within 3 miles from the site. X 1 x 2 x 3 x 4 x 5 x 6 College Enrollment Estimating the Coefficients and Assessing the Model, Example

12. 12 Data were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model: Margin =     Rooms   Nearest   Office    College +  5 Income +  6 Disttwn +  Estimating the Coefficients and Assessing the Model, Example

13. 13 This is the sample regression equation (sometimes called the prediction equation) This is the sample regression equation (sometimes called the prediction equation) MARGIN = 38.14 - 0.0076 ROOMS + 1.65 NEAREST + 0.02 OFFICE +0.21 COLLEGE +0.41 INCOME - 0.23 DISTTWN Regression Analysis, Excel Output La Quinta

14. 14 Model Assessment - Standard Error of Estimate A small value of   indicates (by definition) a small variation of the errors around their mean. Since the mean is zero, small variation of the errors means the errors are close to zero. So we would prefer a model with a small standard deviation of the error rather than a large one. How can we determine whether the standard deviation of the error is small/large?

15. 15 The standard deviation of the error   is estimated by the Standard Error of Estimate s  : Model Assessment - Standard Error of Estimate The magnitude of s  is judged by comparing it to

16. 16 From the printout, s  = 5.5121 Calculating the mean value of y we have Standard Error of Estimate

17. 17 Model Assessment – Coefficient of Determination In our example it seems s  is not particularly small, or is it? If   is small the model fits the data well, and is considered useful. The usefulness of the model is evaluated by the amount of variability in the ‘y’ values explained by the model. This is done by the coefficient of determination. The coefficient of determination is calculated by As you can see, SSE (thus s  ) effects the value of r 2.

18. 18 Coefficient of Determination From the printout, R 2 = 0.5251 that is, 52.51% of the variability in the margin values is explained by this model.

19. 19 To answer the question we test the hypothesis H 0 :  1 =  2 = … =  k = 0 H 1 : At least one  i is not equal to zero. If at least one  i is not equal to zero, the model has some validity. We pose the question: Is there at least one independent variable linearly related to the dependent variable? Testing the Validity of the Model

20. 20 Note, that if all the data points satisfy the linear equation without errors, y i and coincide, and thus SSE = 0. In this case all the variation in y is explained by the regression (SS(Total) = SSR). The total variation in y (SS(Total)) can be explained in part by the regression (SSR) while the rest remains unexplained (SSE): SS(Total) = SSR + SSE or If errors exist in small amounts, SSR will be close to SS(Total) and the ratio SSR/SSE will be large. This leads to the F ratio test presented next. Testing the Validity of the Model

21. 21 Testing for Significance Define the Mean of the Sum of Squares-Regression (MSR) Define the Mean of the Sum of Squares-Error (MSE) The ratio MSR/MSE is F-distributed

22. 22 Rejection region F>F ,k,n-k-1 Testing for Significance Note. A Large F results from a large SSR, which indicates much of the variation in y is explained by the regression model; this is when the model is useful. Hence, the null hypothesis (which states that the model is not useful) should be rejected when F is sufficiently large. Therefore, the rejection region has the form of F > F ,k,n-k-1

23. 23 k = n–k–1 = n–1 = Testing the Model Validity of the La Quinta Inns Regression Model MSE=SSE/(n-k-1) MSR=SSR/k MSR/MSE SSE SSR The F ratio test is performed using the ANOVA portion of the regression output

24. 24 k = n–k–1 = n–1 = If alpha =.05, the critical F is F ,k,n-k-1 = F 0.05,6,100-6-1 =2.17 F = 17.14 > 2.17 Also, the p-value = 3.033(10) -13. Clearly, p-value=3.033(10) -13 < 0.05= , Conclusion: There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis. At least one of the  i is not equal to zero, thus, the independent variable associated with it has linear relationship to y. This linear regression model is useful Testing the Model Validity of the La Quinta Inns Regression Model

25. 25 b 0 = 38.14. This is the y intercept, the value of y when all the variables take the value zero. Since the data range of all the independent variables do not cover the value zero, do not interpret the intercept. Interpreting the Coefficients Interpreting the coefficients b 1 through b k y = b 0 + b 1 x 1 + b 2 x 2 +…+ b k x k y = b 0 + b 1 (x 1 +1) + b 2 x 2 +…+ b k x k = b 0 + b 1 x 1 + b 2 x 2 +…+ b k x k + b 1

26. 26 Interpreting the Coefficients b 1 = – 0.0076. In this model, for each additional room within 3 mile of the La Quinta inn, the operating margin decreases on the average by.0076% (assuming the other variables are held constant).

27. 27 b 2 = 1.65. In this model, for each additional mile that the nearest competitor is to a La Quinta inn, the average operating margin increases by 1.65% when the other variables are held constant. b 3 = 0.02. For each additional 1000 sq-ft of office space, the average increase in operating margin will be.02%. b 4 = 0.21. For each additional thousand students the average operating margin increases by.21% when the other variables remain constant. Interpreting the Coefficients

28. 28 b 5 = 0.41. For additional $1000 increase in median household income, the average operating margin increases by.41%, when the other variables remain constant. b 6 = - 0.23. For each additional mile to the downtown center, the average operating margin decreases by.23%. Interpreting the Coefficients

29. 29 Test statistic d.f. = n - k -1 The hypothesis for each  i is Excel printout H 0 :  i  0 H 1 :  i  0 Testing the Coefficients For example, a test for  1 : t = (-.007618-0)/.001255 = -6.068 Suppose alpha=.01. t.005,100-6-1 =3.39 There is sufficient evidence to reject H 0 at 1% significance level. Moreover the p=value of the test is 2.77(10 -8 ). Clearly H 0 is strongly rejected. The number of rooms is linearly related to the margin.

30. 30 The hypothesis for each  i is Excel printout H 0 :  i  0 H 1 :  i  0 Testing the Coefficients See next the interpretation of the p-value results

31. 31 Interpretation Interpretation of the regression results for this model –The number of hotel and motel rooms, distance to the nearest motel, the amount of office space, and the median household income are linearly related to the operating margin –Students enrollment and distance from downtown are not linearly related to the margin –Preferable locations have only few other motels nearby, much office space, and the surrounding households are affluent.

32. 32 The model can be used for making predictions by –Producing prediction interval estimate of the particular value of y, for given values of x i. –Producing a confidence interval estimate for the expected value of y, for given values of x i. The model can be used to learn about relationships between the independent variables x i, and the dependent variable y, by interpreting the coefficients  i Using the Regression Equation

33. 33 Predict the average operating margin of an inn at a site with the following characteristics: –3815 rooms within 3 miles, –Closet competitor 3.4 miles away, –476,000 sq-ft of office space, –24,500 college students, –$39,000 median household income, –3.6 miles distance to downtown center. MARGIN = 38.14 - 0.0076 (3815) - 1.646 (.9) + 0.02( 476) +0.212 (24.5) - 0.413( 35) + 0.225 (11.2) = 37.1% La Quinta La Quinta Inns, Predictions

34. 34 Interval estimates by Excel (Data analysis plus) It is predicted that the average operating margin will lie within 25.4% and 48.8%, with 95% confidence. It is expected the average operating margin of all sites that fit this category falls within 33% and 41.2% with 95% confidence. The average inn would not be profitable (Less than 50%). La Quinta Inns, Predictions

35. 35 18.2 Qualitative Independent Variables In many real-life situations one or more independent variables are qualitative. Including qualitative variables in a regression analysis model is done via indicator variables. An indicator variable (I) can assume one out of two values, “zero” or “one”. 1 if a first condition out of two is met 0 if a second condition out of two is met I= 1 if data were collected before 1980 0 if data were collected after 1980 1 if the temperature was below 50 o 0 if the temperature was 50 o or more 1 if a degree earned is in Finance 0 if a degree earned is not in Finance

36. 36 Qualitative Independent Variables; Example: Auction Car Price (II) Example 2 - continued –Recall: A car dealer wants to predict the auction price of a car. –The dealer believes now that both odometer reading and car color are variables that affect a car’s price. –Three color categories are considered: White Silver Other colors Note: “Color” is a qualitative variable.

37. 37 Example 2 - continued I 1 = 1 if the color is white 0 if the color is not white I 2 = 1 if the color is silver 0 if the color is not silver The category “Other colors” is defined by: I 1 = 0; I 2 = 0 Qualitative Independent Variables; Example: Auction Car Price (II)

38. 38 Note: To represent the situation of three possible colors we need only two indicator variables. Generally to represent a nominal variable with m possible values, we must create m-1 indicator variables. How Many Indicator Variables?

39. 39 Solution –the proposed model is y =  0 +  1 (Odometer) +  2 I 1 +  3 I 2 +  –The data White color Other color Silver color Qualitative Independent Variables; Example: Auction Car Price (II) Enter the data in Excel as usual

40. 40 Odometer Price Price = 16.837 -.0591(Odometer) +.0911(0) +.3304(1) Price = 16.837 -.0591(Odometer) +.0911(1) +.3304(0) Price = 16.837 -.0591(Odometer) +.0911(0) +.3304(0) 16.837 -.0591(Odometer) 16.928 -.0591(Odometer) 17.167 -.0591(Odometer) The equation for an “other color” car. The equation for a white color car. The equation for a silver color car. From Excel we get the regression equation PRICE = 16.837 -.0591(Odometer) +.0911(I-1) +.3304(I-2) Example: Auction Car Price (II) The Regression Equation

41. 41 From Excel we get the regression equation PRICE = 16701-.0591(Odometer)+.0911(I-1)+.3304(I-2) A white car sells, on the average, for $91.1 more than a car of the “Other color” category A silver color car sells, on the average, for $330.4 more than a car of the “Other color” category. For one additional mile the auction price decreases by 5.91 cents on the average. Example: Auction Car Price (II) The Regression Equation Interpreting the equation

42. 42 There is insufficient evidence to infer that a white color car and a car of “other color” sell for a different auction price. There is sufficient evidence to infer that a silver color car sells for a larger price than a car of the “other color” category. Car Price-Dummy Example: Auction Car Price (II) The Regression Equation

43. 43 Recall: The Dean wanted to evaluate applications for the MBA program by predicting future performance of the applicants. The following three predictors were suggested: –Undergraduate GPA –GMAT score –Years of work experience It is now believed that the type of undergraduate degree should be included in the model. Qualitative Independent Variables; Example: MBA Program Admission (II) Note: The undergraduate degree is qualitative.

44. 44 Qualitative Independent Variables; Example: MBA Program Admission (II) I 1 = 1 if B.A. 0 otherwise I 2 = 1 if B.B.A 0 otherwise The category “Other group” is defined by: I 1 = 0; I 2 = 0; I 3 = 0 I 3 = 1 if B.Sc. or B.Eng. 0 otherwise

45. 45 Qualitative Independent Variables; Example: MBA Program Admission (II) MBA-II

46. 46 Applications in Human Resources Management: Pay-Equity Pay-equity can be handled in two different forms: –Equal pay for equal work –Equal pay for work of equal value. Regression analysis is extensively employed in cases of equal pay for equal work.

47. 47 Human Resources Management: Pay-Equity Example 3 –Is there sex discrimination against female managers in a large firm? –A random sample of 100 managers was selected and data were collected as follows: Annual salary Years of education Years of experience Gender

48. 48 Solution –Construct the following multiple regression model: y =  0 +  1 Education +  2 Experience +  3 Gender +  –Note the nature of the variables: Education – quantitative Experience – quantitative Gender – qualitative (Gender = 1 if male; =0 otherwise). Human Resources Management: Pay-Equity

49. 49 Solution – Continued (HumanResource)HumanResource Human Resources Management: Pay-Equity Analysis and Interpretation The model fits the data quite well. The model is very useful. Experience is a variable strongly related to salary. There is no evidence of sex discrimination.

50. 50 Solution – Continued (HumanResource)HumanResource Human Resources Management: Pay-Equity Analysis and Interpretation Further studying the data we find: Average experience (years) for women is 12. Average experience (years) for men is 17 Average salary for female manager is $76,189 Average salary for male manager is $97,832

51. 51 Review problems