1 Multiple Regression Chapter 18. 2 18.1 Introduction In this chapter we extend the simple linear regression model, and allow for any number of independent.

Slides:



Advertisements
Similar presentations
Chapter 18 Multiple Regression.
Advertisements

Multiple Regression. Introduction In this chapter, we extend the simple linear regression model. Any number of independent variables is now allowed. We.
Lecture Unit Multiple Regression.
Chap 12-1 Statistics for Business and Economics, 6e © 2007 Pearson Education, Inc. Chapter 12 Simple Regression Statistics for Business and Economics 6.
Forecasting Using the Simple Linear Regression Model and Correlation
Regression Analysis Once a linear relationship is defined, the independent variable can be used to forecast the dependent variable. Y ^ = bo + bX bo is.
1 Multiple Regression Chapter Introduction In this chapter we extend the simple linear regression model, and allow for any number of independent.
1 Multiple Regression Model Error Term Assumptions –Example 1: Locating a motor inn Goodness of Fit (R-square) Validity of estimates (t-stats & F-stats)
Multiple Regression Analysis
Lecture 9- Chapter 19 Multiple regression Introduction In this chapter we extend the simple linear regression model and allow for any number of.
Chapter 13 Multiple Regression
Chapter 12 Simple Regression
Basic Business Statistics, 11e © 2009 Prentice-Hall, Inc. Chap 13-1 Chapter 13 Simple Linear Regression Basic Business Statistics 11 th Edition.
Simple Linear Regression
Lecture 25 Multiple Regression Diagnostics (Sections )
Chapter 12 Multiple Regression
Lecture 22 Multiple Regression (Sections )
Chapter 13 Introduction to Linear Regression and Correlation Analysis
1 Multiple Regression. 2 Introduction In this chapter we extend the simple linear regression model, and allow for any number of independent variables.
Statistics for Managers Using Microsoft Excel, 5e © 2008 Prentice-Hall, Inc.Chap 13-1 Statistics for Managers Using Microsoft® Excel 5th Edition Chapter.
SIMPLE LINEAR REGRESSION
Lecture 24 Multiple Regression (Sections )
Pengujian Parameter Koefisien Korelasi Pertemuan 04 Matakuliah: I0174 – Analisis Regresi Tahun: Ganjil 2007/2008.
Chapter Topics Types of Regression Models
Linear Regression and Correlation Analysis
1 Simple Linear Regression Chapter Introduction In this chapter we examine the relationship among interval variables via a mathematical equation.
1 Simple Linear Regression and Correlation Chapter 17.
Lecture 23 Multiple Regression (Sections )
1 4. Multiple Regression I ECON 251 Research Methods.
Linear Regression Example Data
© 2000 Prentice-Hall, Inc. Chap Forecasting Using the Simple Linear Regression Model and Correlation.
Chapter 14 Introduction to Linear Regression and Correlation Analysis
1 Simple Linear Regression Chapter Introduction In Chapters 17 to 19 we examine the relationship between interval variables via a mathematical.
Basic Business Statistics, 10e © 2006 Prentice-Hall, Inc. Chap 13-1 Chapter 13 Simple Linear Regression Basic Business Statistics 10 th Edition.
Chapter 7 Forecasting with Simple Regression
Introduction to Regression Analysis, Chapter 13,
Simple Linear Regression. Introduction In Chapters 17 to 19, we examine the relationship between interval variables via a mathematical equation. The motivation.
Chapter 13 Simple Linear Regression
1 Simple Linear Regression 1. review of least squares procedure 2. inference for least squares lines.
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 13-1 Chapter 13 Introduction to Multiple Regression Statistics for Managers.
Chapter 13 Simple Linear Regression
SIMPLE LINEAR REGRESSION
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 12-1 Chapter 12 Simple Linear Regression Statistics for Managers Using.
1 Least squares procedure Inference for least squares lines Simple Linear Regression.
OPIM 303-Lecture #8 Jose M. Cruz Assistant Professor.
Statistics for Business and Economics 7 th Edition Chapter 11 Simple Regression Copyright © 2010 Pearson Education, Inc. Publishing as Prentice Hall Ch.
© 2003 Prentice-Hall, Inc.Chap 13-1 Basic Business Statistics (9 th Edition) Chapter 13 Simple Linear Regression.
Chap 12-1 A Course In Business Statistics, 4th © 2006 Prentice-Hall, Inc. A Course In Business Statistics 4 th Edition Chapter 12 Introduction to Linear.
EQT 373 Chapter 3 Simple Linear Regression. EQT 373 Learning Objectives In this chapter, you learn: How to use regression analysis to predict the value.
Copyright © 2005 Brooks/Cole, a division of Thomson Learning, Inc Chapter 18 Multiple Regression.
Economics 173 Business Statistics Lecture 20 Fall, 2001© Professor J. Petry
Economics 173 Business Statistics Lecture 19 Fall, 2001© Professor J. Petry
Lecture 10: Correlation and Regression Model.
Statistics for Managers Using Microsoft® Excel 5th Edition
Chapter 12 Simple Linear Regression.
Copyright © 2009 Cengage Learning 17.1 Chapter 19 Multiple Regression.
Statistics for Managers Using Microsoft Excel, 4e © 2004 Prentice-Hall, Inc. Chap 12-1 Chapter 12 Simple Linear Regression Statistics for Managers Using.
Conceptual Foundations © 2008 Pearson Education Australia Lecture slides for this course are based on teaching materials provided/referred by: (1) Statistics.
Economics 173 Business Statistics Lecture 18 Fall, 2001 Professor J. Petry
Multiple Regression Reference: Chapter 18 of Statistics for Management and Economics, 7 th Edition, Gerald Keller. 1.
1 Assessment and Interpretation: MBA Program Admission Policy The dean of a large university wants to raise the admission standards to the popular MBA.
Forecasting. Model with indicator variables The choice of a forecasting technique depends on the components identified in the time series. The techniques.
Lecture 9 Forecasting. Introduction to Forecasting * * * * * * * * o o o o o o o o Model 1Model 2 Which model performs better? There are many forecasting.
1 Simple Linear Regression Chapter Introduction In Chapters 17 to 19 we examine the relationship between interval variables via a mathematical.
Chapter 13 Simple Linear Regression
Inference for Least Squares Lines
Statistics for Managers using Microsoft Excel 3rd Edition
Chapter 13 Simple Linear Regression
Presentation transcript:

1 Multiple Regression Chapter 18

Introduction In this chapter we extend the simple linear regression model, and allow for any number of independent variables. We expect to build a model that fits the data better than the simple linear regression model.

3 We shall use computer printout to –Assess the model How well it fits the data Is it useful Are any required conditions violated? –Employ the model Interpreting the coefficients Predictions using the prediction equation Estimating the expected value of the dependent variable Introduction

4 Coefficients Dependent variableIndependent variables Random error variable 18.2 Model and Required Conditions We allow for k independent variables to potentially be related to the dependent variable y =  0 +  1 x 1 +  2 x 2 + …+  k x k + 

5 Multiple Regression for k = 2, Graphical Demonstration - I y =  0 +  1 x X y X2X2 1 The simple linear regression model allows for one independent variable, “x” y =  0 +  1 x +  The multiple linear regression model allows for more than one independent variable. Y =  0 +  1 x 1 +  2 x 2 +  Note how the straight line becomes a plain, and... y =  0 +  1 x 1 +  2 x 2

6 Multiple Regression for k = 2, Graphical Demonstration - II Note how a parabola becomes a parabolic Surface. X y X2X2 1 y= b 0 + b 1 x 2 y = b 0 + b 1 x b 2 x 2 b0b0

7 The error  is normally distributed. The mean is equal to zero and the standard deviation is constant (    for all values of y. The errors are independent. Required conditions for the error variable

8 –If the model assessment indicates good fit to the data, use it to interpret the coefficients and generate predictions. –Assess the model fit using statistics obtained from the sample. –Diagnose violations of required conditions. Try to remedy problems when identified Estimating the Coefficients and Assessing the Model The procedure used to perform regression analysis: –Obtain the model coefficients and statistics using a statistical software.

9 Example 18.1 Where to locate a new motor inn? –La Quinta Motor Inns is planning an expansion. –Management wishes to predict which sites are likely to be profitable. –Several areas where predictors of profitability can be identified are: Competition Market awareness Demand generators Demographics Physical quality Estimating the Coefficients and Assessing the Model, Example

10 Profitability Competition Market awareness CustomersCommunity Physical Margin RoomsNearestOffice space College enrollment IncomeDisttwn Distance to downtown. Median household income. Distance to the nearest La Quinta inn. Number of hotels/motels rooms within 3 miles from the site.

11 Estimating the Coefficients and Assessing the Model, Example Profitability Competition Market awareness CustomersCommunity Physical Operating Margin RoomsNearestOffice space College enrollment IncomeDisttwn Distance to downtown. Median household income. Distance to the nearest La Quinta inn. Number of hotels/motels rooms within 3 miles from the site.

12 Data were collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model: Margin =     Rooms   Nearest   Office    College +  5 Income +  6 Disttwn Estimating the Coefficients and Assessing the Model, Example Xm18

13 This is the sample regression equation (sometimes called the prediction equation) This is the sample regression equation (sometimes called the prediction equation) Regression Analysis, Excel Output Margin = Number +1.65Nearest Office Space +0.21Enrollment Income Distance

14 Model Assessment The model is assessed using three tools: –The standard error of estimate –The coefficient of determination –The F-test of the analysis of variance The standard error of estimates participates in building the other tools.

15 The standard deviation of the error is estimated by the Standard Error of Estimate : The magnitude of s  is judged by comparing it to Standard Error of Estimate

16 From the printout, s  = 5.51 Calculating the mean value of y we have It seems s  is not particularly small. Question: Can we conclude the model does not fit the data well? Standard Error of Estimate

17 The definition is From the printout, R 2 = % of the variation in operating margin is explained by the six independent variables % remains unexplained. When adjusted for degrees of freedom, Adjusted R 2 = 1-[SSE/(n-k-1)] / [SS(Total)/(n-1)] = = 49.44% Coefficient of Determination

18 We pose the question: Is there at least one independent variable linearly related to the dependent variable? To answer the question we test the hypothesis H 0 :  0 =  1 =  2 = … =  k H 1 : At least one  i is not equal to zero. If at least one  i is not equal to zero, the model has some validity. Testing the Validity of the Model

19 The hypotheses are tested by an ANOVA procedure ( the Excel output) Testing the Validity of the La Quinta Inns Regression Model MSE=SSE/(n-k-1) MSR=SSR/k MSR/MSE SSE SSR k = n–k–1 = n-1 =

20 [Variation in y] = SSR + SSE. Large F results from a large SSR. Then, much of the variation in y is explained by the regression model; the model is useful, and thus, the null hypothesis should be rejected. Therefore, the rejection region is… Rejection region F>F ,k,n-k-1 Testing the Validity of the La Quinta Inns Regression Model

21 F ,k,n-k-1 = F 0.05,6, =2.17 F = > 2.17 Also, the p-value (Significance F) = Reject the null hypothesis. Testing the Validity of the La Quinta Inns Regression Model Conclusion: There is sufficient evidence to reject the null hypothesis in favor of the alternative hypothesis. At least one of the  i is not equal to zero. Thus, at least one independent variable is linearly related to y. This linear regression model is valid

22 b 0 = This is the intercept, the value of y when all the variables take the value zero. Since the data range of all the independent variables do not cover the value zero, do not interpret the intercept. b 1 = – In this model, for each additional room within 3 mile of the La Quinta inn, the operating margin decreases on average by.0076% (assuming the other variables are held constant). Interpreting the Coefficients

23 b 2 = In this model, for each additional mile that the nearest competitor is to a La Quinta inn, the operating margin increases on average by 1.65% when the other variables are held constant. b 3 = For each additional 1000 sq-ft of office space, the operating margin will increase on average by.02% when the other variables are held constant. b 4 = For each additional thousand students the operating margin increases on average by.21% when the other variables are held constant. Interpreting the Coefficients

24 b 5 = For additional $1000 increase in median household income, the operating margin increases on average by.41%, when the other variables remain constant. b 6 = For each additional mile to the downtown center, the operating margin decreases on average by.23% when the other variables are held constant. Interpreting the Coefficients

25 The hypothesis for each  i is Excel printout H 0 :  i  0 H 1 :  i  0 d.f. = n - k -1 Test statistic Testing the Coefficients

26 The model can be used for making predictions by –Producing prediction interval estimate for the particular value of y, for a given values of x i. –Producing a confidence interval estimate for the expected value of y, for given values of x i. The model can be used to learn about relationships between the independent variables x i, and the dependent variable y, by interpreting the coefficients  i Using the Linear Regression Equation

27 Predict the average operating margin of an inn at a site with the following characteristics: –3815 rooms within 3 miles, –Closet competitor.9 miles away, –476,000 sq-ft of office space, –24,500 college students, –$35,000 median household income, –11.2 miles distance to downtown center. MARGIN = (3815) (.9) ( 476) (24.5) ( 35) (11.2) = 37.1% Xm18 La Quinta Inns, Predictions

28 Interval estimates by Excel (Data Analysis Plus) It is predicted, with 95% confidence that the operating margin will lie between 25.4% and 48.8%. It is estimated the average operating margin of all sites that fit this category falls within 33% and 41.2%. The average inn would not be profitable (Less than 50%). La Quinta Inns, Predictions

29 Assessment and Interpretation: MBA Program Admission Policy The dean of a large university wants to raise the admission standards to the popular MBA program. She plans to develop a method that can predict an applicant’s performance in the program. She believes a student’s success can be predicted by: –Undergraduate GPA –Graduate Management Admission Test (GMAT) score –Number of years of work experience

30 MBA Program Admission Policy A randomly selected sample of students who completed the MBA was selected. (See MBA).MBA Develop a plan to decide which applicant to admit.

31 MBA Program Admission Policy Solution –The model to estimate is: y =  0 +  1 x 1 +  2 x 2 +  3 x 3 +  y = MBA GPA x 1 = undergraduate GPA [UnderGPA] x 2 = GMAT score [GMAT] x 3 = years of work experience [Work] –The estimated model: MBA GPA = b 0 + b 1 UnderGPA + b 2 GMAT + b 3 Work

32 MBA Program Admission Policy – Model Diagnostics We estimate the regression model then we check: Normality of errors

33 MBA Program Admission Policy – Model Diagnostics We estimate the regression model then we check: The variance of the error variable

34 MBA Program Admission Policy – Model Diagnostics

35 MBA Program Admission Policy – Model Assessment The model is valid (p-value = …) 46.35% of the variation in MBA GPA is explained by the model. GMAT score and years of work experience are linearly related to MBA GPA. Insufficient evidence of linear relationship between undergraduate GPA and MBA GPA.

36 The conditions required for the model assessment to apply must be checked. –Is the error variable normally distributed? –Is the error variance constant? –Are the errors independent? –Can we identify outlier? –Is multicolinearity (intercorrelation)a problem? 18.4 Regression Diagnostics - II Draw a histogram of the residuals Plot the residuals versus y ^ Plot the residuals versus the time periods

37 Diagnostics: Multicolinearity Example 18.2: Predicting house price ( Xm18-02) Xm18-02 –A real estate agent believes that a house selling price can be predicted using the house size, number of bedrooms, and lot size. –A random sample of 100 houses was drawn and data recorded. –Analyze the relationship among the four variables

38 The proposed model is PRICE =  0 +  1 BEDROOMS +  2 H-SIZE +  3 LOTSIZE +  The model is valid, but no variable is significantly related to the selling price ?! Diagnostics: Multicolinearity

39 Multicolinearity is found to be a problem. Diagnostics: Multicolinearity Multicolinearity causes two kinds of difficulties: –The t statistics appear to be too small. –The  coefficients cannot be interpreted as “slopes”.

40 Remedying Violations of the Required Conditions Nonnormality or heteroscedasticity can be remedied using transformations on the y variable. The transformations can improve the linear relationship between the dependent variable and the independent variables. Many computer software systems allow us to make the transformations easily.

41 A brief list of transformations »y’ = log y (for y > 0) Use when the s  increases with y, or Use when the error distribution is positively skewed »y’ = y 2 Use when the s 2  is proportional to E(y), or Use when the error distribution is negatively skewed »y’ = y 1/2 (for y > 0) Use when the s 2  is proportional to E(y) »y’ = 1/y Use when s 2  increases significantly when y increases beyond some critical value. Reducing Nonnormality by Transformations Transformations, Example.

42 Durbin - Watson Test: Are the Errors Autocorrelated? This test detects first order autocorrelation between consecutive residuals in a time series If autocorrelation exists the error variables are not independent Residual at time i

43 Positive First Order Autocorrelation Residuals Time Positive first order autocorrelation occurs when consecutive residuals tend to be similar. Then, the value of d is small (less than 2). 0 +

44 Negative First Order Autocorrelation Residuals Time Negative first order autocorrelation occurs when consecutive residuals tend to markedly differ. Then, the value of d is large (greater than 2).

45 If d<d L there is enough evidence to show that positive first-order correlation exists If d>d U there is not enough evidence to show that positive first-order correlation exists If d is between d L and d U the test is inconclusive. One tail test for Positive First Order Autocorrelation dLdL First order correlation exists Inconclusive test Positive first order correlation Does not exists dUdU

46 One Tail Test for Negative First Order Autocorrelation If d>4-d L, negative first order correlation exists If d<4-d U, negative first order correlation does not exists if d falls between 4-d U and 4-d L the test is inconclusive. Negative first order correlation exists 4-d U 4-d L Inconclusive test Negative first order correlation does not exist

47 If d 4-d L first order autocorrelation exists If d falls between d L and d U or between 4-d U and 4-d L the test is inconclusive If d falls between d U and 4-d U there is no evidence for first order autocorrelation dLdL dUdU d U 4-d L First order correlation exists First order correlation exists Inconclusive test Inconclusive test First order correlation does not exist First order correlation does not exist Two-Tail Test for First Order Autocorrelation

48 Example 18.3 (Xm18-03)Xm18-03 –How does the weather affect the sales of lift tickets in a ski resort? –Data of the past 20 years sales of tickets, along with the total snowfall and the average temperature during Christmas week in each year, was collected. –The model hypothesized was TICKETS =  0 +  1 SNOWFALL +  2 TEMPERATURE+  –Regression analysis yielded the following results: Testing the Existence of Autocorrelation, Example

49 The Regression Equation – Assessment (I) The model seems to be very poor: The model seems to be very poor: R-square= It is not valid (Signif. F =0.3373) No variable is linearly related to Sales Xm18-03

50 Diagnostics: The Error Distribution The errors histogram The errors may be normally distributed

51 Residual vs. predicted y It appears there is no problem of heteroscedasticity (the error variance seems to be constant). Diagnostics: Heteroscedasticity

52 Residual over time Diagnostics: First Order Autocorrelation The errors are not independent!!

53 Test for positive first order auto- correlation: n=20, k=2. From the Durbin-Watson table we have: d L =1.10, d U =1.54. The statistic d= Conclusion: Because d<d L, there is sufficient evidence to infer that positive first order autocorrelation exists. Using the computer - Excel Tools > Data Analysis > Regression (check the residual option and then OK) Tools > Data Analysis Plus > Durbin Watson Statistic > Highlight the range of the residuals from the regression run > OK The residuals Diagnostics: First Order Autocorrelation

54 The Modified Model: Time Included The modified regression model (Xm18-03mod)Xm18-03mod TICKETS =  0 +  1 SNOWFALL +  2 TEMPERATURE +  3 TIME +  All the required conditions are met for this model. The fit of this model is high R 2 = The model is valid. Significance F = SNOWFALL and TIME are linearly related to ticket sales. TEMPERATURE is not linearly related to ticket sales.