Multiple Regression Models

The Multiple Regression Model The relationship between one dependent & two or more independent variables is a linear function Population Y-intercept Population slopes Dependent (Response) variable for sample Independent (Explanatory) variables for sample model Random Error

Multiple Regression Model: Example ( 0 F) Develop a model for estimating heating oil used for a single family home in the month of January based on average temperature and amount of insulation in inches.

Sample Multiple Regression Model: Example Excel Output For each degree increase in temperature, the estimated average amount of heating oil used is decreased by gallons, holding insulation constant. For each increase in one inch of insulation, the estimated average use of heating oil is decreased by gallons, holding temperature constant.

Slope (b i ) The average Y changes by b i each time X i is increased or decreased by 1 unit holding all other variables constant. For example: If b 1 = -2, then fuel oil usage (Y) is expected to decrease by an estimated 2 gallons for each 1 degree increase in temperature (X 1 ) given the inches of insulation (X 2 ). Interpretation of Estimated Coefficients

Intercept (b 0 ) The intercept (b 0 ) is the estimated average value of Y when all X i = 0. Interpretation of Estimated Coefficients

Using The Model to Make Predictions Predict the amount of heating oil used for a home if the average temperature is 30 0 and the insulation is 6 inches. The predicted heating oil used is gallons

Developing the Model Checking for problems. Being sure the model passes all tests for model quality.

Identifying Problems Do all the residual tests listed for simple regression. Check for multicolinearity.

Multicolinearity This occurs when there is a high correlation between the explanatory variables. This leads to unstable coefficients. The VIF used to measure colinearity (values exceeding 5 are not good and exceeding 10 are a big problem): = Coefficient of Multiple Determination of X j with all the others

Is the fit to the data good?

Coefficient of Multiple Determination Excel Output r 2 Adjusted r 2 The r 2 is adjusted downward to reflect small sample sizes.

Do the variables collectively pass the test?

Testing for Overall Significance Shows if there is a linear relationship between all of the X variables taken together and Y Hypothesis: H 0 :  1 =  2 = … =  p = 0 (No linear relationships) H 1 : At least one  i  0 (At least one independent variable effects Y)

Test for Overall Significance Excel Output: Example p = 2, the number of explanatory variables n - 1 MSR MSE p value = F Test Statistic

F H 0 :  1 =  2 = … =  p = 0 H 1 : At least one  I  0  =.05 df = 2 and 12 Critical value(s): Test Statistic: Decision: Conclusion: Reject at  = 0.05 There is evidence that at least one independent variable affects Y.  = 0.05 F  Test for Overall Significance (Excel Output)

Test for Significance: Individual Variables Shows if there is a linear relationship between each variable X i and Y. Hypotheses: H 0 :  i = 0 (No linear relationship) H 1 :  i  0 (Linear relationship between X i and Y)

T Test Statistic Excel Output: Example t Test Statistic for X 1 (Temperature) t Test Statistic for X 2 (Insulation)

H 0 :  1 = 0 h 1 :  1  0 df = n-2 = 12 critical value(s): Test Statistic: Decision: Conclusion: Reject H 0 at  = 0.05 There is evidence of a significant effect of temperature on oil consumption. t Reject H Does temperature have a significant effect on monthly consumption of heating oil? Test at  = t Test : Example Solution t Test Statistic =

Confidence Interval Estimate For The Slope Provide the 95% confidence interval for the population slope  1 (the effect of temperature on oil consumption)   1  The estimated average consumption of oil is reduced by between 4.7 gallons to 6.17 gallons per each increase of 1 0 F.

Special Regression Topics

Dummy-variable Models Create a categorical variable (dummy variable) with 2 levels:  For example, yes and no or male and female.  The date is coded as 0 or 1. The coding makes the intercepts different. This analysis assumes equal slopes. The regression model has same form:

Dummy-variable Models Assumption Given: Y = Assessed Value of House X 1 = Square footage of House X 2 = Desirability of Neighborhood = Desirable (X 2 = 1) Undesirable (X 2 = 0) 0 if undesirable 1 if desirable Same slopes

Dummy-variable Models Assumption X 1 (Square footage) Y (Assessed Value) Desirable Location Undesirable b 0 + b 2 b0b0 Same slopes Intercepts different

Interpretation of the Dummy Variable Coefficient For example: : GPA 0 Female 1 Male : Annual salary of college graduate in thousand $ This 6 is interpreted as given the same GPA, the male college graduate is making an estimated 6 thousand dollars more than female on average. :