1. Chapter 18
Multiple Regression

2. 18.1 Introduction
In this chapter we extend the simple linear regression model, and allow for any number of independent variables.
We expect to build a model that fits the data better than the simple linear regression model.

3. We will use computer printout to
Assess the model
How well it fits the data
Is it useful
Are any required conditions violated?
Employ the model
Interpreting the coefficients
Predictions using the prediction equation
Estimating the expected value of the dependent variable

4. 18.2 Model and Required Conditions
We allow for k independent variables to potentially be related to the dependent variable
y = b0 + b1x1+ b2x2 + …+ bkxk + e
Coefficients
Random error variable
Dependent variable
Independent variables

5. y y = b0 + b1x y = b0 + b1x y = b0 + b1x y = b0 + b1x
The simple linear regression model
allows for one independent variable, “x”
y =b0 + b1x + e y
y = b0 + b1x
y = b0 + b1x
y = b0 + b1x
y = b0 + b1x
Note how the straight line
becomes a plain, and...
y = b0 + b1x1 + b2x2
y = b0 + b1x1 + b2x2
y = b0 + b1x1 + b2x2
y = b0 + b1x1 + b2x2
y = b0 + b1x1 + b2x2
y = b0 + b1x1 + b2x2 X 1
y = b0 + b1x1 + b2x2
The multiple linear regression model
allows for more than one independent variable.
Y = b0 + b1x1 + b2x2 + e X2

6. y
y= b0+ b1x2 b0 X 1
y = b0 + b1x12 + b2x2
… a parabola becomes a parabolic surface X2

7. Required conditions for the error variable e
The error e is normally distributed with mean equal to zero and a constant standard deviation se (independent of the value of y). se is unknown.
The errors are independent.
These conditions are required in order to
estimate the model coefficients,
assess the resulting model.

8. 18.3 Estimating the Coefficients and Assessing the Model
The procedure
Obtain the model coefficients and statistics using a statistical computer software.
Diagnose violations of required conditions. Try to remedy problems when identified.
Assess the model fit and usefulness using the model statistics.
If the model passes the assessment tests, use it to interpret the coefficients and generate predictions.

9. Example 18.1 Where to locate a new motor inn?
La Quinta Motor Inns is planning an expansion.
Management wishes to predict which sites are likely to be profitable.
Several areas where predictors of profitability can be identified are:
Competition
Market awareness
Demand generators
Demographics
Physical quality

10. Profitability Margin Competition Market awareness Customers Community
Physical
Rooms
Nearest
Office
space
College
enrollment
Income
Disttwn
Number of
hotels/motels
rooms within
3 miles from
the site.
Distance to
the nearest
La Quinta inn.
Median
household
income.
Distance to
downtown.

11. Margin =b0 + b1Rooms + b2Nearest + b3Office + b4College
Data was collected from randomly selected 100 inns that belong to La Quinta, and ran for the following suggested model:
Margin =b0 + b1Rooms + b2Nearest + b3Office + b4College
+ b5Income + b6Disttwn +

12. Let us assess this equation
This is the sample regression equation
(sometimes called the prediction equation)
Excel output
MARGIN = ROOMS NEAREST
+ 0.02OFFICE COLLEGE
INCOME DISTTWN
Let us assess this equation

13. Standard error of estimate
We need to estimate the standard error of estimate
Compare se to the mean value of y
From the printout, Standard Error =
Calculating the mean value of y we have
It seems se is not particularly small.
Can we conclude the model does not fit the data well?

14. Coefficient of determination
The definition is
From the printout, R2 =
52.51% of the variation in the measure of profitability is explained by the linear regression model formulated above.
When adjusted for degrees of freedom, Adjusted R2 = 1-[SSE/(n-k-1)] / [SS(Total)/(n-1)] =
= 49.44%

15. Testing the validity of the model
We pose the question:
Is there at least one independent variable linearly related to the dependent variable?
To answer the question we test the hypothesis
H0: b1 = b2 = … = bk = 0
H1: At least one bi is not equal to zero.
If at least one bi is not equal to zero, the model is valid.

16. To test these hypotheses we perform an analysis of variance procedure.
The F test
Construct the F statistic
Rejection region
F>Fa,k,n-k-1
MSR=SSR/k
MSE
MSR F =
[Variation in y] = SSR + SSE.
Large F results from a large SSR.
Then, much of the variation in y is
explained by the regression model.
The null hypothesis should
be rejected; thus, the model is valid.
MSE=SSE/(n-k-1)
Required conditions must
be satisfied.

17. Example 18.1 - continued Excel provides the following ANOVA results
MSR/MSE
MSE
SSE
SSR
MSR

18. Example 18.1 - continued Excel provides the following ANOVA results
Conclusion: There is sufficient evidence to reject
the null hypothesis in favor of the alternative hypothesis.
At least one of the bi is not equal to zero. Thus, at least one independent variable is linearly related to y.
This linear regression model is valid
Fa,k,n-k-1 = F0.05,6, =2.17
F = > 2.17
Also, the p-value (Significance F) = (10)-13
Clearly, a = 0.05> (10)-13, and the null hypothesis
is rejected.

19. Let us interpret the coefficients
This is the intercept, the value of y when all the variables take the value zero. Since the data range of all the independent variables do not cover the value zero, do not interpret the intercept.
In this model, for each additional 1000 rooms within 3 mile of the La Quinta inn, the operating margin decreases on the average by 7.6% (assuming the other variables are held constant).

20. In this model, for each additional mile that the nearest competitor is to La Quinta inn, the average operating margin decreases by 1.65%
For each additional 1000 sq-ft of office space, the average increase in operating margin will be .02%.
For additional thousand students MARGIN increases by .21%.
For additional $1000 increase in median household income, MARGIN decreases by .41%
For each additional mile to the downtown center, MARGIN increases by .23% on the average

21. Testing the coefficients
The hypothesis for each bi
Excel printout
Test statistic
H0: bi = 0
H1: bi = 0
d.f. = n - k -1

22. Using the linear regression equation
The model can be used by
Producing a prediction interval for the particular value of y, for a given set of values of xi.
Producing an interval estimate for the expected value of y, for a given set of values of xi.
The model can be used to learn about relationships between the independent variables xi, and the dependent variable y, by interpreting the coefficients bi

23. Example 18.1 - continued. Produce predictions
Predict the MARGIN of an inn at a site with the following characteristics:
3815 rooms within 3 miles,
Closet competitor 3.4 miles away,
476,000 sq-ft of office space,
24,500 college students,
$39,000 median household income,
3.6 miles distance to downtown center.
MARGIN = (3815) (3.4) (476)
+0.212(24.5) (39) (3.6) = 37.1%

24. 18.4 Regression Diagnostics - II
The required conditions for the model assessment to apply must be checked.
Is the error variable normally distributed?
Is the error variance constant?
Are the errors independent?
Can we identify outliers?
Is multicollinearity a problem?
Draw a histogram of the residuals
Plot the residuals versus y ^
Plot the residuals versus the time periods

25. Example 18.2 House price and multicollinearity
A real estate agent believes that a house selling price can be predicted using the house size, number of bedrooms, and lot size.
A random sample of 100 houses was drawn and data recorded.
Analyze the relationship among the four variables

26. Solution
The proposed model is PRICE = b0 + b1BEDROOMS + b2H-SIZE +b3LOTSIZE + e
Excel solution
The model is valid, but no
variable is significantly related
to the selling price !!

27. Multicollinearity causes two kinds of difficulties:
However,
when regressing the price on each independent variable alone, it is found that each variable is strongly related to the selling price.
Multicollinearity is the source of this problem.
Multicollinearity causes two kinds of difficulties:
The t statistics appear to be too small.
The b coefficients cannot be interpreted as “slopes”.

28. Remedying violations of the required conditions
Nonnormality or heteroscedasticity can be remedied using transformations on the y variable.
The transformations can improve the linear relationship between the dependent variable and the independent variables.
Many computer software systems allow us to make the transformations easily.

29. A brief list of transformations
y’ = log y (for y > 0)
Use when the se increases with y, or
Use when the error distribution is positively skewed
y’ = y2
Use when the s2e is proportional to E(y), or
Use when the error distribution is negatively skewed
y’ = y1/2 (for y > 0)
Use when the s2e is proportional to E(y)
y’ = 1/y
Use when s2e increases significantly when y increases beyond some value.

30. Example 18.3: Analysis, diagnostics, transformations.
A statistics professor wanted to know whether time limit affect the marks on a quiz?
A random sample of 100 students was split into 5 groups.
Each student wrote a quiz, but each group was given a different time limit. See data below. M a r k s
Analyze these results, and include diagnostics

31. The model tested: MARK = b0 + b1TIME + e The errors seem to be
normally distributed
This model is useful and
provides a good fit.

32. Two transformations are used to remedy this problem: 1. y’ = logey
The standard error of estimate seems to increase
with the predicted value of y.
Two transformations are used to remedy this problem:
1. y’ = logey
2. y’ = 1/y

33. Let us see what happens when a transformation is applied
The original data, where “Mark”
is a function of “Time”
The modified data, where
LogMark is a function of “Time"
Loge23 = 3.135
40, 3.135
40,23
40, 2.89
Loge18 = 2.89
40,18

34. The new regression analysis and the diagnostics are:
The model tested:
LOGMARK = b’0 + b’1TIME + e’
Predicted LogMark = Time
This model is useful and
provides a good fit.

35. The errors seem to be
normally distributed
The standard errors still changes
with the predicted y, but the change
is smaller than before.

36. How do we use the modified model to predict?
Let TIME = 55 minutes
LogMark = Time = (55) = 3.323
To find the predicted mark, take the antilog:
antiloge3.323 = e3.323 =
How do we use the modified model to predict?

37. 18.5 Regression Diagnostics - III
The Durbin - Watson Test
This test detects first order auto-correlation between consecutive residuals in a time series
If autocorrelation exists the error variables are not independent
Residual at time i

38. Positive first order autocorrelation
Positive first order autocorrelation occurs when
consecutive residuals tend to be similar. Then,
the value of d is small (less than 2). +
Residuals + + +
Time + + + +
Negative first order autocorrelation
Negative first order autocorrelation occurs when
consecutive residuals tend to markedly differ.
Then, the value of d is large (greater than 2).
Residuals + + + + +
Time + +

39. One tail test for positive first order auto-correlation
If d<dL there is enough evidence to show that positive first-order correlation exists
If d>dU there is not enough evidence to show that positive first-order correlation exists
If d is between dL and dU the test is inconclusive.
One tail test for negative first order auto-correlation
If d>4-dL, negative first order correlation exists
If d<4-dU, negative first order correlation does not exists
if d falls between 4-dU and 4-dL the test is inconclusive.

40. Two-tail test for first order auto-correlation
If d<dL or d>4-dL first order auto-correlation exists
If d falls between dL and dU or between 4-dU and 4-dL the test is inconclusive
If d falls between dU and 4-dU there is no evidence for first order auto-correlation
First order
correlation
does not
exist
First order
correlation
exists
Inconclusive
test dL dU 2
4-dU
4-dL 4

41. TICKETS=b0+b1SNOWFALL+b2TEMPERATURE+e
Example 18.4
How does the weather affect the sales of lift tickets in a ski resort?
Data of the past 20 years sales of tickets, along with the total snowfall and the average temperature during Christmas week in each year, was collected.
The model hypothesized was
TICKETS=b0+b1SNOWFALL+b2TEMPERATURE+e
Regression analysis yielded the following results:

42. Diagnosis of the required conditions resulted with
The model seems to be very poor:
The fit is very low (R-square=0.12),
It is not valid (Signif. F =0.33)
No variable is linearly related to Sales
Diagnosis of the required
conditions resulted with
the following findings

43. Residual vs. predicted y
The error distribution
The error variance
is constant
The errors may be
normally distributed
Residual over time
The errors are
not independent

44. Using the computer - Excel
Test for positive first order auto-correlation:
n=20, k=2. From the Durbin-Watson table we have:
dL=1.10, dU= The statistic d=0.59
Conclusion: Because d<dL , there is sufficient evidence
to infer that positive first order auto-correlation exists.
Using the computer - Excel
Tools > data Analysis > Regression (check the residual option and then OK)
Tools > Data Analysis Plus > Durbin Watson Statistic > Highlight the range of the residuals
from the regression run > OK
The residuals

45. The modified regression model
TICKETS=b0+ b1SNOWFALL+ b2TEMPERATURE+ b3YEARS+e
The autocorrelation has occurred over time.
Therefore, a time dependent variable added
to the model may correct the problem
All the required conditions are met for this model.
The fit of this model is high R2 = 0.74.
The model is useful. Significance F = 5.93 E-5.
SNOWFALL and YEARS are linearly related to ticket sales.
TEMPERATURE is not linearly related to ticket sales.