1. BUSI 410 Business Analytics
Module 19: Multiple Regression

2. Last lecture
Simple linear regression (with one driver)
Reading regression output
Presenting regression equation in a report

3. Something is missing…
We assumed 𝜀 is normal, is independent of the driver, and is independent of each other. Is it so?
Skew: 1.2

4. Multiple regression
Multiple regression—a regression with multiple independent variables (but still a single dependent variable)
Multiple regression accounts for the joint impact of multiple drivers

5. Sakura Motors: What drives car emission?
Use you logic to choose drivers
Fuel economy
Acceleration
Weight
Passenger capacity
Engine displacement
Cylinders
Horsepower

6. Sakura Motors: Choose a varied sample
We can explain variation in emission by variations in the drivers… only if there are variations!

7. Sakura Motors: Regression output looks good
R Square – % of variation in the dependent variable explained by the variation in the independent variables
SUMMARY OUTPUT
Regression Statistics
Multiple R
R Square
Adjusted R Square
Standard Error
Observations 34
ANOVA df SS MS F
Significance F
Regression 7
3.05E-13
Residual 26
Total 33
Coefficients
t Stat
P-value
Lower 95%
Upper 95%
Intercept
5.48E-05
MPG
1.53E-06
seconds
liters
pounds (K)
cylinders
horsepower
passengers
Significance F – the p-value for H0: all slopes are zero (R Square = 0) vs. H1: at least one slope is non-zero (R Square > 0)

8. Sakura Motors: A closer look
Coefficients
P-value
Intercept
E-05
MPG
E-06
seconds
liters
pounds (K)
cylinders
horsepower
passengers

9. Sakura Motors: A closer look
Insignificant drivers
Coefficients
P-value
Intercept
E-05
MPG
E-06
seconds
liters
pounds (K)
cylinders
horsepower
passengers
“Wrong” signs

10. (Multi)collinearity: Two or more drivers being highly correlated
Rule of thumb: multicollinearity if |correlation| > 0.7 between drivers
Multicollinearity symptoms
Important drivers appear insignificant
Coefficients have “wrong” signs
Increased forecast standard error

11. Sakura Motors: Confirming multicollinearity
MPG
seconds
liters
pounds (K)
cylinders
horsepower
passengers 1
-0.05
-0.81
-0.77
-0.74
-0.53
-0.17
-0.01
-0.19
-0.36
0.84
0.92
0.76
0.59
0.81
0.72
0.70
0.77
0.60
0.55

12. How to reduce multicollinearity?
Remove irrelevant or redundant information
Use parsimony (the “KISS” principle)
“Everything should be made as simple as possible, but not simpler” – Albert Einstein
Use liters to represent engine size (eliminate cylinders, horsepower)
Use pounds to represent car size (eliminate passengers)

13. Sakura Motors: Partial regression output
SUMMARY OUTPUT
Regression Statistics
Multiple R
R Square
(was )
Adjusted R Square
Standard Error
(was )
Observations 34
ANOVA df SS MS F
Significance F
Regression 4
E-16
Residual 29
Total 33
Coefficients
t Stat
P-value
Lower 95%
Upper 95%
Intercept
2.62E-05
MPG
2.38E-07
seconds
liters
pounds (K)
all signs “correct”
all drivers significant at 0.1

14. Sakura Motors: Is the partial model worse?
Removing a driver always decreases R2
Partial F-test is a hypothesis test for
H0: 𝑅 𝑝 2 = 𝑅 𝑓 2 (the partial model has as much explanatory power as the full model)
H1: 𝑅 𝑝 2 < 𝑅 𝑓 2 (the partial model has less explanatory power than the full model)
The p-value is equal to
F.DIST.RT(Reduced R2 per removed driver/Full model’s unexplained variations per residual DF, # of removed drivers, Full model’s residual DF)
F.DIST.RT( − / 1− ,3,26)=0.89

15. Exercise: Tackle multicollinearity
Predict salary using age, experience and education?
Drop age
Predict basketball performance using height and weight?
Use height and body mass index (BMI)
Predict election result using % of income groups?
Drop (any) one group

16. Sakura Motors: Final check on residuals
Residuals seem independent of drivers

17. Sakura Motors: Final check on residuals
Skewness = 0.46 Residuals seem approximately normal

18. Today’s assignment
Lab Practice 7, due on 11/9 by class time
Homework 4, due on 11/14 by class time
Group Case Project, due today by midnight
Project self- and peer-evaluation (Canvas=>Quizzes), 1 bonus pt

19. For next class
Bring your laptop, textbook and course pack to Lab Session