1. ANOVA for Regression
ANOVA tests whether the regression model has any explanatory power.
In the case of simple regression analysis the ANOVA test and the test for b1 are identical.

2. ANOVA for Regression MSE = SSE/(n-2) MSR = SSR/p
where p=number of independent variables
F = MSR/MSE

3. ANOVA Hypothesis Test H0: b1 = 0 Ha: b1 ≠ 0 Reject H0 if: F > Fa
Or if:
p < a

4. Regression and ANOVA Source of variation Sum of squares
Degrees of freedom
Mean Square F
Regression
SSR 1
MSR=SSR/1
F=MSR/MSE
Error
SSE
n-2
MSE=SSE/(n-2)
Total
SST
n-1

5. ANOVA and Regression ANOVA df SS MS F Significance F Regression 1 3364df SS MS F
Significance F
Regression 1
3364
273
1.23E-15
Residual 27
3334
12.3
Total 28
3697
Fa = given a=.05, df num. = 1, df denom. = 27

6. Issues with Hypothesis Test Results
Correlation does NOT prove causation
The test does not prove we used the correct functional form

7. Output with Temperature as Y
SUMMARY OUTPUT
Regression Statistics
Multiple R
R Square
Adjusted R Square
Standard Error
Observations 29
ANOVA df SS MS F
Significance F
Regression 1
E-15
Residual 27
Total 28
Coefficients
t Stat
P-value
Lower 95%
Upper 95%
Intercept
4.24E-28
Thousands of cubic feet
1.23E-15

10. Confidence Interval for Estimated Mean Value of y
xp = particular or given value of x
yp = value of the dependent variable for xp
E(yp) = expected value of yp
or E(y|x= xp)

11. Confidence Interval for Estimated Mean Value of y

12. Computing b0 and b1, Example
From example of car age, price: x y 1 15 -3 3 -9 9 14 -1 2 -2 11 4 12 8 5 -4
-20 25
Sum = 20 60
-30 36
Mean =
b1 =
-0.83
b0 =
15.33

13. Confidence Interval of Conditional Meanx y 1 15 9
14.5
6.2
0.3 3 14
12.84
0.7
1.3 4 11
3.4 12
12.01
0.0 8 25
7.86
17.4 16
Sum=20
Sum=60 36
SSR=25.0
SSE=5.0
SST=30
Mean=4
Mean=12
b1=-0.833
b0=15.33
r2 = 25/30 = .833

14. Confidence Interval of Conditional Mean

15. Confidence Interval of Conditional Mean
Given 1-a = .95 and df = 3:

16. Confidence Interval for Predicted Values of y
A confidence interval for a predicted value of y must take into account both random error in the estimate of b1 and the random deviations of individual values from the regression line.

17. Confidence Interval for Estimated Mean Value of y

18. Confidence Interval of Individual Value

19. Confidence Interval of Conditional Mean
Given 1-a = .95 and df = 3:

20. Residual Plots Against x
Residual – the difference between the observed value and the predicted value
Look for:
Evidence of a nonconstant variance
Nonlinear relationship

21. Regression and Outliers
Outliers can have a disproportionate effect on the estimated regression line.
Coefficients
Intercept
X Variable 1

22. Regression and Outliers
One solution is to estimate the model with and without the outlier.
Questions to ask:
Is the value a error?
Does the value reflect some unique circumstance?
Is the data point providing unique information about values outside of the range of other observations?

23. Chapter 15
Multiple Regression

24. Regression Multiple Regression Model
y = b0 + b1x1 + b2x2 + … + bpxp + e
Multiple Regression Equation
y = b0 + b1x1 + b2x2 + … + bpxp
Estimated Multiple Regression Equation

25. Car Data MPG Weight Year Cylinders 18 3504 70 8 15 3693 3436 16 343317
3449
4341 14
4354
4312
4425
3850
3563
3609 …

26. Multiple Regression, Example
Coefficients
Standard Error
t Stat
Intercept
46.3
0.800
57.8
Weight
-29.4
R Square
0.687
Coefficients
Standard Error
t Stat
Intercept
-14.7
3.96
-3.71
Weight
-31.0
Year
0.763
0.0490
15.5
R Square
0.807

27. Multiple Regression, Example
Coefficients
Standard Error
t Stat
Intercept
-14.4
4.03
-3.58
Weight
-14.1
Year
0.760
0.0498
15.2
Cylinders
0.232
-0.319
R Square
0.807
Predicted MPG for car weighing 4000 lbs built in 1980 with 6 cylinders:
(4000)+.76(80)-.0741(6)
= =19.88

28. Multiple Regression Model
SST = SSR + SSE

29. Multiple Coefficient of Determination
The share of the variation explained by the estimated model.
R2 = SSR/SST

30. F Test for Overall Significance
H0: b1 = b1 = = bp
Ha: One or more of the parameters is not equal to zero
Reject H0 if: F > Fa Or
Reject H0 if: p-value < a
F = MSR/MSE

31. ANOVA Table for Multiple Regression Model
Source
Sum of Squares
Degrees of Freedom
Mean Squares F
Regression
SSR p
MSR = SSR/p
F=MSR/MSE
Error
SSE
n-p-1
MSE = SSE/(n-p-1)
Total
SST
n-1

32. t Test for Coefficients
H0: b1 = 0
Ha: b1 ≠ 0
Reject H0 if:
t < -ta/2 or t > ta/2
Or if:
p < a
t = b1/sb1
With a t distribution of n-p-1 df

33. Multicollinearity
When two or more independent variables are highly correlated.
When multicollinearity is severe the estimated values of coefficients will be unreliable
Two guidelines for multicollinearity:
If the absolute value of the correlation coefficient for two independent variables exceeds 0.7
If the correlation coefficient for independent variable and some other independent variable is greater than the correlation with the dependent variable

34. Multicollinearity MPG Weight Year Cylinders 1 -0.829 0.578 -0.300
MPG
Weight
Year
Cylinders 1
-0.829
0.578
-0.300
-0.773
0.895
-0.344