1. Regression model with multiple predictors
Set 8
Regression model with multiple predictors

2. Multivariate Data +1 of variables One response (dependent) variable
y = price of an item purchased by the individual
K explanatory (independent) variables
x1 = income of an individual
x2 = education of an individual .
xK = age of an individual
Data for individual i : (x1i ,x2i ,. . . , xKi ,yi )

3. Graphs and Summary Measures
Matrix plot
Correlation matrix

4. Multiple Linear Regression Model
yi = b0 + b1x1i bKxKi + ei
Response = Model + Error term
xk is an explanatory (independent) variable
b0 and b1, . . ., bK are unknown parameters
b0, intercept
b1, . . ., bK, are regression coefficients
ei, unknown error, E(ei)=0, Var(ei)= s2

5. Least Square (LS) Estimation
Estimate the unknown regression
my|x’s = b0 + b1x bKxK
by the LS regression line
Given in regression output
LS estimate of the intercept parameter b0
LS model passes through the mean values

6. Statistics for each coefficient bj
Estimate bj for each coefficient bj
Standard error of the estimate for each coefficient: SE(bj)
T statistic for bj = 0 (xj can be dropped from the model)
Statistics that can be computed
Margin of error using t table:
ME(bj) = t times SE(bj), df=n-K-1
Interval estimate for bj:
[bj - ME(bj), bj + ME(bj)]
All values inside the interval for bj are acceptable
Any value outside of the interval for bj is not acceptable
If zero is inside the interval, xj can be dropped from the model

7. ANOVA Table Sums of Square
SS Total = SS Error SS Regression
Degrees of freedom:
n = n – K K
Given in regression output

8. ANOVA Table Mean Squares & F
Mean Square Error
s2 is the LS estimate of s2
Standard Error of regression
Mean Square regression
F-ratio (Test for the regression relationship)
Null model H: b1 = b2 = = bK = 0
df1=K, df2=n-K-1
Given in regression output

9. Analysis of Variance (ANOVA) Table
Source df
Regression K
Error
n-K-1
Total
n-1 SS
SSR
SSE
SST MS
MSR
MSE
F ratio

10. R2 and Adjusted R2
Fraction of variation of Y explained by the regression model
R2 increases as predictors are added to the model
F-ratio
Adjusted R2
R2adj takes the number of predictors into account
Useful for comparing models with different numbers of predictors
Given in regression output

11. The Special Case of K=1 R2=Corr2(Y, X) F ratio = (T ratio)2
When there is only one variable in the model, K=1
R2=Corr2(Y, X)
F ratio = (T ratio)2
When df1=1, F for a=.05 is equal to t2 for a=.025 with df2

12. Partial F Test Test for dropping more than one variable from the model
Compute the estimate of the model without the variables to be dropped (reduce model)
Partial F test
df1=Kfull-Kreduced, df2=n - Kfull -1 (Same as SSEfull )
Reject the reduced model if F is large
Sequential SS: Place predictors to be tested last in the models
Alternative formula with R2’s of the full and reduced models

13. Margins of Error for Predictions
Margin of error using t table: df=n-K-1
Prediction of the mean outcome
t (SE of prediction of the mean outcome)
Interval estimate
Given as an option output
Prediction of a single outcome
t (SE of prediction of the single outcome)
The interval for prediction for a single outcome is wider than the interval for the mean outcome

14. Residuals
Fitted values
LS Residuals

15. Residual Diagnostics Distribution of the residuals should be normal
Distribution plot
Normal probability plot
Tests of normality
Residuals must be patternless
Time series (sequence) plot of residuals
Plot of residuals against the fitted values
Plot of residuals against each xj