Anaregweek11 Regression diagnostics

Regression Diagnostics Partial regression plots Studentized deleted residuals Hat matrix diagonals Dffits, Cook’s D, DFBETAS Variance inflation factor Tolerance

NKNW Example NKNW p 389, section 11.1 Y is amount of life insurance X 1 is average annual income X 2 is a risk aversion score n = 18 managers

Manajer i Income X i1 Risk X i2 Life Insurance Y i Manajer i Income X i1 Risk X i2 Life Insurance Y i

Partial regression plots Also called added variable plots or adjusted variable plots One plot for each X i

Partial regression plots (2) Consider X 1 –Use the other X’s to predict Y –Use the other X’s to predict X 1 –Plot the residuals from the first regression vs the residuals from the second regression

Partial regression plots (3) These plots can detect –Nonlinear relationships –Heterogeneous variances –Outliers

Output Source DF F Value Pr > F Model <.0001 Error 15 C Total 17 Root MSE R-Square

Output (2) Par St Var Est Err t Pr > |t| Int <.0001 income <.0001 risk

Plot the residuals vs each Indep Variables From the regression of Y on X 1 and X 2 we plot the residual against each of indep. Variable. The plot of residual against X 1 indicates a curvelinear effect. Therefore, we need to check further by looking at the partial regression plot

Plot the residuals vs Risk

Plot the residuals vs income

The partial regression plots To generate the partial regression plots Regress Y and X 1 each on X 2. Get the residual from each regression namely e(Y|X 2 ) and e(X 1 |X 2 ) Plot e(Y|X 2 ) against e(X 1 |X 2 ) Do the same for Y and X 2 each on X 1.

The partial regression plots (2)

The partial regression plots(3)

Residuals There are several versions –Residuals e i = Y i – Ŷ i –Studentized residuals e i / √MSE –Deleted residuals : d i = e i / (1-h ii ) where h ii is the leverage –Studentized deleted residuals d i * = d i / s(d i ) Where Or equivalenly

Residuals (2) We use the notation (i) to indicate that case i has been deleted from the computations X (i) is the X matrix with case i deleted MSE (i) is the MSE with case i deleted

Residuals (3) When we examine the residuals we are looking for –Outliers –Non normal error distributions –Influential observations

Hat matrix diagonals h ii is a measure of how much Y i is contributing to the prediction Y i (hat) Ŷ 1 = h 11 Y 1 + h 12 Y 2 + h 13 Y 3 + … h ii is sometimes called the leverage of the i th observation

Hat matrix diagonals (2) 0 < h ii < 1 Σ h ii = p We would like h ii to be small The average value is p/n Values far from this average point to cases that should be examined carefully

Hat diagonals Hat Diag Obs H

DFFITS A measure of the influence of case i on Ŷ i It is a standardized version of the difference between Ŷ i computed with and without case i It is closely related to h ii

Cook’s Distance A measure of the influence of case i on all of the Ŷ i ’s It is a standardized version of the sum of squares of the differences between the predicted values computed with and without case i

DFBETAS A measure of the influence of case i on each of the regression coefficients It is a standardized version of the difference between the regression coefficient computed with and without case i

Variance Inflation Factor The VIF is related to the variance of the estimated regression coefficients We calculate it for each explanatory variable One suggested rule is that a value of 10 or more indicates excessive multicollinearity

Tolerance TOL = (1 – R 2 k ) Where R 2 k is the squared multiple correlation obtained in a regression where all other explanatory variables are used to predict X k TOL = 1/VIF Described in comment on p 411

Output (Tolerance) Variable Tolerance Intercept. income risk

Last slide Read NKNW Chapter 11