Multiple Linear Regression Regression Diagnostics

Find Scores That Contribute to violation of assumptions. Are suspect because they are far removed from the centroid (multidimensional mean) Have undue influence on the solution.

Outliers Among the Predictors Leverage, hi or Hat Diagonal The larger this statistic, the greater the distance between the data point and the centroid in p-dimensional space. Investigate cases with hi greater than 2(p-1)/N. p is the number of parameters in the model, including the intercept.

Distance from the Regression Surface Standardized Residual (aka Studentized Residual) Difference between actual Y and predicted Y divided by an appropriate standard error Rstudent (aka Studentized Deleted Residual) – same except for each case the regression surface is that obtained when this individual case is removed. Investigate if greater than 2.

Influence on the Solution Cook’s D – how much would the regression surface change if this case were removed Investigate cases with D > 1. Dfbetas – how much would one parameter (slope or intercept) change if this case were removed Investigate cases with values > 2.

SAS Code data regdiag; input SpermCount Together LastEjac @@; SR_LastEjac = sqrt(1+LastEjac); cards; *<data here>; proc univariate plot; var SpermCount -- SR_LastEjac; proc reg; model SpermCount = Together SR_LastEjac / influence r ; run; *<nonsignificant results>; data culled; set regdiag; If SpermCount < 700; proc reg; model SpermCount = Together SR_LastEjac / influence r ; *<Significant results>; title 'One Outlier Culled'; run;

Simple Example Y = sperm count X1 = % time recently spent with mate X2 = time since last ejaculation Output Statistics Obs Student Residual Cook's D RStudent Hat Diag H DFBETAS Intercept Together SR_LastEjac 5 1.012 0.426 1.0139 0.5551 -0.0160 0.0288 -0.0236 8 -0.183 0.006 -0.1715 0.3605 -0.0959 0.1083 0.0437 9 -1.240 0.098 -1.2906 0.1600 -0.0398 -0.2265 0.0999 10 -1.270 0.261 -1.3296 0.3270 -0.2614 -0.2321 0.4657 11 2.643 1.183 6.9409 0.3369 1.6194 1.0137 -2.6903

Leverage Investigate cases with values greater than 2(3)/11 = .55. Case 5 is above this cutoff. It is a univariate outlier on the LastEjac variable. Further investigation indicates the case is valid, so we retain it.

Residuals Case 11 has large residuals, it should be investigated. Notice that Rstudent is much larger than the standardized residual This indicates that removing this case has a large effect on the solution. Output Statistics Obs Student Residual Cook's D RStudent Hat Diag H DFBETAS Intercept Together SR_LastEjac 11 2.643 1.183 6.9409 0.3369 1.6194 1.0137 -2.6903

Influence Case 11 has a high value of Cook’s D. It has a high Dfbeta for the time since last ejaculation predictor, even after I transformed that variable to reduce skewness. Upon investigation, it was found that this subject did not follow the instructions for gathering the data. His scores were deleted.

Plots of Residuals These can also be useful, but It takes some practice to get good at detecting problems from such plots Plot the residual versus predicted Y

Heteroscedasticity

Trying Squaring One Predictor

Residuals not Normal and Variance not Constant