Diagnostics – Part I Using plots to check to see if the assumptions we made about the model are realistic

Diagnostic methods Some simple (but subjective) plots. (Now) Formal statistical tests. (Next)

Review of some simple plots … while checking scope of model

Dot Plot

Summarizes quantitative data. Horizontal axis represents measurement scale. Plot one dot for each data point.

Stem-and-Leaf Plot Stem-and-leaf of Shoes N = 139 Leaf Unit = (33)

Stem-and-Leaf Plot Summarizes quantitative data. Each data point is broken down into a “stem” and a “leaf.” First, “stems” are aligned in a column. Then, “leaves” are attached to the stems.

Box Plot

Summarizes quantitative data. Vertical (or horizontal) axis represents measurement scale. Lines in box represent the 25th percentile (“first quartile”), the 50th percentile (“median”), and the 75th percentile (“third quartile”), respectively.

Box Plot (cont’d) “Whiskers” are drawn to the most extreme data points that are not more than 1.5 times the length of the box beyond either quartile. –Whiskers are useful for identifying outliers. “Outliers,” or extreme observations, are denoted by asterisks. –Generally, data points falling beyond the whiskers are considered outliers.

Okay, now the really new stuff…

Simple linear regression model Error terms have mean 0, i.e., E(  i ) = 0.  i and  j are uncorrelated (independent). Error terms have same variance, i.e., Var(  i ) =  2. Error terms  i are normally distributed. The response Y i is a function of a systematic linear component and a random error component: with assumptions that:

Why should we keep nagging ourselves about the model? All of the estimates, confidence intervals, prediction intervals, hypothesis tests, etc. have been developed assuming that the model is correct. If the model is incorrect, then the formulas and methods we use are at risk of being incorrect. (Some are more forgiving than others.)

Things that can go wrong with the model Regression function is not linear. Error terms do not have constant variance. Error terms are not independent. The model fits all but one or a few outlier observations. Error terms are not normally distributed. Important predictor variable(s) has been left out of the model.

Residual analysis … the basic idea We would think the observed residuals: would reflect the properties assumed for the unknown true error terms: So, investigate the observed residuals to see if they behave “properly.”

Some points of clarification about residuals The mean of the residuals, e-bar, is 0. So, no need to check that the mean of the residuals is 0 – the LS estimation method has made it so. The residuals are not independent, since they are all a function of the same estimated regression function.

Example: Alcohol consumption (X) and Arm muscle strength (Y)

A well-behaved “residuals vs. fits” plot

Characteristics of a well-behaved “residual versus fits” plot The residuals “bounce randomly” around the 0 line. (Linear is reasonable). No one residual “stands out” from the basic random pattern of residuals. (No outliers). The residuals roughly form a “horizontal band” around 0 line. (Constant variance).

“Residuals versus predictor” plot offers nothing different.

Example: Is tire tread wear linearly related to mileage? mileagegroove X = mileage in 1000 miles Y = groove depth in mils (0.001 inches)

Example: Is tire tread wear linearly related to mileage?

A “residual versus fits” plot suggesting relationship is not linear

How a non-linear function shows up on a “residual versus fits” plot The residuals depart from 0 in a systematic fashion, such as being positive for small X values, negative for medium X values, and positive again for large X values.

Example: How is plutonium activity related to alpha particle counts?

A residual versus fits plot suggesting non-constant error variance

How non-constant error variance shows up on a “residual vs. fits” plot The plot has a “fanning” effect, such as the residuals being close to 0 for small X values and being much more spread out for large X values. The “fanning” effect can also be in the reverse direction. Or, the spread of the residuals can vary in some complex fashion.

Example: Relationship between tobacco use and alcohol use? Region Alcohol Tobacco North Yorkshire Northeast EastMidlands WestMidlands EastAnglia Southeast Southwest Wales Scotland Northern Ireland Family Expenditure Survey of British Dept. of Employment X = average weekly expenditure on tobacco Y = average weekly expenditure on alcohol

Example: Relationship between tobacco use and alcohol use?

A “residual versus fits” plot suggesting an outlier exists. “outlier”

How large does a residual need to be before being flagged? The magnitude of the residuals depends on the units of the response variable. Make the residuals “unitless” by dividing by their standard deviation. That is, use “standardized residuals.” Then, an observation with a standardized residual greater than 2 or smaller than -2 should be flagged for further investigation.

Standardized residuals versus fits plot

Minitab identifies observations with large standardized residuals … Unusual Observations Obs Tobacco Alcohol Fit SE Fit Resid St Resid R R denotes an observation with a large standardized residual.

Anscombe data set #3

A “residual versus fits” plot suggesting an outlier exists.

How an outlier shows up on a “residuals vs. fits” plot The observation’s residual stands apart from the basic random pattern of the rest of the residuals. The random pattern of the residual plot can even disappear if one outlier really deviates from the straight line of the rest of the data.

Other simple plots that might help spot an outlier Boxplots Stem-n-leaf plots Dotplots

Boxplot of residuals for Alcohol (Y) and Tobacco (X) example

Dotplot of residuals for Alcohol (Y) and Tobacco (X) example

“Residuals vs. order” plots to assess non-independence of error terms If the data are obtained in a time (or space) sequence, a “residuals vs. order” plot helps to see if there is any correlation between error terms that are near each other in the sequence. A horizontal band bouncing randomly around 0 suggests errors are independent, while a systematic pattern suggests not.

“Residuals vs order” plots suggesting non-independence of error terms

Normal probability plot to assess normality of error terms Plot of residuals on horizontal axis against expected values of the residuals under normality (normal scores) on vertical axis. Plot that is nearly linear suggests normality of error terms.

Normal probability plot interpretation skewed right skewed left normal

Normal probability plot for Alcohol (X) and Strength (Y) example

Normal probability plot for Tree diameter (X) and C-dating Age (Y)

“Residuals vs omitted predictors” plots To determine whether there are any other key variables that could provide additional predictive power to the response. Look for systematic patterns. If the plot reveals that the residuals vary systematically, we don’t say the original model is wrong. It’s just that it can be improved.

“Residuals vs omitted” plot

In summary, …

Nonlinearity of regression function Scatter plot of response versus predictor (Standardized) residuals versus fits plot (Standardized) residuals versus predictor plot

Nonconstancy or error variance (Standardized) residuals versus fits plot (Standardized) residuals versus predictor plot

Presence of outliers (Standardized) residuals versus fits plot (Standardized) residuals versus predictor plot Box plots, stem-n-leaf plots, dot plots of (standardized) residuals

Non-independence of error terms (Standardized) residuals versus order plot

Non-normality of error terms Normality probability plots Box plots, dotplots, stem-n-leaf plots Mean far from median?

Residual vs … plots in Minitab Stat >> Regression >> Regression. Specify predictor and response. Under Graphs…, specify whether regular or standardized residuals desired. Select which residual plots are desired. If residual versus predictor plot desired, specify predictor in box. Select OK.

Boxplots, dotplots, etc. of residuals Stat >> Regression >> Regression … Specify predictor and response. Under Storage…, select residuals and/or standardized residuals. They will be stored in worksheet. Then … Graph >> Boxplot… or Graph >>Dotplot… or Graph>>Stemleaf…