Multiple regression

Example: Brain and body size predictive of intelligence? Sample of n = 38 college students Response (Y): intelligence based on the PIQ (performance) scores from the (revised) Wechsler Adult Intelligence Scale. Predictor (X 1 ): Brain size based on MRI scans (given as count/10,000) Predictor (X 2 ): Height in inches Predictor (X 3 ): Weight in pounds

Scatter matrix plots Scatter plots of response versus predictor helps in determining nature and strength of relationships. Scatter plots of predictor versus predictor helps in studying their relationships, as well as identifying scope of model and outliers.

Scatter matrix plot

Matrix plot in Minitab Select Graph >> Matrix plot … Specify all of the variables (response and predictors) you want graphed. Select OK.

Correlation matrix Correlations: PIQ, MRI, Height, Weight PIQ MRI Height MRI Height Weight Cell Contents: Pearson correlation

Correlation matrix in Minitab Stat >> Basic statistics >> Correlation… Select all of the variables (response and predictors). To get a “crisper” table, de-select default “Display p-values”

Linear regression model with 3 predictors where: Y i = intelligence (PIQ) if student i X i1 = brain size of student i (MRI) X i2 = height of student i (Height) X i3 = weight of student i (Weight)

Fitting multiple regression model in Minitab It’s basically the same as fitting a simple linear regression model. Stat >> Regression >> Regression… Select response and all predictors. Specify all desired options as you would for simple linear regression.

The regression equation is PIQ = MRI Height Weight Predictor Coef SE Coef T P Constant MRI Height Weight How likely is it that b 3 = would be as extreme as it is (?!) if β 3 = 0?

Confidence intervals for β k Sample estimate ± margin of error Predictor Coef SE Coef T P Weight

The regression equation is PIQ = MRI Height Predictor Coef SE Coef T P Constant MRI Height S = R-Sq = 29.5% R-Sq(adj) = 25.5% Coefficient of (multiple) determination Adjusted coefficient of (multiple) determination

Coefficient of (multiple) determination Basically same as before. R 2 = SSR/SSTO = proportionate reduction in total variation in Y associated with using set of X 1, …, X p-1 variables. Again, a large R 2 value does not necessarily imply that the fitted model is a useful one.

Adjusted coefficient of multiple determination Problem: adding more X variables can only increase R 2, because SSTO never changes for a given set of data. But, the remaining error (quantified by SSE) can only get smaller (or stay the same) when more predictor variables are considered. Solution: adjust R 2 to take into account the number of predictors in the model.

Adjusted coefficient of multiple determination

PIQ = MRI Height S = R-Sq = 29.5% R-Sq(adj) = 25.5% Analysis of Variance Source DF SS MS F P Regression Error Total Calculation of R 2 (adj): Interpretation of R 2 (adj):

Impact of the adjustment It’s a trade-off. R-Sq(adj) may even become smaller when another predictor variable is introduced into the model. The regression equation is PIQ = MRI Height S = R-Sq = 29.5% R-Sq(adj) = 25.5% The regression equation is PIQ = MRI Height Weight S = R-Sq = 29.5% R-Sq(adj) = 23.3%

The regression equation is PIQ = MRI Height Analysis of Variance Source DF SS MS F P Regression Error Total Is there a relationship between the response variable and the set of predictor variables? How likely is it that the sample would yield such an extreme F-statistic if the null hypothesis were true?

Caution when predicting or estimating response

What is scope of model?

Predicted Values for New Observations New Obs Fit SE Fit 95.0% CI 95.0% PI (106.64,119.68) (73.02,153.30) (100.19,117.78) (68.41,149.56) Values of Predictors for New Observations New Obs MRI Height S = 19.51

Diagnostics and remedial measures Most procedures carry directly over (with minor modification) from simple linear regression to multiple linear regression. But, some procedures are specific only to multiple linear regression (chapters 9, 10)

Residuals against each predictor Gives an indication of the adequacy of the regression function with respect to each specific predictor variable.

Unusual Observations Obs MRI PIQ Fit SEFit Residual StResid R R denotes an obs’n with a large standardized residual

Residuals versus omitted predictors As usual. Plus, also consider plotting residuals against interaction terms, such as X 1 X 2, because they too are potentially important omitted variables.

Regression interaction terms in Minitab Use the calculator to create a new variable (MRI*Ht). Select Calc >> Calculator. Specify “Store result in variable” (MRI*Ht) Specify Expression: MRI*Height Select OK. The new (interaction) predictor variable will appear in worksheet.

Modified Levene test MRIGrp 1: le : gt 90.5

LOF Test Requires that there are at least some repeats of the same values across all predictor variables. X 1 = 59, X 2 = 63 and X 1 =59 and X 2 =63 is an example of a repeat. X 1 = 59, X 2 = 63 and X 1 =59 and X 2 =66 is not an example of a repeat.

Row MRI Height Row MRI Height

Attempted LOF Test The regression equation is PIQ = MRI Height Analysis of Variance Source DF SS MS F P Regression Error Total No replicates. Cannot do pure error test.