Topic 14: Inference in Multiple Regression

Outline Review multiple linear regression Inference of regression coefficients –Application to book example Inference of mean –Application to book example Inference of future observation Diagnostics and remedies

Data for Multiple Regression Y i is the response variable X i1, X i2, …, X i,p-1 are the p-1 explanatory variables Y i, X i1, X i2, …, X i,p-1 are the data for case i, where i = 1 to n

Multiple Regression Model Y i = β 0 + β 1 X i1 + β 2 X i2 +…+ β p-1 X i,p-1 + e i Y i is the value of the response variable for the i th case β 0 is the intercept β 1, β 2, …, β p-1 are the regression coefficients for the explanatory variables e i are independent Normally distributed random errors with mean 0 and variance σ 2

Least Squares Solutions s 2 = MSE= s = Root MSE

ANOVA F-test H 0 : β 1 = β 2 = … = β p-1 = 0 H a : β k ≠ 0, for at least one k=1,2,…,p-1 Under H 0, F ~ F(p-1,n-p) Reject H 0 if F is large, using P-value we reject if the P-value ≤ 0.05

Inference for individual regression coefficients We can show b ~ N(β, σ 2 (X΄X) -1 ) Define

Significance Test for β k H 0 : β k = 0 Same test statistic t * = b k /s(b k ) Still use df E which now equals n-p P-value computed from t(n-p) dist This tests the significance of a variable given the other variables are already in the model (i.e., fitted last)

Confidence interval for β k CI: b k ± t c s(b k ), where t c = t(.975, n-p) Same form as before but df E now equals n-p This interval describes region of b k given the other variables are in the model

Example II (KNNL p 236) Dwaine Studios, Inc. operates portrait studios in 21 cities of medium size Y i is sales in city i X 1 : population aged 16 and under X 2 : per capita disposable income

Read in the data data a1; infile ‘../data/ch06fi05.txt'; input young income sales; proc print data=a1; run;

Partial Proc Print Results Obs young income sales

Proc Reg proc reg data=a1; model sales=young income; run;

Output Analysis of Variance SourceDF Sum of Squares Mean SquareF ValuePr > F Model <.0001 Error Corrected Total Root MSE R-Square0.917 At least one variable is helpful in predicting in sales

Output Parameter Estimates VariableDF Parameter Estimate Standard Errort ValuePr > |t| Intercept young <.0001 income Both variables are helpful in explaining sales after the other is already in the model

CLB option Used to get confidence intervals for each coefficient proc reg data=a1; model sales=young income/clb; run;

Output Parameter Estimates VariableDF Parameter Estimate Standard Error 95% Confidence Limits Intercept young income

What if just young fit? Parameter Estimates VariableDF Parameter Estimate Standard Error 95% Confidence Limits Intercept young CIs for both the intercept and young change dramatically when just young as explanatory variable

Estimation of E(Y h ) X h is now a vector that looks like (1, X h1, X h2, …, X h,p-1 )΄ We want a point estimate and a confidence interval for the subpopulation mean corresponding to the set of explanatory variables X h

Theory for E(Y h )

Using CLM option proc reg data=a1; model sales=young income/clm; id young income; run; Adds them to output table

CLM Output Output Statistics Obsyoungincome Dependent Variable Predicted Value Std Error Mean Predict95% CL Mean

Prediction of Y h X h is still a vector of form (1, X h1, X h2, …, X h,p-1 )΄ We want a prediction of Y h based on a set of predictor values with an interval that expresses the uncertainty in our prediction

Theory for Y h

Using the CLI option proc reg data=a1; model sales=young income/cli; id young income; run; Adds them to output table

CLI Output Output Statistics Obsyoungincome Dependent Variable Predicted Value Std Error Mean Predict95% CL Predict

Diagnostics Look at the distribution of each variable Look at the relationship between pairs of variables Plot the residuals versus –the predicted/fitted values –each explanatory variable –time (if available)

Diagnostics Are the residuals approximately Normal –Look at a histogram –Normal quantile plot Is the variance constant –Plot the residuals vs anything that might be related to the variance (e.g. residuals vs predicted values & residuals versus each X)

Remedies Similar remedies as simple regression Transformations such as Box-Cox Analyze with/without outliers More detail in KNNL Ch 9 and 10

Background Reading We finished Chapter 6. Program used to generate output for confidence intervals for means and prediction intervals is topic14.sas