6-3 Multiple Regression Estimation of Parameters in Multiple Regression

6-3 Multiple Regression Estimation of Parameters in Multiple Regression The least squares function is given by The least squares estimates must satisfy

6-3 Multiple Regression Estimation of Parameters in Multiple Regression The least squares normal equations are The solution to the normal equations are the least squares estimators of the regression coefficients.

6-3 Multiple Regression X’X in Multiple Regression

6-3 Multiple Regression

6-3.1 Estimation of Parameters in Multiple Regression

6-3 Multiple Regression

6-3.2 Inferences in Multiple Regression Test for Significance of Regression

6-3 Multiple Regression Inferences in Multiple Regression Inference on Individual Regression Coefficients

6-3 Multiple Regression Inferences in Multiple Regression Confidence Intervals on the Mean Response and Prediction Intervals

6-3 Multiple Regression Confidence Intervals on the Mean Response and Prediction Intervals The response at the point of interest is and the corresponding predicted value is

6-3 Multiple Regression Inferences in Multiple Regression Confidence Intervals on the Mean Response and Prediction Intervals

6-3 Multiple Regression Checking Model Adequacy Residual Analysis

6-3 Multiple Regression Checking Model Adequacy Residual Analysis

6-3 Multiple Regression Checking Model Adequacy Residual Analysis

6-3 Multiple Regression Checking Model Adequacy Residual Analysis

6-3 Multiple Regression Checking Model Adequacy Residual Analysis

6-3 Multiple Regression Checking Model Adequacy Influential Observations The disposition of points in the x-space is important in determining the properties of the model in R 2, the regression coefficients, and the magnitude of the error mean squares. A large value of D i implies that the ith points is influential. A value of D i >1 would indicate that the point is influential.

6-3 Multiple Regression Checking Model Adequacy

OPTIONS NOOVP NODATE NONUMBER LS=140; DATA ex67; INPUT strength length height label strength='Pull Strength' length='Wire length' height='Die Height'; CARDS; PROC SGSCATTER data=ex67; MATRIX STRENGTH LENGTH HEIGHT; TITLE 'Scatter Plot Matrix for Wire Bond Data'; PROC REG data=ex67; MODEL strength=length height/xpx r CLB CLM CLI; TITLE 'Multiple Regression'; DATA EX67N; INPUT LENGTH HEIGHT DATALINES; DATA EX67N1; SET EX67 EX67N; PROC REG DATA=EX67N1; MODEL STRENGTH=LENGTH HEIGHT/CLM CLI; TITLE 'CIs FOR MEAN RESPONSE AND FUTURE OBSERVATION'; RUN; QUIT; Example Multiple Regression PLOT npp.*Residual.; /* Normal Probability Plot */ PLOT RESIDual.*Pred.; /* Residual Plot */ PLOT Residual.*length; PLOT Residual.*height;

6-3 Multiple Regression

The REG Procedure Model: MODEL1 Model Crossproducts X'X X'Y Y'Y Variable Label Intercept length height strength Intercept Intercept length Wire length height Die Height strength The REG Procedure Model: MODEL1 Dependent Variable: strength Number of Observations Read 25 Number of Observations Used 25 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable Label DF Estimate Error t Value Pr > |t| 95% Confidence Limits Intercept Intercept length Wire length < height Die Height Multiple Regression

Multiple Regression The REG Procedure Model: MODEL1 Dependent Variable: strength Output Statistics Dependent Predicted Std Error Std Error Student Cook's Obs Variable Value Mean Predict 95% CL Mean 95% CL Predict Residual Residual Residual D | |* | | *| | | **| | | *| | | **| | | |* | | |* | | |* | | ***| | | | | | *| | | | | | | | | | | | |***** | | | | | |**** | | **| | | *| | | | | | **| | | |* | | | | | |* | | | | Sum of Residuals 0 Sum of Squared Residuals Predicted Residual SS (PRESS) Multiple Regression

CIs FOR MEAN RESPONSE AND FUTURE OBSERVATION The REG Procedure Model: MODEL1 Dependent Variable: strength Pull Strength Number of Observations Read 27 Number of Observations Used 25 Number of Observations with Missing Values 2 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model <.0001 Error Corrected Total Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable Label DF Estimate Error t Value Pr > |t| Intercept Intercept length Wire length <.0001 height Die Height Multiple Regression

CIs FOR MEAN RESPONSE AND FUTURE OBSERVATION The REG Procedure Model: MODEL1 Dependent Variable: strength Pull Strength Output Statistics Dependent Predicted Std Error Obs Variable Value Mean Predict 95% CL Mean 95% CL Predict Residual Sum of Residuals 0 Sum of Squared Residuals Predicted Residual SS (PRESS) Multiple Regression

6-3.3 Checking Model Adequacy

6-3 Multiple Regression Checking Model Adequacy

6-3 Multiple Regression Checking Model Adequacy Multicollinearity

6-3 Multiple Regression Consider the Full Model

OPTIONS NOOVP NODATE NONUMBER LS=100; DATA appraise; INPUT price units age size parking area cond$ CARDS; F G G E G G G G G G G F E G G G F E G F G E F E PROC CORR DATA=APPRAISE; VAR PRICE UNITS AGE SIZE PARKING AREA; TITLE 'CORRELATIONS OF VARIABLES IN MODEL'; PROC REG DATA=APPRAISE; MODEL PRICE=UNITS AGE SIZE PARKING AREA/R VIF; TITLE 'ALL VARIABLES IN MODEL'; PROC REG DATA=APPRAISE; MODEL PRICE=UNITS AGE AREA/R INFLUENCE; TITLE 'REDUCED MODEL'; RUN; QUIT; Example 6-3 Multiple Regression

CORRELATIONS OF VARIABLES IN MODEL CORR 프로시저 6 Variables: price units age size parking area 단순 통계량 변수 N 평균 표준편차 합 최소값 최대값 price units age size parking area 피어슨 상관 계수, N = 24 H0: Rho=0 가정하에서 Prob > |r| price units age size parking area price < < <.0001 units < < <.0001 age size <.0001 < parking area <.0001 < Multiple Regression

ALL VARIABLES IN MODEL The REG Procedure Model: MODEL1 Dependent Variable: price Number of Observations Read 24 Number of Observations Used 24 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model E E <.0001 Error Corrected Total E12 Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variance Variable DF Estimate Error t Value Pr > |t| Inflation Intercept units age size parking area < Multiple Regression

ALL VARIABLES IN MODEL The REG Procedure Model: MODEL1 Dependent Variable: price Output Statistics Dependent Predicted Std Error Std Error Student Cook's Obs Variable Value Mean Predict Residual Residual Residual D | *| | | *| | | | | | **| | | | | | *| | | | | | |** | | | | | |*** | | ***| | | | | | |** | | *| | | |** | | | | | | | | | | | **| | | | | | | | | |**** | | *| | | |**** | Sum of Residuals 0 Sum of Squared Residuals Predicted Residual SS (PRESS) Multiple Regression

A Test for the Significance of a Group of Regressors (Partial F-Test) Suppose that the full model has k regressors, and we are interested in testing whether the last k-r of them can be deleted from the model. This smaller model is called the reduced model. That is, the full model is Then, to test the hypotheses

6-3 Multiple Regression A Test for the Significance of a Group of Regressors (Partial F-Test)

6-3 Multiple Regression

Notes on the Reduced Model

6-3 Multiple Regression Examining the Final Model

6-3 Multiple Regression One More Diagnostic

REDUCED MODEL (NO SIZE AND PARKING) The REG Procedure Model: MODEL1 Dependent Variable: price Number of Observations Read 24 Number of Observations Used 24 Analysis of Variance Sum of Mean Source DF Squares Square F Value Pr > F Model E E <.0001 Error Corrected Total E12 Root MSE R-Square Dependent Mean Adj R-Sq Coeff Var Parameter Estimates Parameter Standard Variable DF Estimate Error t Value Pr > |t| Intercept <.0001 units age area < Multiple Regression

REDUCED MODEL The REG Procedure Model: MODEL1 Dependent Variable: price Output Statistics Dependent Predicted Std Error Std Error Student Cook's Obs Variable Value Mean Predict Residual Residual Residual D | *| | | **| | | | | | | | | | | | | | | | | | | | | |** | | |* | | ***| | | |* | | |** | | *| | | |** | | | | | | | | | | | ***| | | | | | *| | | |*** | | *| | | |**** | Multiple Regression

REDUCED MODEL The REG Procedure Output Statistics Hat Diag Cov DFBETAS Obs RStudent H Ratio DFFITS Intercept units age area Multiple Regression Sum of Residuals 0 Sum of Squared Residuals Predicted Residual SS (PRESS)

6-3 Multiple Regression