1. Multiple Regression I KNNL – Chapter 6

2. Models with Multiple Predictors Most Practical Problems have more than one potential predictor variable Goal is to determine effects (if any) of each predictor, controlling for others Can include polynomial terms to allow for nonlinear relations Can include product terms to allow for interactions when effect of one variable depends on level of another variable Can include “dummy” variables for categorical predictors

3. First-Order Model with 2 Numeric Predictors

4. Interpretation of Regression Coefficients Additive: E{Y} =  0  1 X 1 +  2 X 2 ≡ Mean of Y @ X 1, X 2  0 ≡ Intercept, Mean of Y when X 1 =X 2 =0  1 ≡ Slope with Respect to X 1 (effect of increasing X 1 by 1 unit, while holding X 2 constant)  2 ≡ Slope with Respect to X 2 (effect of increasing X 2 by 1 unit, while holding X 1 constant) These can also be obtained by taking the partial derivatives of E{Y} with respect to X 1 and X 2, respectively Interaction Model: E{Y} =  0  1 X 1 +  2 X 2 +  3 X 1 X 2 When X 2 = 0: Effect of increasing X 1 by 1:  1 (1)+  3 (1)(0)=  1 When X 2 = 1: Effect of increasing X 1 by 1:  1 (1)+  3 (1)(1)=  1 +  3 The effect of increasing X 1 depends on level of X 2, and vice versa

5. General Linear Regression Model

6. Special Types of Variables/Models - I p-1 distinct numeric predictors (attributes)  Y = Sales, X 1 =Advertising, X 2 =Price Categorical Predictors – Indicator (Dummy) variables, representing m-1 levels of a m level categorical variable  Y = Salary, X 1 =Experience, X 2 =1 if College Grad, 0 if Not Polynomial Terms – Allow for bends in the Regression  Y=MPG, X 1 =Speed, X 2 =Speed 2 Transformed Variables – Transformed Y variable to achieve linearity Y’=ln(Y) Y’=1/Y

7. Special Types of Variables/Models - II Interaction Effects – Effect of one predictor depends on levels of other predictors  Y = Salary, X 1 =Experience, X 2 =1 if Coll Grad, 0 if Not, X 3 =X 1 X 2  E(Y) =     X 1   X 2   X 1 X 2  Non-College Grads (X 2 =0):  E(Y) =     X 1   (0)   X 1 (0)=     X 1  College Grads (X 2 =1):  E(Y) =     X 1   (1)   X 1 (1)=          X 1 Response Surface Models  E(Y) =     X 1   X 1 2   X 2   X 2 2   X 1 X 2 Note: Although the Response Surface Model has polynomial terms, it is linear wrt Regression parameters

8. Matrix Form of Regression Model

9. Least Squares Estimation of Regression Coefficients

10. Fitted Values and Residuals

11. Analysis of Variance – Sums of Squares

12. ANOVA Table, F-test, and R 2

13. Inferences Regarding Regression Parameters

14. Estimating Mean Response at Specific X-levels

15. Predicting New Response(s) at Specific X-levels