1 Multiple Regression and Correlation KNN Ch. 6 CC Ch 3

2  An Extension of Simple Linear Regression  Interpretation of parameters is important: For example, how would you interpret  1 in the above model?  Can be expressed in short form as,  The geometric interpretation is a Response Surface.

3 Meaning of Multiple Reg. Coefficients Meaning of the y-intercept  0 : –If the scope of the model includes X 1 =0, X 2 =0, etc. then  0 is the mean response E{Y} at X 1 =0, X 2 =0, etc. Otherwise, the y- intercept has no particular meaning Meaning of the slope  1 : –Indicates the change in the mean response E{Y} (expected change in Y) per unit increase in X 1, when X 2 and all the other predictors are held constant.

4 Model Types (What is linearity ?) Polynomial n Qualitative Variables n Non-linear ? Is this allowed ?

5 The Matrix Representation

6 Formulae for Simple Regression Apply Quadratic Forms ! Each of the “A” matrices are symmetric. H is “Idempotent”. It’s the “Hat” Matrix

7 A Simple Example

8  All tests and diagnostics similar to simple regression  F-test for regression  R 2 and Adjusted R 2  Estimation of Mean Response and Prediction of New Observation  Simultaneous CIs for Several Mean Responses - Working-Hotelling or Bonferroni (See page 234)  Prediction of Mean of “m” new observations at X h  Prediction of “g” new observations - Scheffe´ or Bonferroni (See page 235) Tests, Estimation and Diagnostics

9  3-D scatter plots  Residual Plots  Correlation test for Normality  Brown-Forsythe (Modified Levine test for heteroscedasticity  Breusch-Pagan test for heteroscedasticity  F-test for lack of fit  Finally, the Box-Cox procedure as a remedial measure

10 Hidden Extrapolations Caution should be exercised for the prediction not to fall outside of the scope of the model (observed range of the predictor variables X i ). The point shown below is within the ranges of X 1 and X 2 individually, but is well outside the joint region of observations. What to do? Wait until we get to Leverage values (KNN ch. 10) X1X1 X2X2 Region covered by X 1 and X 2 jointly Individual X 1 range Individual X 2 range 0

11 A Different Perspective (optional)  A Bivariate MR model with standardized variables Where, the  s are standardized partial regression coefficients and are given as, Note that,  1 =  Y1.2 * s Y /s X1 and  2 =  Y2.1 * s Y /s X2 The term “partial” above is used because the terms have been adjusted to allow for the correlation between independent variables. (Check by substituting r 12 =0)

12 A Different Perspective (optional)  The Coefficient of Multiple Determination  Semi-partial Correlation Coefficients and Venn Diagrams  Partial Correlation Coefficients and Venn Diagrams.  Separating direct, indirect, spurious and entirely indirect effects