1. 1 G89.2229 Lect 3M Regression line review Estimating regression coefficients from moments Marginal variance Two predictors: Example 1 Multiple regression as a method for statistical adjustment Two predictors: Example 2 G89.2229 Multiple Regression Week 3 (Monday)

2. 2 G89.2229 Lect 3M Linear Regression Approximate E(Y|X) with linear model »E(Y|X) = b 0 + b 1 X »Y = b 0 + b 1 X + e We choose values of b 0 and b 1 that minimize variance of e »Ordinary Least Squares (OLS) Y = -.254 +.391X + e

3. 3 G89.2229 Lect 3M Example revisited (estimates from Excel) Data are consistent with intercept (b 0 ) of zero. Data are consistent with population slope in range (.28,.51).

4. 4 G89.2229 Lect 3M Regression Estimates from Sample Moments OLS estimates always satisfy the following relations Let S X, S Y, and S XY be the sample standard deviations of X and Y, and the covariance »The estimated standard error of the slope b 1 is given by

5. 5 G89.2229 Lect 3M Marginal from Conditional Expectations Suppose that 60% of the NYU undergrads were female (X=1) and 40% were male (X=0). Suppose E(Y|X=1)=5’5” and E(Y|X=0)=5’9” What is E(Y)? »E(Y|X=1)P(X=1)+E(Y|X=0)P(X=0) =5’5”(.60)+5’9”(.40) =5’6.6” »E(Y)=E[E(Y|X)] where the expectation outside the brackets is over the distribution of X and the Expectation inside is over the distribution of Y (for each given X)

6. 6 G89.2229 Lect 3M Combining Conditional Variances Suppose we have Var(Y|X=1) and Var(Y|X=0) but want the variance of the marginal distribution of Y, Var(Y). It can be shown that Var(Y)= Var[E(Y|X)] + E[Var(Y|X)] »In the right, the expectation operators outside the brackets are for values of X, and inside they are for values of Y. From Excel example, we get estimates »Var(Dep)=.42 »Var[E(Y|X)] =.17 variance of fit »E[Var(Y|X)]=.25 variance of residual

7. 7 G89.2229 Lect 3M Details of Numerical Example The variance of the depressed mood is partitioned into fitted variance and residual variance »0.42 = 0.17 + 0.25 »R 2 =.17/.42 =.405

8. 8 G89.2229 Lect 3M Regression with two predictors Suppose we predict Depressed mood on day 29 as a function of »Depressed mood on day 28, »Anxious mood on day 28 The regression equation: Y = b 0 + b 1 X 1 + b 2 X 2 + e This is a simple extension of bivariate regression »Approximation of E(Y|X 1 X 2 ) »Choose values of b 0, b 1, and b 2 that minimize the squared residuals. The interpretation can be more complicated.

9. 9 G89.2229 Lect 3M Thinking about E(Y|X 1 X 2 ) What is the expected value of the outcome (Y) for pairs of values of X 1 and X 2 ? »The joint distribution of X 1 and X 2 does not affect the form of the regression equation » X 1 and X 2 can be observed in nature, or they can be manipulated The coefficient b 1 is represents the expected effect of X 1 when X 2 is taken into account »"Adjusting for X 2 " »"Controlling for X 2 " »"Conditional on X 2 "

10. 10 G89.2229 Lect 3M The geometry of E(Y|X 1 X 2 ) = b 0 + b 1 X 1 + b 2 X 2 E(Y|X 1 X 2 )= b 0 + b 1 X 1 + b 2 X 2 describes a plane. »For every change in either X 1 or X 2, E(Y|X 1 X 2 ) changes linearly The distribution of X 1 and X 2 does not affect the form of the plane itself, but it does affect the estimates of b 0, b 1, and b 2. MOST IMPORTANTLY, The distribution of X 1 and X 2 has a profound effect on how the semipartial effects, b 0, b 1, and b 2 relate to bivariate correlations.

11. 11 G89.2229 Lect 3M When X 1 and X 2 are uncorrelated In experiments, we manipulate X 1 and X 2 so that they are balanced, and uncorrelated »X 1 and X 2 are said to be "orthogonal" When X 1 and X 2 are orthogonal the semipartial effects are identical to the bivariate effects of X 1 or X 2 on Y X1X1 X2X2

12. 12 G89.2229 Lect 3M When X 1 and X 2 are correlated In surveys we usually observe X 1 and X 2 without adjusting their relative frequency »As an exception, sometimes we oversample certain rare values When X 1 and X 2 are correlated the semipartial effects are different from the bivariate effects of X 1 or X 2 on Y X2X2 X1X1