LESSON 5 Multiple Regression Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-1

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-2 Multiple Regression We know how to regress Y on a constant and a single X variable  1 is the change in Y from a 1-unit change in X

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-3 Multiple Regression (cont.) Usually we will want to include more than one independent variable. How can we extend our procedures to permit multiple X variables?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-4 Gauss–Markov DGP with Multiple X ’s

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-5 BLUE Estimators Ordinary Least Squares is still BLUE The OLS formula for multiple X ’s requires matrix algebra, but is very similar to the formula for a single X

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-6 BLUE Estimators (cont.) Intuitions from the single variable formulas tend to generalize to multiple variables. We’ll trust the computer to get the formulas right. Let’s focus on interpretation.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-7 Single Variable Regression

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-8 Multiple Regression  1 is the change in Y from a 1-unit change in X 1

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 7-9 Multiple Regression (cont.) How can we interpret  1 now?  1 is the change in Y from a 1-unit change in X 1, holding X 2 …X k FIXED

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Multiple Regression (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Multiple Regression (cont.) How do we implement multiple regression with our software?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth Regress GDP growth from 1960–1985 on – GDP per capita in 1960 (GDP60) – Primary school enrollment in 1960 (PRIM60) – Secondary school enrollment in 1960 (SEC60) – Government spending as a share of GDP (G/Y) – Number of coups per year (REV) – Number of assassinations per year (ASSASSIN) – Measure of Investment Price Distortions (PPI60DEV)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Hit Table Ext.1.1 A Multiple Regression Model of per Capita GDP Growth.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth (cont.) A 1-unit increase in GDP in 1960 predicts a unit decrease in GDP growth, holding fixed the level of PRIM60, SEC60, G/Y, REV, ASSASSIN, and PPI60DEV.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth (cont.) Before we controlled for other variables, we found a POSITIVE relationship between growth and GDP per capita in After controlling for measures of human capital and political stability, the relationship is negative, in accordance with “catch up” theory.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth (cont.) Countries with high values of GDP per capita in 1960 ALSO had high values of schooling and a low number of coups/assassinations. Part of the relationship between growth and GDP per capita is actually reflecting the influence of schooling and political stability. Holding those other variables constant lets us isolate the effect of just GDP per capita.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth The Growth of GDP from 1960–1985 was higher: 1.The lower starting GDP, and 2.The higher the initial level of human capital. Poor countries tended to “catch up” to richer countries as long as the poor country began with a comparable level of human capital, but not otherwise.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth (cont.) Bigger government consumption is correlated with lower growth; bigger government investment is only weakly correlated with growth. Politically unstable countries tended to have weaker growth. Price distortions are negatively related to growth.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Growth (cont.) The analysis leaves largely unexplained the very slow growth of Sub-Saharan African countries and Latin American countries.

20 Omitted Variable Bias

21 Omitted variable bias, ctd.

22 Omitted variable bias, ctd.

23 The omitted variable bias formula:

24 Measures of Fit for Multiple Regression

25 S e and RMSE

26 R 2 and

27 R 2 and, ctd.

28 Measures of fit, ctd.

29 The Least Squares Assumptions for Multiple Regression

Testing Hypotheses Copyright © 2006 Pearson Addison-Wesley. All rights reserved t- test – individual test F-test – joint test

Dummy Variables Used to capture qualitative explanatory variables Used to capture any event that has only two possible outcomes e.g. race, gender, geographic region of residence etc.

Use of Intercept Dummy Most common use of dummy variables. Modifies the regression model intercept parameter e.g. Let test the “location”, “location” “location” model of real estate Suppose we take into account location near say a university or golf course

P t = β o + β 1 S t +β 2 D t + ε t St = square footage D = dummy variable to represent if the characteristic is present or not D = 1if property is in a desirable neighborhood 0if not in a desirable neighborhood

Effect of the dummy variable is best seen by examining the E(Pt). If model is specified correctly, E(ε t ) = 0 E(P t ) = ( β o + β 2 ) + β 1 S t when D=1 β o + β 1 S t when D = 0

B2 is the location premium in this case. It is the difference between the Price of a house in a desirable are and one in a not so desirable area, all things held constant The dummy variable is to capture the shift in the intercept as a result of some qualitative variable  Dt is an intercept dummy variable

Dt is treated as any explanatory variable. You can construct a confidence interval for B2 You can test if B2 is significantly different from zero. In such a test, if you accept Ho, then there is no difference between the two categories.

Application of Intercept Dummy Variable Wages = B0 + B1EXP + B2RACE +B3SEX + Et Race = 1 if white 0 if non white Sex = 1 if male 0 if female

WAGES = 40, EXP RACE +1082SEX Mean salary for black female 40, EXP Mean salary for white female 41, EXP +1102

Mean salary for Asian male Mean salary for white male What sucks more, being female or non white?

Determining the # of dummies to use If h categories, then use h-1 dummies Category left out defines reference group If you use h dummies you’d fall into the dummy trap

Slope Dummy Variables Allows for different slope in the relationship Use an interaction variable between the actual variable and a dummy variable e.g. Pt = Bo + B1Sqfootage+B2(Sqfootage*D)+et D= 1 desirable area, 0 otherwise

Captures the effect of location and size on the price of a house E(Pt) = B0 + (B1+B2)Sqfoot if D=1 = BO + B1Sqfoot if D = 0  in the desirable area, price per square foot is b1+b2, and it is b1 in other areas If we believe that a house location affects both the intercept and the slope then the model is

Pt = B0 +B1sqfoot +B2(sqfoot*D) + B3D +et

44 Dummies for Multiple Categories We can use dummy variables to control for something with multiple categories Suppose everyone in your data is either a HS dropout, HS grad only, or college grad To compare HS and college grads to HS dropouts, include 2 dummy variables hsgrad = 1 if HS grad only, 0 otherwise; and colgrad = 1 if college grad, 0 otherwise

45 Multiple Categories (cont) Any categorical variable can be turned into a set of dummy variables Because the base group is represented by the intercept, if there are n categories there should be n – 1 dummy variables If there are a lot of categories, it may make sense to group some together Example: top 10 ranking, 11 – 25, etc.

46 Interactions Among Dummies Interacting dummy variables is like subdividing the group Example: have dummies for male, as well as hsgrad and colgrad Add male*hsgrad and male*colgrad, for a total of 5 dummy variables –> 6 categories Base group is female HS dropouts hsgrad is for female HS grads, colgrad is for female college grads The interactions reflect male HS grads and male college grads

47 More on Dummy Interactions Formally, the model is y =  0 +  1 male +  2 hsgrad +  3 colgrad +  4 male*hsgrad +  5 male*colgrad +  1 x + u, then, for example: If male = 0 and hsgrad = 0 and colgrad = 0 y =  0 +  1 x + u If male = 0 and hsgrad = 1 and colgrad = 0 y =  0 +  2 hsgrad +  1 x + u If male = 1 and hsgrad = 0 and colgrad = 1 y =  0 +  1 male +  3 colgrad +  5 male*colgrad +  1 x + u

48 Other Interactions with Dummies Can also consider interacting a dummy variable, d, with a continuous variable, x y =  0 +  1 d +  1 x +  2 d*x + u If d = 0, then y =  0 +  1 x + u If d = 1, then y = (  0 +  1 ) + (  1 +  2 ) x + u This is interpreted as a change in the slope

49 y x y =  0 +  1 x y = (  0 +  0 ) + (  1 +  1 ) x Example of  0 > 0 and  1 < 0 d = 1 d = 0