MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE 1 This sequence provides a geometrical interpretation of a multiple regression model with two explanatory variables. EARNINGS =  1 +  2 S +  3 EXP + u S 11 EARNINGS EXP

2 Specifically, we will look at an earnings function model where hourly earnings, EARNINGS, depend on years of schooling (highest grade completed), S, and years of work experience, EXP. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u S 11 EARNINGS EXP

3 The model has three dimensions, one each for EARNINGS, S, and EXP. The starting point for investigating the determination of EARNINGS is the intercept,  1. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u S 11 EARNINGS EXP

4 Literally the intercept gives EARNINGS for those respondents who have no schooling and no work experience. However, there were no respondents with less than 6 years of schooling. Hence a literal interpretation of  1 would be unwise. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u S 11 EARNINGS EXP

pure S effect 5 S 11 EARNINGS  1 +  2 S MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u The next term on the right side of the equation gives the effect of variations in S. A one year increase in S causes EARNINGS to increase by  2 dollars, holding EXP constant. EXP

pure EXP effect 6 S 11  1 +  3 EXP EARNINGS EXP MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u Similarly, the third term gives the effect of variations in EXP. A one year increase in EXP causes earnings to increase by  3 dollars, holding S constant.

pure EXP effect 7 S 11  1 +  3 EXP  1 +  2 S +  3 EXP EARNINGS EXP  1 +  2 S combined effect of S and EXP MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u Different combinations of S and EXP give rise to values of EARNINGS which lie on the plane shown in the diagram, defined by the equation EARNINGS =  1 +  2 S +  3 EXP. This is the nonstochastic (nonrandom) component of the model. pure S effect

8 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u The final element of the model is the disturbance term, u. This causes the actual values of EARNINGS to deviate from the plane. In this observation, u happens to have a positive value. pure EXP effect S 11  1 +  3 EXP EARNINGS EXP u  1 +  2 S pure S effect  1 +  2 S +  3 EXP combined effect of S and EXP  1 +  2 S +  3 EXP + u

9 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u A sample consists of a number of observations generated in this way. Note that the interpretation of the model does not depend on whether S and EXP are correlated or not. pure EXP effect S 11  1 +  3 EXP EARNINGS EXP u  1 +  2 S pure S effect  1 +  2 S +  3 EXP combined effect of S and EXP  1 +  2 S +  3 EXP + u

pure EXP effect 10 S 11  1 +  3 EXP  1 +  2 S +  3 EXP + u EARNINGS EXP u However we do assume that the effects of S and EXP on EARNINGS are additive. The impact of a difference in S on EARNINGS is not affected by the value of EXP, or vice versa. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE EARNINGS =  1 +  2 S +  3 EXP + u  1 +  2 S pure S effect  1 +  2 S +  3 EXP combined effect of S and EXP

The regression coefficients are derived using the same least squares principle used in simple regression analysis. The fitted value of Y in observation i depends on our choice of b 1, b 2, and b MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

The residual e i in observation i is the difference between the actual and fitted values of Y. 12 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

We define RSS, the sum of the squares of the residuals, and choose b 1, b 2, and b 3 so as to minimize it. 13 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

First, we expand RSS as shown. 14 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

15 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Then we use the first order conditions for minimizing it. Fitted modelTrue model

We thus obtain three equations in three unknowns. Solving for b 1, b 2, and b 3, we obtain the expressions shown above. (The expression for b 3 is the same as that for b 2, with the subscripts 2 and 3 interchanged everywhere.) 16 MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

17 The expression for b 1 is a straightforward extension of the expression for it in simple regression analysis. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

18 However, the expressions for the slope coefficients are considerably more complex than that for the slope coefficient in simple regression analysis. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

19 For the general case when there are many explanatory variables, ordinary algebra is inadequate. It is necessary to switch to matrix algebra. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE Fitted modelTrue model

. reg EARNINGS S EXP Source | SS df MS Number of obs = F( 2, 537) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | _cons | Here is the regression output for the earnings function using Data Set MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE

21 It indicates that earnings increase by $2.68 for every extra year of schooling and by $0.56 for every extra year of work experience. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE. reg EARNINGS S EXP Source | SS df MS Number of obs = F( 2, 537) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | _cons |

22 Literally, the intercept indicates that an individual who had no schooling or work experience would have hourly earnings of –$ MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE. reg EARNINGS S EXP Source | SS df MS Number of obs = F( 2, 537) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | _cons |

23 Obviously, this is impossible. The lowest value of S in the sample was 6. We have obtained a nonsense estimate because we have extrapolated too far from the data range. MULTIPLE REGRESSION WITH TWO EXPLANATORY VARIABLES: EXAMPLE. reg EARNINGS S EXP Source | SS df MS Number of obs = F( 2, 537) = Model | Prob > F = Residual | R-squared = Adj R-squared = Total | Root MSE = EARNINGS | Coef. Std. Err. t P>|t| [95% Conf. Interval] S | EXP | _cons |

Copyright Christopher Dougherty These slideshows may be downloaded by anyone, anywhere for personal use. Subject to respect for copyright and, where appropriate, attribution, they may be used as a resource for teaching an econometrics course. There is no need to refer to the author. The content of this slideshow comes from Section 3.2 of C. Dougherty, Introduction to Econometrics, fourth edition 2011, Oxford University Press. Additional (free) resources for both students and instructors may be downloaded from the OUP Online Resource Centre Individuals studying econometrics on their own who feel that they might benefit from participation in a formal course should consider the London School of Economics summer school course EC212 Introduction to Econometrics or the University of London International Programmes distance learning course EC2020 Elements of Econometrics