Copyright © 2006 Pearson Addison-Wesley. All rights reserved. Lecture 6: Interpreting Regression Results Logarithms (Chapter 4.5) Standard Errors (Chapter 5)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-2 Agenda for Today Review Logarithms in Econometrics (Chapter 4.5) Residuals (Chapter 5.1) Estimating an Estimator’s Variance (Chapter 5.2) Confidence Intervals (Chapter 5.4)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-3 Review As a starting place for analysis, we need to write down all our assumptions about the way the underlying process works, and about how that process led to our data. These assumptions are called the “Data Generating Process.” Then we can derive estimators that have good properties for the Data Generating Process we have assumed.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-4 Review: The Gauss–Markov DGP Y =  X +  E(  i ) = 0 Var(  i ) =   2 Cov(  i,  j ) = 0, for i ≠ j X ’s fixed across samples (so we can treat them like constants). We want to estimate 

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-5 Review

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-6 Review (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-7 Review: BLUE Estimators A Strategy for Finding the Best Linear Unbiased Estimator: 1.Start with linear estimators:  w i Y i 2.Impose the unbiasedness condition  w i X i =1 3.Calculate the variance of a linear estimator: Var(  w i Y i ) =   2  w i 2 – Use calculus to find the w i that give the smallest variance subject to the unbiasedness condition Result: the BLUE Estimator for Our DGP

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-8 Review: BLUE Estimators (cont.) Ordinary Least Squares (OLS) is BLUE for our Gauss–Markov DGP. This result is called the “Gauss– Markov Theorem.”

Copyright © 2006 Pearson Addison-Wesley. All rights reserved. 6-9 Review: BLUE Estimators (cont.) OLS is a very good strategy for the Gauss–Markov DGP. OLS is unbiased: our guesses are right on average. OLS is efficient: it has a small variance (or at least the smallest possible variance for unbiased linear estimators).

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review: BLUE Estimators (cont.) Our guesses will tend to be close to right (or at least as close to right as we can get). Warning: the minimum variance could still be pretty large!

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Gauss–Markov with an Intercept

Copyright © 2006 Pearson Addison-Wesley. All rights reserved BLUE Estimator of  1 This estimator is OLS for the DGP with an intercept. It is the Best (minimum variance) Linear Unbiased Estimator for the Gauss– Markov DGP with an intercept.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve The US data from 1958–1969 suggest a trade-off between inflation and unemployment.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve (cont.) How do we interpret these numbers? If Inflation were 0, our best guess of Unemployment would be 0.06 percentage points. A one percentage point increase of Inflation decreases our predicted Unemployment level by 0.55 percentage points.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Figure 4.2 U.S. Unemployment and Inflation, 1958–1969

Copyright © 2006 Pearson Addison-Wesley. All rights reserved TABLE 4.1 The Phillips Curve

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: Phillips Curve A straight line seems not quite to capture the data. A curve would fit better. Can we use linear estimators to fit non-linear relationships?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Logarithms in Econometrics (Chapter 4.5) How can we use a linear estimator to fit a non-linear relationship? A trick: do the “nonlinear” part BEFORE plugging into OLS.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Logarithms in Econometrics (cont.) A trick: we can do something “nonlinear” BEFORE plugging into OLS. OLS doesn’t know (or care) where the Y ’s and X ’s came from. Instead of plugging X in directly, we can do stuff to it first.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Logarithms in Econometrics (cont.) We can do “stuff” to X and Y before we plug them into our OLS formula. If we do something nonlinear to Y and/or X, then our linear regression of new-nonlinear- Y on new-nonlinear- X will plot a nonlinear relationship between Y and X. What can we do to X and Y to fit a curve?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Logarithms in Econometrics (cont.) LOGARITHMS!!!!!!!!!!!! Regress How do we interpret our estimated coefficients? Our fitted value for  1 no longer tells us the effect of a 1-unit change in X on Y. It tells us the effect of a 1-unit change in log( X ) on log( Y ).

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Logarithms in Econometrics (cont.) Regress Our fitted value for  1 no longer tells us the effect of a 1-unit change in X on Y. It tells us the effect of a 1-unit change in log( X ) on log( Y ). Unit changes in log- X translate into PERCENTAGE changes in X.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Regress Unit changes in log( X ) translate into PERCENTAGE changes in X. Our estimate tells us the percentage change in Y we predict from a 1-percent change in X. Logarithms in Econometrics (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved The percentage change in Y from a 1-percent change in X has a special name in economics—elasticity! Taking logs of both Y and X lets us estimate elasticities! Logarithms in Econometrics (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve If we log both Unemployment and Inflation, then we can predict the percentage change in unemployment resulting from a one percent change in inflation. Percentage changes are nonlinear. Changing inflation from 0.01 to 0.02 is a 100% increase. Changing inflation from 0.02 to 0.03 is only a 50% increase.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Figure 4.6 A Logarithmic Phillips Curve

Copyright © 2006 Pearson Addison-Wesley. All rights reserved TABLE 4.2 A Logarithmic Phillips Curve

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve log(Unemployment) = – 0.30·log(Inflation) A 1-unit increase in log(Inflation) leads to a unit change in log(Unemployment). Huh? A 1-percent increase in Inflation leads to a -0.30% change in Unemployment. Much more natural interpretation.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve (cont.) Caution: do not confuse percentage points changes and percentage changes. If inflation is 15%, a 10 percentage point change moves inflation to 25% If inflation is 15%, a 10 percent change moves inflation to 16.5%

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (Chapter 5.1) So far, we have learned how to predict lines with an intercept and a single explanator. We have focused on making our “best guess” estimates of the intercept and slope of the line.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Once we have run our regression, we can make our best guess about what the Y value would be for an observation with some specified value of X. Our best guesses about the Y values are called “predictions.” If we have an unbiased estimator, we expect our predictions to be right on average.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Our best guesses about the Y values are called “predictions.” If we have an unbiased estimator, we expect our predictions to be right on average. For any given observation, our predictions will be WRONG.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Our “best guess” of Y i (conditioned on knowing X i ) is: On average, we expect a one-unit increase in X to increase Y by units.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Our “best guess” of Y i (conditioned on knowing X i ) is: An individual observation, though, is subject to various factors that are outside our model, such as omitted variables or simple chance.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Our “best guess” of Y i (conditioned on knowing X i ) is: Because of these other factors not in our model,

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) That our predictions are “wrong” does NOT mean that our model is wrong. After all, our model always said there was a stochastic error term in addition to the deterministic component. So far, we have only estimated the deterministic part of our model (the  ’s).

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) In our DGP, the non-deterministic component of our model is called “the error term.” We often denote it with an . In our data, the non-predicted component of an actual Y observation is called “the residual.” We often denote it with an e.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Figure 5.1 A Fitted Line and Its Residuals

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals Residuals are, in part, an accounting gimmick. They make our model “add up.” Residuals are ALSO a window into the unseen error term of the model. If our estimated   ’s are “good,” then our estimated residuals mirror the   ’s.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) We can use the residuals to tell us something about the error terms. So what? The error terms are random and vary from sample to sample. We already KNOW they average to 0. What we DON’T know is the VARIANCE of the error terms,   2.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) By convention, instead of using to denote our estimate of the   2, we use s 2 The definition of a variance is However, simply plugging e i in place of  i would yield a BIASED estimator of s 2

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Dividing by n-k-1 corrects the bias. k is the number of X variables in our regression. So far, we have learned only how to deal with the case k = 1

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Residuals (cont.) Once we have estimated s 2, we can use it to estimate the variance of our estimator. Then we can use that estimated variance to compute Confidence Intervals, which are a guide to the precision of our estimates.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (Chapter 5.2) We chose OLS as our Best Linear Unbiased Estimator because it is unbiased and efficient (for the Gauss–Markov DGP): – Unbiased: the estimate is “right” in expectation – Efficient: the estimator has the lowest possible variance Our variance is the lowest possible, but what is that variance?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.) We can calculate the variance of a linear estimator using the algebra of variances and our Gauss–Markov assumptions.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve log(Unemployment) =  0 +  1 · log(Inflation) A 1-percent increase in Inflation leads on average to a  1 % change in Unemployment. We estimate our best guess of  1.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved TABLE 4.2 A Logarithmic Phillips Curve

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve Our “best guess” for the value of each coefficient is: With the fitted coefficients, we can – Predict Y values – Make our best guess of the sign of each  – Make our best guess of the magnitude of each 

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve (cont.) Our “best guess” for the value of each coefficient is: But how good are these guesses?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve (cont.) We cannot know exactly how good our estimates are, because we do not know the true parameter values (or else we wouldn’t need to estimate them!) We know our estimates are right “on average.” To gauge how far off our individual estimator could be, we need to estimate the variance of the linear estimator.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (Chapter 5.2)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.) We can calculate  x i 2 directly. The other component of the variance of the estimator is   2, the variance of the error term.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.) We do not observe the error terms directly, so we cannot calculate their variance directly However, we can proxy for   2 using s 2

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.) In practice, we typically use not the variance of the estimator, but the standard deviation of the estimator. By convention, the standard deviation of an estimator is referred to as the “standard error.”

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Checking Understanding

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Checking Understanding (cont.) 1.If the residuals have a lower variance, e.s.e.’s will be lower. Efficient estimators have the lowest possible  e i 2 2.The higher our sample size n, the lower the e.s.e. Bigger samples are better.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Checking Understanding (cont.) 3.The fewer parameters we’re estimating, all else equal, the lower our estimated standard errors. 4.The more broadly our X variable is dispersed, the lower our standard errors. We need variation in X to get a tight estimate of its coefficient.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance The estimator for the constant has different weights, so the formula is a little different.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve What is the variance of our estimators for the log-log Phillips Curve? We need to know 3 things: – SSR = –  x i 2 = –n = 12 The SSR and n are often reported in regression output. To perform this computation by hand, you would probably need to generate the  x i 2 separately.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Example: The Phillips Curve (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved TABLE 4.2 A Logarithmic Phillips Curve

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Estimating an Estimator’s Variance Our estimate for the coefficient on log(inflation) is The standard error of the estimator is (or, using the more precise output of the computer, 0.089). We want to use the distribution of the estimator to gauge the precision of our estimate. That is, how close could we reasonably believe we’ve gotten to the true coefficient?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (Chapter 5.4) So far, we have estimated “best guesses.” Looking just at “best guesses” though can be a bit misleading. Our “best guess” could be pretty far from the truth. We can get a sense of how far off our “best guess” COULD be, by looking at ALL the “good guesses.” A “good guess” is a guess with a reasonable chance of being right.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) A “Confidence Interval” is a range of plausible parameter values. To interpret Confidence Intervals, we need to think of generating many, many samples from our Data Generating Process. For each sample, we observe a C.I. Our 95% C.I. will contain the “true” value in 95% of the samples.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) In 95% of samples, our 95% C.I. will contain the “true value.” We could also calculate 90% or 99% C.I.’s (or for any other “confidence level”) If the C.I. is “wide” (i.e. it contains many values), then it is plausible that our “best guess” estimate is far away from the true value.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) For the Phillips Curve, our “best guess” estimate of the coefficient on log(inflation) is A 1% increase in inflation leads to a predicted % change in unemployment. The 95% C.I. is (-0.496, ). We could plausibly guess that the effect was %, %, or any number in between.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) For the Phillips Curve, our “best guess” estimate of the coefficient on log(inflation) is The 95% C.I. is (-0.496, ). Was our estimate precise or not? In practice, whether or not we have “good” precision depends on how precise we need to be for our application. If we get the same policy recommendation for all values in our C.I., then we don’t care much which value is “right.”

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) In this case, we probably DO care whether the true value is closer to or to Decreasing unemployment by 0.496% for a 1% increase in inflation may be a very attractive trade-off, but decreasing it by only 0.101% may be much less attractive. Policy makers would probably desire a much more precise estimate.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) For the Phillips Curve, our “best guess” estimate of the coefficient on log(inflation) is The 95% C.I. is (-0.496, ). Our estimate is right in the middle of the C.I. We generate the C.I. by starting with our estimate, and adding a (carefully calculated) “fudge factor.”

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) The starting point for our C.I. is our estimate. Our “best guess” must be a “plausible guess.” We then add or subtract a (carefully chosen) multiplier times our estimated standard error. The smaller our e.s.e., the tighter our C.I. Where does the multiplier come from?

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) Our C.I. comes from the distribution of the estimator. The estimator follows the “ t ” distribution. The shape of the t distribution depends on n-k-1. When n-k-1 is very large, the t distribution is approximately the same as the normal distribution.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Figure SA.11 The Distribution of the t- Statistic Given the Null Hypothesis is True

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) When n-k-1 is large:

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Confidence Intervals (cont.) For our Phillips Curve example, n-k-1 = 10. The multiplier for the 95% level is (As n-k-1 gets larger, our multiplier gets smaller, and our C.I. gets tighter.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved We can perform functions on our variables BEFORE we plug them into OLS. Unit changes in log( X ) translate into PERCENTAGE changes in X. Our estimate tells us the percentage change in Y we predict from a 1-percent change in X. Review

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.) Our “best guess” of Y i (conditioned on knowing X i ) is An individual observation, though, is subject to various factors that are outside our model, such as omitted variables or simple chance.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.) In our DGP, the non-deterministic component of our model is called “the error term.” We often denote it with an . In our data, the non-predicted component of an actual Y observation is called “the residual.” We often denote it with an e.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.) In practice, we typically use not the variance of the estimator, but the standard deviation of the estimator. By convention, the standard deviation of an estimator is referred to as the “standard error.”

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.)

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.) A “Confidence Interval” is a range of plausible parameter values. To interpret Confidence Intervals, we need to think of generating many, many samples from our Data Generating Process. For each sample, we observe a C.I. Our 95% C.I. will contain the “true” value in 95% of the samples.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.) In 95% of samples, our 95% Confidence Interval will contain the “true value.” We could also calculate 90% or 99% C.I.’s (or for any other “confidence level”) If the C.I. is “wide” (i.e. it contains many values), then it is plausible that our “best guess” estimate is far away from the true value.

Copyright © 2006 Pearson Addison-Wesley. All rights reserved Review (cont.)