1. 1 MF-852 Financial Econometrics Lecture 6 Linear Regression I Roy J. Epstein Fall 2003

2. 2 The Two-Variable Linear Model Y: dependent variable of interest May be complicated, e.g., health care expenditure X: an explanatory variable that “causes” Y. E.g., income. How to quantify effect of X on Y?

3. 3 A Linear Regression Most simple model is linear relationship: Y i =  +  X i + e i Linear model is always an approximation. What are  and  ? Why is there an error term e i ? e i is also called the “residual”

4. 4 Linear Regression: Purpose Two goals: Make useful prediction of Y, given X. Get estimate of .

5. 5 The Error Term The error term should be pure “noise” that is not useful for predicting Y. What are desirable properties for e i ?

6. 6 Error Term: First Property E(e i ) = 0, on average error does not predict a value for Y. Means that E(Y) =  +  X i, so the regression prediction is unbiased.

7. 7 Error Term: Second Property var(e i ) =  2 Each Y observation has same variance. Means that all observations equally informative. Needed for accurate calculation of standard errors of  and .

8. 8 Error Term: Third Property Cov(X, e i ) = 0 Needed for accurate estimate of . Otherwise effect of e on Y would be attributed to X.

9. 9 Error Term: Fourth Property Covar(e i, e j ) = 0 Error for one observation independent of error for another observation. Needed for accurate calculation of standard errors of  and .

10. 10 Ordinary Least Squares (OLS) Regression Rule: minimize variance of the prediction error e i = Y i – (  +  X i ). If error has zero variance, then prediction would be perfect! OLS estimation: find  and  to make as small as possible.

11. 11 Ordinary Least Squares (OLS) Treat model as a prediction of Y. Rule: minimize variance of the prediction error e i = Y i – (  +  X i ). If error has zero variance, then prediction would be perfect! OLS estimation: find  and  to make as small as possible.

12. 12 Data and the Regression Line

13. 13 Actual and Fitted Relationships What are the data points? What is the regression line? What are the error terms?

14. 14 The Estimated Residuals

15. 15 Regressions in Excel Make sure you have installed the Data Analysis Tool Pack! You need this for Excel to do regressions automatically.

16. 16 OLS Regression Coefficients The estimated coefficients are random variables. In this example,  = – 0.173, standard error = 1.32  = 0.144, standard error = 0.0094

17. 17 Statistical Significance Suppose H 0 :  = 0 Is the estimated  statistically significant? Suppose H 0 :  = 0 Is the estimated  statistically significant? Suppose H 0 :  = 0 AND  = 0 Is the joint hypothesis accepted or rejected?

18. 18 More Hypothesis Tests Suppose H 0 :  = 0.16 Do you accept or reject H 0 ? Suppose H 0 :  = 2 Do you accept or reject H 0 ?