1. 6. Simple Regression and OLS Estimation
Chapter 6 will expand on concepts introduced in Chapter 5 to cover the following:
Estimating parameters using Ordinary Least Squares (OLS) Estimation
Hypothesis tests of OLS coefficients
Confidence intervals of OLS coefficients
Excel Regressions 6

2. 6. Regression & OLS 6.1 The OLS Estimator and its Properties
6.2 OLS Estimators and Goodness of Fit
6.3 Confidence Intervals for Simple Regression Models
6.4 Hypothesis Testing in a Simple Regression Context
6.6 Examples of Simple Regression Models
6.7 Conclusion

3. 6.1 The OLS Estimator and its Properties
We’ve seen the true economic relationship:
Yi = β1 + β2Xi + єi
Where єi and therefore Yi are random and the other terms are non-random
When this relationship is unknown, we’ve seen how to estimate the relationship
Using:

4. 6.1 Properties of the OLS Estimator
There exist a variety of methods to estimate the coefficients of our model (β1 and β2)
Why use Ordinary Least Squares (OLS)?
OLS minimizes the sum of squared errors, creating a line that fits best with the observations
With certain assumptions, OLS exhibits beneficial statistical properties. In particular, OLS is BLUE.

5. 6.1 The OLS Estimator
These OLS estimates create a straight line going through the “middle” of the estimates:

6. 6.1.1 Fitted or Predicted Values
From the above we see that often the actual data points lie above or below the estimated line.
Points on the line give us ESTIMATED y values for each given x.
The predicted or fitted y values are found using our x data and our estimated β’s:

7. 6.1.1 Estimators Example Price 4 3 6 Pbar = 4 Quantity 10 15 20
Qbar=15
Ols Estimation (From Chapter 5)

8. 6.1.1 Estimating Errors or Residuals
The estimated y values (yhat) are rarely equal to their actual values (y).
The difference is the estimated error term:
Since we are indifferent whether our estimates are above or below the actual, we can square these estimated errors. A higher squared error means an estimate farther from the actual

9. 6.1.1 Estimators Example Price 4 3 6 Pbar = 4 Quantity 10 15 20
Qbar=15

10. 6.1.2 Statistical Properties of OLS
In our model:
Y, the dependent variable, is made up of two components:
β1 + β 2Xi – a non-random component that indicates the effect of X on Y. In this course, X is non-random.
Єi – a random error term representing other influences on Y.

11. 6.1.2 Statistical Properties of OLS
Error Assumptions:
E(єi) = 0; we expect no error; we assume the model is complete
Var(єi) = σ2; the error term has a constant variance
Cov(єi, єj) = 0; error terms from two different observations are uncorrelated. If the last error was positive, the next error need not be negative.

12. 6.1.2 Statistical Properties of OLS
OLS Estimators are Random Variables:
Y depends on є and is thus random.
β1hat and β2hat depend on Y…
Therefore they are random
All random variables have probability distributions, expected values, and variances
These characteristics give rise to certain OLS estimator properties.

13. 6.1.2 OLS is BLUE B est L inear U nbiased E stimator
We use Ordinary Least Squares estimation because, given certain assumptions, it is BLUE:
B est
L inear
U nbiased
E stimator

14. An estimator is unbiased if it expects the true value: E(dhat) = d
β2hat = ∑(Xi-Xbar)(Yi-Ybar)
∑(Xi-Xbar)2
β2hat = ∑(Xi-Xbar)(Yi)
∑(Xi-Xbar)Xi
By a mathematical property.

15. 6.1.2 U nbiased β2hat = ∑(Xi-Xbar)(Yi) -------------------
∑(Xi-Xbar)Xi
E(β2hat) = ∑(Xi-Xbar)E(Yi)
Since only Yi is variable.

16. 6.1.2 U nbiased E(β2hat) = ∑(Xi-Xbar)E(β1 + β2Xi + єi)
∑(Xi-Xbar)Xi
Since Yi = β1 + β2Xi + єi
E(β2hat) = ∑(Xi-Xbar)(β1 + β2Xi + 0)
Since β1, β2, and Xi are non-random and E(єi)=0.

17. 6.1.2 U nbiased
E(β2hat) = β1∑(Xi-Xbar) + β2∑(Xi-Xbar) Xi
∑(Xi-Xbar)Xi
By simple algebra.
E(β2hat) = β1∑(Xi-Xbar) + β2∑(Xi-Xbar) Xi
∑(Xi-Xbar)Xi ∑(Xi-Xbar)Xi
Since there exists a common denominator.

18. 6.1.2 U nbiased E(β2hat) = β1(0) + β2 --------------- ∑(Xi-Xbar)Xi
Since the sum of the difference between an observation and its mean is zero, by definition,
E(β2hat) = β2
= β2
The proof that E(β1hat)= β1 is similar.

19. 6.1.2 U nbiased
E(β2hat) = β2
This means that on average, OLS estimation will estimate the correct coefficients.
Definition: If the expected value of an estimator is equal to the parameter that it is being used to estimate, the estimator is unbiased.

20. 6.1.2 L inear
The OLS estimators are linear in the dependent variable (Y):
-Y’s are never raised to a power other than 1
-no non-linear operations are performed on the Y’s
Note: Since X’s are squared in the β1hat and β2hat formulae, OLS is not linear in the X’s (which doesn’t matter for BLUE)

21. 6.1.2 B est
Of all linear unbiased estimators, OLS has the smallest variance.
-there is a greater likelihood of obtaining an estimate close to the actual parameter
Large variance => High probability of obtaining an estimate far from the center
Small variance => Low probability of obtaining an estimate far from the center

22. 6.1.2 E stimator
By definition, the OLS estimator is an estimator; it estimates values for β1 and β 2.

23. 6.1.2 Normality of Y
In order to conduct hypothesis tests and construct confidence intervals from OLS, we need to know the exact distributions of β1hat and β2hat (Otherwise, we can’t use statistical tables..)
We will see that if
The error term is normally distributed
Then
2) Y is normally distributed
3) β1hat and β2hat are normally distributed 6

24. 6.1.2 Normality of Y So far, we have assumed:
The error term, єi, is random with
E(єi)=0; no expected error
Var(єi)=σ2; constant variance
Cov(єi, єj)=0; no covariance between errors
Now we add the assumption that the error term is normally distributed. Therefore:
iid
Єi ~ N(0,σ2)
(iid means identically and independently distributed) 6

25. (Y is normally distributed with mean β1+ β2Xi and variance σ2.)
6.1.2 Normality of Y
If the error is normally distributed, so will be the Y term (since the randomness of Y depends on the randomness of the error term). Therefore:
E(Yi) = E(β1+ β2Xi+єi)= β1+ β2Xi
Var(Yi) = Var(β1+ β2Xi+єi)=Var(єi) = σ2
(Given all our previous assumptions.)
Therefore:
Yi ~ N(β1+ β2Xi, σ2)
(Y is normally distributed with mean β1+ β2Xi and variance σ2.) 6

26. 6.1.2 Normality of OLS
Since β1hat and β2hat are linear functions of Y: 6

27. 6.1.2 Normality of OLS
If we know σ, we can construct standard normal variables (z=(x-μ)/σ): 6

28. 6.1.2 Normality of OLS Since we don’t know σ2, we can estimate it:
This gives us estimates of the variance of our coefficients: 6

29. 6.1.2 Normality of OLS
The square root of the estimated variance is referred to as the standard error (se) (as opposed to standard deviation)
Using our assumptions:
(β1hat- β1)/se(β1hat) has a t distribution with N-2 degrees of freedom
(β2hat- β2)/se(β2hat) has a t distribution with N-2 degrees of freedom 6

30. 6.2 OLS Estimators and Goodness of Fit
On average, OLS works well:
The average of the estimated errors is zero
The average of the estimated Y’s is always the average of the observed Y’s
Note: The proof of the above is covered in Appendix A. 6

31. 6.2 Measuring Goodness of Fit
These conditions hold regardless of the quality of the model.
Ie: You could estimate average grades as a function of locust population in Mexico. OLS would be a good estimator even though the model is useless.
“Goodness of Fit” measures how well the economic model fits the data.
R2 is the most common measure of goodness of fit.
R2 CANNOT be compared across models. 6

32. 6.2 Measuring Goodness of Fit
R2 is constructed by dividing the variation of Y into two parts:
Variation in fitted Yhat terms. This is explained by the model
Variation in the estimated errors. This is NOT explained by the model. 6

33. 6.2 Measuring Goodness of Fit
R2 is the proportion of variation explained by the model. It is expressed as:
The ratio of explained variation to total variation in Y Or
b) 1 minus the ratio of unexplained variation to total variation in Y
0<R2<1
R2=0; model has no explanatory power
R2=1; model completely explains variations in Y (and generally that means you did something wrong) 6

34. 6.3 Confidence Intervals for Simple Economic Models
As covered previously, ordinary least squares estimation derives POINT ESTIMATES for our coefficients (β1 and β2).
-These are unlikely to be perfectly accurate.
Alternately, Confidence Intervals provide for us an estimate of a range for our coefficients.
-We are reasonably certain that our value lies within that range. 6

35. 6.3.1 Deriving a Confidence Interval
Step 1: Recall Distribution
We know that:
(β1hat-β1)/se(β1hat) has a t distribution with N-2 degrees of freedom
(β2hat- β2)/se(β2hat) has a t distribution with N-2 degrees of freedom 6

36. 6.3.1 Deriving a Confidence Interval
Step 2: Establish Probability:
Using t-tables with N-2 degrees of freedom, we find t* such that:
P(-t*<t<t*)=1-α
Note that ±t* cuts off α/2 of each tail.
Ie: if N=25 and α=0.10, t*=1.71 6

37. 6.3.1 Deriving a Confidence Interval
Step 3: Combine
Steps 1 and 2 combine to give us:
(1-α)% t
-t* t* 6

38. 6.3.1 Deriving a Confidence Interval
Step 4: Rearrange for CI: OR
By repeatedly calculating Confidence Intervals using OLS, 100(1- α)% of these CI’s will contain the true value of the parameter (β1 or β2). 6

39. 6.3.1 Confident Example If N=400, construct a 95% CI for B1:
Suppose OLS Gives us the Output:
If N=400, construct a 95% CI for B1:
To cut off 2.5% of each tail with df=infinity,
t*=1.96 6

40. 6.3.1 Confident Example If N=25, construct a 90% CI for B2:
Suppose OLS Gives us the Output:
If N=25, construct a 90% CI for B2:
To cut off 5% of each tail with df=23,
t*=1.71 6

41. 6.3.1 Confident Example Suppose OLS Gives us the Output:
In repeated samples, 95% (90%) of such confidence intervals will contain the true parameter β1 (β2). β1 = The value of Y when X is zero. β2 = The change in Y when X increases by 1.
Here, we are confident that X has a positive effect on Y.
We are confident that when X=0, Y is positive. 6

42. 6.4 Hypothesis Testing in a Simple Regression Context
As econometricians, we have questions.
-Do intelligent baby toys affect baby intellect?
-Do scarves have an elastic or inelastic demand?
Data has answers.
-Through hypothesis testing 6

43. 6.4.1 Setting Up the Hypothesis Test
State null and alternate hypotheses:
Ho: β2=2
Ha: β2≠2
2) Select a level of significance
α=Prob(Type 1 Error)
Let α=0.05
3) Determine critical t values (df=n-2)
If N=25, t* = ±2.069 6

44. 6.4.1 Setting Up the Hypothesis Test
4) Calculate test statistic
If β2hat=6.465 and se(β2hat)=1.034,
t=(β2hat- β2)/se(β2hat)
=( )/1.034)
= 4.318
5) Decide (Reject and do not reject regions)
Since t=4.318>t*=2.069, reject H0 6

45. 6.4.1 Setting Up the Hypothesis Test
6) Interpret
At a 5% level of significance, the change in Y due to a 1 unit change in X is not equal to 2. 6

46. 6.4.1 Example 2 Given a sample size of 26, we estimate the formula:
We want to test whether studying has any effect on grades.
H0: β2=0
Ha: β2≠0 6

47. 6.4.1 Example 2 Given α=0.01 and n-2=24, t*=±2.80 t=(0.01-0)/0.005 =2
Since t<t*, Do not reject H0
At a 1% level of significance, studying has no effect on mark. 6

48. 6.6 OLS Estimation
Often the calculation required for OLS estimation is greater than pen and paper is capable of.
When this occurs, an econometric program (such as Excel) is used to make the calculations.
The results should be expressed in this form:
For Example: 6