1. Lecture 18 Preview: Explanatory Variable/Error Term Independence Premise, Consistency, and Instrumental Variables
Review
Regression Model
Standard Ordinary Least Squares (OLS) Premises
Estimation Procedures Embedded within the Ordinary Least Squares (OLS) Estimation Procedure
Taking Stock and a Preview: The Ordinary Least Squares (OLS) Estimation Procedure
A Closer Look at the Explanatory Variable/Error Term Independence Premise
Explanatory Variable/Error Term Correlation and Bias
Geometric Motivation
Confirming Our Logic
Estimation Procedures: Large and Small Sample Properties
Unbiased and Consistent Estimation Procedure
Unbiased but Not Consistent Estimation Procedure
Biased but Consistent Estimation Procedure
The Ordinary Least Squares (OLS) Estimation Procedure, and Consistency
Instrumental Variable (IV) Estimation Procedure: A Two Regression Procedure
Mechanics
The Two “Good” Instrument Conditions

2. Regression Model
yt = Dependent variable
xt = Explanatory variable
et = Error term
yt = Const + xxt + et
Const and x are the parameters
t = 1, 2, …, T
The error term is a random variable representing random influences: Mean[et] = 0
Standard Ordinary Least Squares (OLS) Premises
Error Term Equal Variance Premise: The variance of the error term’s probability distribution for each observation is the same.
Error Term/Error Term Independence Premise: The error terms are independent.
Explanatory Variable/Error Term Independence Premise: The explanatory variables, the xt’s, and the error terms, the et’s, are not correlated.
OLS Estimation Procedure Includes Three Estimation Procedures
Value of the parameters, Const and x:
bx =
bConst =
Question: What happens when the explanatory variable/error term independence premise is violated?
Variance of the error term’s probability distribution, Var[e]:
SSR
EstVar[e] =
Degrees of Freedom
Variance of the coefficient estimate’s probability distribution, Var[bx]:
EstVar[bx] =
Good News: When the standard premises are satisfied each of these procedures is unbiased.
Good News: When the standard premises are satisfied the OLS estimation procedure for the coefficient value is the best linear unbiased estimation procedure (BLUE).
Crucial Point: When the ordinary least squares (OLS) estimation procedure performs its calculations, it implicitly assumes that the three standard (OLS) premises are satisfied.

3. Satisfied: Explanatory Variable and Error Terms Are Independent
Taking Stock and a Preview: The Ordinary Least Squares (OLS) Estimation Procedure
OLS Bias Question: Is the explanatory variable/error term independence premise satisfied or violated?
Satisfied: Explanatory Variable and Error Terms Are Independent
Violated: Explanatory Variable and Error Terms Are Correlated
Is the OLS estimation procedure for the value of the coefficient biased or unbiased?
No – Biased
Yes – Unbiased
OLS Reliability Question: Are the error term equal variance and the error term/error term independence premises satisfied or violated?
Satisfied
Violated
Can the OLS calculation for the coefficient’s standard error be “trusted?”
Yes No
Is the OLS estimation procedure for the value of the coefficient BLUE?
Yes No
Preview: When the explanatory variable/error term independence premise is violated and consequently the ordinary least squares (OLS) estimation procedure is biased, other estimation procedures can be used to mitigate although not completely remedy the bias problem.

4. Explanatory Variable/Error Term Independence Premise: The explanatory variables, the xt’s, and the error terms, the et’s, are not correlated.
Question: What happens when this premise is violated?
Claim: When the explanatory variables and the error terms are correlated the ordinary least squares estimation procedure for the coefficient value is biased.
Question: What does explanatory variable/error term independence and correlation “look like?”
Explanatory Variable/Error Term Independence: CorrX&E = 0.
The explanatory variable and the error terms appear to be independent.
After many, many repetitions, each student’s mean is approximately 0.
 Lab 18.1

5. Explanatory Variable/Error Term Positive Correlation: CorrX&E = .6
 Lab 18.1
The explanatory variable and the error terms appear to be positively correlated.
After many, many repetitions:
When the value of the explanatory variable is low the error term is typically negative.
When the value of the explanatory variable is high the error term is typically positive.
Explanatory Variable/Error Term Negative Correlation: CorrX&E = .6
 Lab 18.1
The explanatory variable and the error terms appear to be negatively correlated.
After many, many repetitions:
When the value of the explanatory variable is low the error term is typically positive.
When the value of the explanatory variable is high the error term is typically negative.

6. Estimated equation y = bConst + bxx Actual equation y = Const + xx
Consequences of Explanatory Variable/Error Term Correlation
Explanatory variable, xt, and error term, et, are positively correlated et xt
xt up  et up
Plot the y’s on the diagram:
yt = Const + xxt + et yt
Estimated equation y = bConst + bxx
Find the best fitting line:
Explanatory variable and error term are positively correlated
Actual equation y = Const + xx
 Estimated equation more steeply sloped that actual equation
 OLS estimation procedure for coefficient value is biased upward xt

7. Actual equation y = Const + xx Estimated equation y = bConst + bxx
Explanatory variable, xt, and error term, xt, are negatively correlated et
Explanatory variable, xt, and error term, et, are negatively correlated xt
xt up  et down
Plot the y’s on the diagram: yt
yt = Const + xxt + et
Actual equation y = Const + xx
Find the best fitting line:
Explanatory variable and error term are negatively correlated
 Estimated equation less steeply sloped that actual equation
Estimated equation y = bConst + bxx
 OLS estimation procedure for coefficient value is biased downward xt

8. Explanatory Variable/Error Term Independence Premise: The explanatory variables, the xt’s, and the error terms, the et’s, are not correlated.
Our Suspicions
When the explanatory variable and the error terms are positively correlated, the ordinary least squares (OLS) estimation procedure for the coefficient value will be biased upward.
When the explanatory variable and the error terms are negatively correlated, the ordinary least squares (OLS) estimation procedure for the coefficient value will be biased downward.
 Lab 18.2
2.0 0.0 4.0
6.1 4.1 3.6
2.1 4.1 3.6
Estimation Corr Sample Actual Mean of Magnitude Variance of
Procedure X&E Size Coef Coef Ests of Bias Coef Ests
OLS
OLS
OLS 
Explanatory variable and error term are positively correlated
Explanatory variable and error term are uncorrelated
Explanatory variable and error term are negatively correlated
 OLS estimation procedure for coefficient value is biased upward
 OLS estimation procedure for coefficient value is unbiased
 OLS estimation procedure for coefficient value is biased downward

9. Estimation Procedures: Unbiased versus Biased and Consistent versus Inconsistent
Unbiased: Small Sample Property. The estimation procedure does not systematically underestimate or overestimate the actual value.
Formally, the mean of the estimate’s probability distribution equals the actual value.
Mean[Est] = Actual Value
When the estimate’s probability distribution is symmetric, the chances that the estimate is greater than the actual value equal the chances that it is less.
Unbiasedness is called a small sample property because it does not depend on the sample size.
Unbiasedness depends only of the mean of the estimate’s probability distribution.
Consistent: Large Sample Property. Both the mean and variance of the estimate’s probability distribution are important for consistency:
Mean of the estimate’s probability distribution:
Either
The estimation procedure is unbiased: Mean[Est] = Actual Value or
The estimation procedure is biased, but the magnitude of the bias diminishes as the sample size becomes larger.
Formally, as the sample size approaches infinity the mean approaches the actual value: As Sample Size  : Mean[Est]  Actual Value
Variance of the estimate’s probability distribution: The variance diminishes as the sample size becomes larger
Formally, as the sample size approaches infinity the variance approaches 0: As Sample Size  : Variance[Est]  0

10. All Estimation Procedures
Unbiased
Consistent
To get a better sense of the two different properties of estimation procedures we shall consider three estimation procedures:
Unbiased and Consistent
Unbiased but Not Consistent
Biased but Consistent

11.  Lab 18.3  Lab 18.4 Categorizing Estimation Procedures
Does Mean[Est] equal the Actual Value?
Yes - Unbiased
No - Biased
Does Mean[Est]  Actual Value as the sample size  ?
Biased and Not Consistent No
Yes
Does Var[Est]  0 as the sample size  ?
Does Var[Est]  0 as the sample size  ? No
Yes No
Yes
Unbiased and Consistent
Unbiased but Not Consistent
Biased but Consistent
 Lab 18.3
 Lab 18.4
After Many, Many Repetitions
Estimation Corr Sample Actual Mean of Magnitude Variance of
Procedure X&E Size Coef Coef Ests of Bias Coef Ests
OLS
  2.50
OLS
  1.14
Any Two
  7.5
Any Two
  17.3

12. Illustrating a Consistent but Biased Estimation an Procedure: Revisit Our Friend Clint
Random Sample Procedure: Write the name of each individual in the population on a 35 card
Perform the following procedure 16 times:
Thoroughly shuffle the cards.
Randomly draw one card.
Ask that individual if he/she is voting for Clint and record the answer.
Replace the card.
Calculate the fraction of the sample supporting Clint.
Nonrandom Sample Procedure:
Leave Clint’s dorm room and ask the first 16 people you run into if he/she is voting for Clint.
Calculate the fraction of the sample supporting Clint.
Questions: Compared to the general student population:
Are the students who live near Clint are more likely to be Clint’s friend?
Yes
Are the students who live near Clint more likely to vote for him?
Yes
Since your starting point is Clint’s dorm room, is it likely that you will poll students who are more supportive of Clint than the general student population?
Yes
Would you be biasing your poll in Clint’s favor?
Yes

13.  Lab 18.5 Consistent Estimation Procedure Simulation
Sampling Population Sample Mean (Average) Magnitude Variance
Technique Fraction Size of Estimates of Bias of Estimates
Random
Random
Random
Nonrandom
Nonrandom
Nonrandom
.50 .00 .016
.50 .00 .010
.50 .00 .0025
.56 .06 .015
.54 .04 .010
.51 .01 .0025
Is the random procedure unbiased?
Yes
Is the nonrandom procedure unbiased? No
Is the random procedure consistent?
Yes
Is the nonrandom procedure consistent?
Yes
As the sample size increases the magnitude of the bias diminishes.
As the sample size increases the variance of the estimates diminishes.
The nonrandom procedure is biased but consistent.

14. The Explanatory Variable/Error Term Premise, the Ordinary Least Squares (OLS) Estimation Procedure, and Consistency
Review: We have already shown that the ordinary least squares (OLS) estimation procedure biased when explanatory variable/error term correlation is present.
2.0 0.0 4.0
6.1 4.1 3.6
2.1 4.1 3.6
Estimation Corr Sample Actual Mean of Magnitude Variance of
Procedure X&E Size Coef Coef Ests of Bias Coef Ests
OLS
OLS
OLS 
Question: But might the ordinary least squares (OLS) estimation procedure still be consistent when explanatory variable/error term correlation is present?
 Lab 18.6
Estimation Corr Sample Actual Mean of Magnitude Variance of
Procedure X&E Size Coef Coef Ests of Bias Coef Ests
OLS
OLS
OLS
6.1 4.1 3.6
6.1 4.1 1.7
6.1 4.1 1.2
There is nothing but bad news.
In the presence of explanatory variable/error term correlation, the ordinary least squares (OLS) estimation procedure is:
Biased
Question: Where do we go from here?
Not Consistent

15. The Instrumental Variable (IV) Estimation Procedure
Question: Why is there a problem?
yt = Const xxt t
When xt and t are correlated 
xt is a “problem” explanatory variable
“Problem” Explanatory Variable: xt is the “problem” explanatory variable:
The explanatory variable, xt, is correlated with the error term, t.
 The explanatory variable/error term independence premise is violated.
 The ordinary least squares (OLS) estimation procedure for the coefficient value is biased.

16. Addressing the “Problem” Explanatory Variable Using Instrumental Variables
Choose an Instrument: A “good” instrument, zt, must meet two conditions.
Good Instrument Condition 1: Correlated with the “problem” explanatory variable, xt.
Good Instrument Condition 2: Uncorrelated with the error term, t.
Instrumental Variables (IV) Regression 1: Use the instrument, zt, to provide an “estimate” of the problem explanatory variable, xt.
Dependent Variable: “Problem” explanatory variable, xt.
Explanatory Variable: Instrument, zt.
Estimate of the problem explanatory variable: Estxt = aConst + azzt where aConst and az are the estimates of the constant and coefficient in this regression, IV Regression 1.
Instrumental Variables (IV) Regression 2: In the original model, replace the “problem” explanatory variable, xt, with its surrogate, Estxt, the estimate of the “problem” explanatory variable provided by the instrument, zt, from IV Regression 1.
Dependent Variable: Original dependent variable, yt.
Explanatory Variable: Estimate of the “problem” explanatory variable based on the results from IV Regression 1, Estxt.

17. The “Good” Instrument Conditions
Good Instrument Condition 1: The instrument, zt, must be correlated with the “problem” explanatory variable, xt.
Focus on Instrumental Variables (IV) Regression 1:
Use the instrument, zt, to provide an “estimate” of the problem explanatory variable, xt.
Dependent Variable: “Problem” Explanatory Variable, xt.
Explanatory Variable: Instrument, zt
We are using the instrument to create a surrogate for the “problem” explanatory variable:
Estxt = aConst + azzt
The estimate, Estxt, will be a “good” surrogate only if the instrument is correlated with the problem explanatory variable.
Only if Estxt, is a good predictor of the “problem” explanatory variable.

18. Instrumental Variables (IV) Regression 2
Good Instrument Condition 2: The instrument, zt, must be independent of the error term, t.
Instrumental Variables (IV) Regression 2
In the original model, replace the “problem” explanatory variable, xt, with its surrogate, Estxt, the estimate of the “problem” explanatory variable provided by the instrument, zt, from IV Regression 1.
Problem: xt and t
are correlated.
Original Model:
yt = Const xxt t
Replace the “problem” explanatory variable with its surrogate
yt = Const xEstxt t
To avoid violating the explanatory variable/error term independence premise
 Estxt and t must be independent
Estxt = aConst + azzt
zt and t must be independent.

19. Justifying the Instrumental Variable (IV) Approach: A Simulation
Model: yt = Const + xxt + et
Defaults
IV is selected indicating that the instrumental variable (IV) estimation procedure we just described will be used to estimate the value of the explanatory variable’s coefficient.
The Corr X&E list the value .30 is specified. The correlation coefficient for the explanatory variable and error term equals .30. Hence, the explanatory variable/error term independence premise is violated.
Two new correlation lists appear in this simulation: Corr X&Z and Corr Z&E. The two new lists reflect the two conditions required for a good instrument.
The Corr X&Z list specifies the correlation coefficient for the explanatory variable and the instrument. To be a “good” instrument the explanatory variable and the instrument must be correlated. The default value is .50.
The Corr Z&E specifies the correlation coefficient for the instrument and error term. To be a “good” instrument the instrument and error term must be independent. The default value is .00; that is, the instrument and error term are independent.
 Lab 18.7
Estimation Corr Corr Corr Sample Actual Mean of Magnitude Variance of
Procedure X&Z Z&E X&E Size Coef Coef Ests of Bias Coef Ests IV IV IV
1.61 .39 20.3
1.82 .18 8.7
1.88 .12 5.5
Question: Is the IV estimation procedure: Unbiased?
No.
Consistent?
Yes.

20. Estimation Corr Corr Corr Sample Actual Mean of Magnitude Variance of
Procedure X&Z Z&E X&E Size Coef Coef Ests of Bias Coef Ests
1.61 .39 20.3
1.82 .18 8.7
1.88 .12 5.5 IV IV IV
Question: Is the IV estimation procedure: Unbiased?
No.
Consistent?
Yes.
Good Instrument Condition 1: Instrument/”Problem” Explanatory Variable Correlation Suppose that the instrument is more highly correlated with the “problem” explanatory variable:
Estimation Corr Corr Corr Sample Actual Mean of Magnitude Variance of
Procedure X&Z Z&E X&E Size Coef Coef Ests of Bias Coef Ests
1.95 .05 2.3 IV
Question: Is the instrument better?
Yes. Both the magnitude of the bias and the variance of the estimates is less.
Intuition:
zt more highly correlated with xt
 Lab 18.7 
Estxt is a better predictor for xt 
Estxt is a better surrogate for xt 
Instrument variables (IV) estimation procedure is better.

21. Good Instrument Condition 2: Instrument/Error Term Independence: Suppose that the instrument is correlated with the error term:
 Lab 18.7
Estimation Corr Corr Corr Sample Actual Mean of Magnitude Variance of
Procedure X&Z Z&E X&E Size Coef Coef Ests of Bias Coef Ests
3.69 1.69 6.8
3.74 1.74 3.2
3.76 1.76 2.1 IV IV IV
Question: Is the IV estimation procedure: Unbiased?
No.
Consistent?
No.