1. Studenmund(2006): Chapter 8
Lecture #8
Studenmund(2006): Chapter 8
Multicollinearity
Objectives
Perfect and imperfect multicollinearity
Effects of multicollinearity
Detecting multicollinearity
Remedies for multicollinearity

2. or 1X1+ 2X2 + 3X3 +…+ iXi = 0 The nature of Multicollinearity
Perfect multicollinearity:
When there are some functional relationships existing
among independent variables, that is  iXi = 0
or 1X1+ 2X2 + 3X3 +…+ iXi = 0
Such as 1X1+ 2X2 = 0  X1= -2X2
If multicollinearity is perfect, the regression coefficients of
the Xi variables, is, are indeterminate and their standard
errors, Se(i)s, are infinite.

3. Y = 0 + 1X1 + 2X2 +  ^
Example:
3-variable Case: =
(yx1)(x22) - (yx2)(x1x2)
(x12)(x22) - (x1x2)2 1 ^
(yx2)(x12) - (yx1)(x1x2) 2
If x2 = x1, =
(yx1)(2x12) - (yx1)(x1x1)
(x12)(2 x12) - 2(x1x1)2 1 ^
Indeterminate =
(yx1)(x12) - (yx1)(x1x1)
(x12)(2 x12) - 2(x1x1)2 2 ^
Similarly
If x2 = x1
Indeterminate

4. = 0  0 If multicollinearity is imperfect,
x2 = 1 x1+  where  is a stochastic error
(or x2 = 0+ 1 x1+  )
Then the regression coefficients, although determinate,
possess large standard errors, which means the
coefficients can be estimated but with less accuracy. =
(yx1)(2x12 +  2 ) - ( yx1 + y )( x1x1+ x1 )
(x12)(2 x12 +  2 ) - ( x1x1 + x1 )2 1 ^
 0
= 0
(Why?)

5. Example: Production function Yi = 0 + 1X1i + 2X2i + 3X3i + i
122 10 50 52
170 15 75
202 18 90 97
270 24
120
129
330 30
150
152
Y: Output
X1: Capital
X2: Labor
X3: Land
X2i = 5X1i.
There are always 5 workers per machine.
X1 = 5X2

6. b. Yi = 0 + 1X1i + 2X2i + 3X3i + i X1: Nominal interest rate;
Example: Perfect multicollinearity
a. Suppose D1, D2, D3 and D4 = 1 for spring, summer, autumn and winter, respectively.
Yi = 0 + 1D1i + 2D2i + 3D3i + 4D4i + 1X1i + i.
b. Yi = 0 + 1X1i + 2X2i + 3X3i + i
X1: Nominal interest rate;
X2: Real interest rate;
X3: CPI
a. D1 + D2 + D3 + D4 = 1.
The dummy variable trap is a special case of perfect multicollinearity.
b. X2 = X3 – X1.
c. Xt = Xt – Xt-1.
Usually, perfect multicollinearity appears in inappropriate models. The problem can be found and cured by a careful investigation on the definition of variables.
c. Yt = 0 + 1Xt + 2Xt + 3Xt-1 + t
Where Xt = (Xt – Xt-1) is called “first different”

7. Imperfect Multicollinearity
Yi = 0 + 1X1i + 2X2i + … + KXKi + i
When some independent variables are linearly correlated but the relation is not exact, there is imperfect multicollinearity.
0 + 1X1i + 2X2i +  + KXKi + ui = 0
where u is a random error term and k  0 for some k.
Although the relationship between variables might be fairly strong, it is not strong enough to allow one variable to be completely explained by other variables. Some unexplained variation still remains.
Imperfect multicollinearity will cause problems when linear relationship among independent variables is strong enough to affect significantly the estimation of the coefficients of the variables.
The stronger the linear relationship among independent variables, the more likely that some estimates are affected.
When will it be a problem?

8. Consequences of imperfect multicollinearity
1. The estimated coefficients are still BLUE,
however, OLS estimators have large variances and covariances,
thus making the estimation with less accuracy.
2. The estimation confidence intervals tend to be
much wider, leading to accept the “zero null
hypothesis” more readily.
3. The t-statistics of coefficients tend to be
statistically insignificant.
Can be
detected from
regression
results
4. The R2 can be very high.
5. The OLS estimators and their standard errors
can be sensitive to small change in the data.

9. OLS estimators are still BLUE under imperfect multicollinearity
Why???
Remarks:
Unbiasedness is a repeated sampling property, not about the properties of estimators in any given sample
Minimum variance does not mean small variance
Imperfect multicollinearity is just a sample phenomenon
The existence of imperfect multicollinearity does not violate any classical assumptions.
Empirically, data of different variables are linearly correlated in some way. That is, imperfect multicollinearity is an empirical fact. It causes problems when the collinearity is severe.
Whether the data are correlated enough to have a significant effect on the estimation of the equation depends on the particular sample drawn, and each sample must be investigated.
Since perfect multicollinearity is fairly easy to avoid, econometricians concern mainly imperfect multicollinearity.

10. Effects of Imperfect Multicollinearity
Unaffected:
OLS estimators are still BLUE.
The overall fit of the equation
The estimation of the coefficients of non-multicollinear variables

11. The variances of OLS estimators increase with the degree of multicollinearity
Regression model:
Yi = 0 + 1X1i + 2X2i + i
High correlation between X1 and X2
Difficult to isolate effects of X1 and X2 from
each other
This is the principal consequence of multicollinearity. Since two or more of the explanatory variables are significantly related, it becomes difficult to precisely identify the separate effects of the multicollinear variables.
It is likely to make large errors in estimation. There is a higher probability of obtaining a beta hat that is dramatically different from the true beta.

12. Closer relation between X1 and X2
larger r212
larger VIF
larger variances
where VIFk = 1/(1-Rk²), k=1,...,K and Rk² is the coefficient of determination of regressing Xk on all other (K-1) explanatory variables.
r12 is the correlation coefficient between X1 and X2.

13. Multicollinearity increases the likelihood of obtaining an unexpected sign for a coefficient.

14. Larger
a. More likely to get unexpected signs.
tends to be large
Larger variances tend to increase the standard errors of estimated coefficients.
c. Larger standard errors  Lower t-values

15. d. Larger standard errors
 Wider confidence intervals
Less precise interval estimates.

16. Detection of Multicollinearity
Example: Data set: CONS8 (pp. 254 – 255)
COi = 0 + 1Ydi + 2LAi + i
CO: Annual consumption expenditure
Yd: Annual disposable income
LA: Liquid assets
96% of the total variation of Consumption can be explained by the model. That means the joint explanatory power of Income and Wealth is very high.
No variable is significant individually. That means the model has no explanatory power.
The conclusions of joint test and individual tests are not consistent.

17. Less significant t-values
Studenmund (2006) - Eq. 8.9, pp254
Since LA (liquid assets, saving, etc.)
is highly related to YD (disposable income)
Drop one variable
Results:
High R2 and Adjusted R2
Less significant t-values

18. OLS estimates and SE’s can be sensitive to specification and small changes in data
Specification changes:
Add or drop variables
Small changes:
Add or drop some observations
Change some data values
The addition or deletion of an explanatory variable or of a few observations will often cause major changes in the values of the beta hats when significant muticollinearity exists.
If you drop a variable, even one that appears to be statistically insignificant, the coefficients of the remaining variables in the equation will sometimes change dramatically.
These large changes occur because OLS estimation is sometimes forced to emphasize small differences between variables to distinguish the effect of one multicollinear variable from another. If two variables are virtually identical throughout most of the sample, the estimation procedure relies on the observations in which the variables move differently in order to distinguish between them. As a result, a specification change that drops a variable that has an unusual value for one of these crucial observations can cause the estimated coefficients of the multicollinear variables to change dramatically.

19. High Simple Correlation Coefficients
In the previous example, the correlation coefficient between Yd and LA is
Remark: High rij for any i and j is a sufficient indicator for the existence of multicollinearity but not necessary.

20. Variance Inflation Factors (VIF) method
Procedures:
Obtain
Rule of thumb: VIF > 5  multicollinearity
Notes: (a.) Using VIF is not a statistical test.
(b.) The cutting point is arbitrary.

21. Remedial Measures 1. Drop the Redundant Variable
Using theories to pick the variable(s) to drop.
Do not drop a variable that is strongly supported by theory. (Danger of specification error)

22. Insignificant
Since M1 and M2
are highly related
Other examples: CPI <=> WPI;
CD rate <=> TB rate
GDP  GNP  GNI

23. Check after dropping variables:
The estimation of the coefficients of other variables are not affected. (necessary)
R2 does not fall much when some collinear variables are dropped. (necessary)
More significant t-values vs. smaller standard errors (likely)

24. 2. Redesigning the Regression Model
There is no definite rule for this method.
Example (Studenmund(2006), pp.268)
Ft = average pounds of fish consumed per capita
PFt = price index for fish
PBt = price index for beef
Ydt = real per capita disposable income
N = the # of Catholic
P = dummy = 1 after the Pop’s 1966 decision, = 0 otherwise

25. Most t-values are insignificant
High correlations
VIFPF = 43.4
VIFlnYd =23.3
VIFPB = 18.9
VIFN =18.5
VIFP =4.4
Signs are unexpected
Most t-values are insignificant

26. Use the Relative Prices (RPt = PFt/PBt)
Drop N, but not improved
Improved
Use the Relative Prices
(RPt = PFt/PBt)
Ft = 0 + 1RPt + 2lnYdt + 3Pt + t

27. Using the lagged term of RP to allow the lag effect in the regression
Improved much
Using the lagged term of RP to allow the lag effect in the regression
Ft = 0 + 1RPt-1 + 2lnYdt + 3Pt + t

28. 3. Using A Priori Information
From previous empirical work, e.g.
Consi = 0 + 1Incomei + 2Wealthi + i
and a priori information: 2 = 0.1.
Then construct a new variable or proxy,
(Cons*i = Consi – 0.1Wealthi)
Run OLS: Cons*i = 0 + 1Incomei + i
If b2 = 0.1b1, for example, then
cons = b0 + b1x + e
where x = income + 0.1wealth.

29. 4. Transformation of the Model
Taking first differences of time series data.
Origin regression model:
Yt = 0 + 1X1t + 2X2t + t
Transforming model: First differencing
Yt = ’0 +’1X1t + ’2X2t + ut
Where Yt = Yt- Yt-1, (Yt-1 is called a lagged term)
X1t = X1t- X1,t-1, X2t = X2t- X2,t-1,
Eg.
X2: 2, 1, 3, dX2: ., -1, 2, 1
X3: 1, 2, 4, dX3: ., 1, 2, -1
r(X2, X3) = 0.6
r(dX2, dX3) =
There may be some other problems if the difference equation is used.

30. 5. Collect More Data (expand sample size)
Larger sample size means smaller variance of estimators.
6. Doing Nothing:
Multicollinearity does not always cause problems. So, for example, a high correlation coefficient does not necessarily yield insignificant t-values and wrong signs.
The danger of specification bias. The correction may lead to worse results unless the restrictions are correct.
Unless multicollearity causes serious biased, and the change of specification give better results.