Lecturer Dr. Veronika Alhanaqtah ECONOMETRICS Lecturer Dr. Veronika Alhanaqtah

Topic 4.3. Problems when building multiple linear regression with the least square method: Autocorrelation Nature of autocorrelation Autocorrelation of the first order Consequences of autocorrelation Correction of autocorrelation: Robust standard errors Detecting of autocorrelation Graphical analysis of residuals The Durbin-Watson test The Breusch-Godfrey test

OLS assumptions and Gauss-Markov theorem The expected value of a residual ε is equal to zero for all observations: M ε i =0 The variance of each residual is constant (equal, homoscedastic) for every observation: Var ε i =Var ε j = σ 2 Residuals ε i and ε j are uncorrelated between observations: Cov ε i , ε j |X =E ε i ε j =0 i≠j A residual ε is independent on explainable variables (regressors): σ ε i x i =cov ε i , x i =0 Model is linear in relation to its parameters. The Gauss-Markov theorem: if the set of OLS assumptions is hold, then OLS estimators are BLUE-estimators (unbiased, consistent, efficient).

1. Nature of autocorrelation Important assumption of the Gauss-Markov theorem is: the errors are uncorrelated between observations: 𝐶𝑜𝑣 𝜀 𝑖 , 𝜀 𝑗 |𝑋 = 𝐸 𝜀 𝑖 𝜀 𝑗 =0 𝑖≠𝑗 If this assumption is violated, we have autocorrelation.

1. Nature of autocorrelation We usually see autocorrelation in the context of time series  data, panel data, cluster samples, hierarchical data, repeated measures data, longitudinal data, and other data with dependencies. In such cases generalized least squares (GLM) provides a better alternative than the OLS. In economic data it is more common to see positive autocorrelation (𝜎 𝜀 𝑡−1 , 𝜀 𝑡 >0), than negative autocorrelation (𝜎 𝜀 𝑡−1 , 𝜀 𝑡 <0).

1. Nature of autocorrelation Example of positive autocorrelation: investigating the demand for soft drinks Y dependent on income X, using monthly data. The trend relationship shows the increase of demand Y when the income X increases. This trend could be represented as the linear function 𝑌= 𝛽 1 + 𝛽 2 𝑋. Summer Winter X Y

1. Nature of autocorrelation Example of negative autocorrelation: Negative autocorrelation means that positive deviation is followed by the negative deviation, and vice versa. This situation can happen when we analyze the same relationship, as above, but use seasonal data (winter – summer).

1. Nature of autocorrelation Reasons of autocorrelation: mistakes in model specification time lag in change of economic parameters web-effect data smoothing

1. Reasons of autocorrelation Mistakes of model specification. Sometimes a researcher doesn’t include in a model some important explainable variable (regressor) or choose incorrect shape of a function, which has to explain a relationship between variables. For example, we analyze a dependence of marginal costs MC on total output Q. In fact, this relationship must be described by a quadratic function: 𝑀𝐶= 𝛽 0 + 𝛽 1 𝑄+ 𝛽 2 𝑄 2 +𝜀. But a researcher, by mistake, describes this relationship by a linear function: 𝑀𝐶= 𝛽 0 + 𝛽 1 𝑄+𝜀.

1. Reasons of autocorrelation Time lag. Many economic parameters are cyclical, as a consequence of undulating economic cycles. Changes doesn’t happen immediately. It takes some time or a time lag. Web-effect. In many spheres of economic activity, parameters react to changes of economic conditions with delay, or time lag. For example, supply of agricultural products react to price changes with delay (equal to the agricultural season). High price of agricultural products in the previous year, most likely, leads to effect of its overproduction in the current year, and, as a consequence, the price will decrease, and so on. In this case, deviation of residuals from each other is not accidental (or random). Data smoothing. Very often the data over some long period of time are averaged along the subintervals. To smooth a data set is to create an approximating function that attempts to capture important patterns in the data, while leaving out noise or other fine-scale structures.

1. Reasons of autocorrelation When is it logical to expect autocorrelation? Observations are close in time and space terms. For example, we study a dependence of a number of clients of a firm on a number of advertisements. It is logical to expect that observations, related to yesterday-today points in time are highly related. Presence of non-observable factor which influences on “neighbor” observations. For example, we study data on regions. We are interested in factors that influence migration flows. We can often see that migration flows of neighbor regions are highly related. And if two observations relate to two neighbor regions, then residuals 𝜀 𝑖 and 𝜀 𝑗 are related to each other.

1. Reasons of autocorrelation Autocorrelation is examined in details: time series analysis; spatial econometrics; panel data. These are almost independent disciplines.

2. Autocorrelation of the first order Autocorrelation may have very complicated structure: AR, MA, ARMA, ARIMA, VAR, VMA, VARMA, VECM, ARCH, GARCH, EGARCH, FIGARCH, TARCH, AVARCH, ZARCH, CCC, DCC, BEKK, VEC, DLM ….

2. Autocorrelation of the first order 𝜺 𝒕 = 𝝆𝜺 𝒕−𝟏 + 𝒖 𝒕 𝝆=𝟏: The equation means that today’s error is equal to some constant, multiplied by yesterday’s error, plus its own residual (−1< 𝜌<1). Assumptions: ut – are independent on each other; ut are independent on regressors; ut are distributed equally; 𝐸 𝑢 𝑡 =0, 𝑉𝑎𝑟 𝑢 𝑡 = 𝜎 𝑢 2 , where 𝐸 𝑢 𝑡 is expected value (population mean), 𝑉𝑎𝑟 is variance, which is constant. Question: What structure of autocorrelation does appear in this case? How does 𝐶𝑜𝑟𝑟( 𝜀 𝑡 , 𝜀 𝑡−1 ) look like?

2. Autocorrelation of the first order Question: What structure of autocorrelation does appear in this case? How does 𝐶𝑜𝑟𝑟( 𝜀 𝑡 , 𝜀 𝑡−1 ) look like? 𝜺 𝒕 = 𝒖 𝒕 + 𝟏 𝟐 𝒖 𝒕−𝟏 + 𝟏 𝟐 𝟐 𝒖 𝒕−𝟐 + 𝟏 𝟐 𝟑 𝒖 𝒕−𝟑 +… We see that when an index t changes, neither variance 𝑉𝑎𝑟 𝜀 𝑡 = 𝜎 𝜀 2 (constant) nor covariance 𝐶𝑜𝑣( 𝜀 𝑡 , 𝜀 𝑡−1 ) changes. Covariance between two neighbor errors 𝐶𝑜𝑣 𝜀 𝑡 , 𝜀 𝑡−1 = 𝛾 1 does not depend on t. It means that characteristics of 𝜀 𝑡 does not change in time. This property is called stationarity.

2. Autocorrelation of the first order Question: What structure of autocorrelation does appear in this case? How does 𝐶𝑜𝑟𝑟( 𝜀 𝑡 , 𝜀 𝑡−1 ) look like? Structure of autocorrelation of the first order: 𝑪𝒐𝒓𝒓 𝜺 𝒕 , 𝜺 𝒕−𝒌 = 𝝆 𝒌 It means that the relationship decrease over time. The further two errors are from each other, the weaker the correlation between them.

3. Consequences of autocorrelation (1) Estimates of 𝜷 -coefficients are still linear and unbiased, but are not efficient. Efficiency implies estimates with the smallest variance. Linearity means that estimates of coefficients are still linear with respect to y. Unbiasedness means that, on average, estimates of 𝛽 fall into unknown coefficient β: 𝐸 𝛽 |𝑋 =𝛽, 𝐸 𝛽 =𝛽. (2) Variance (dispersion) of estimates of coefficients are biased. Very often, variance, computed by the standard formulas, are underestimated, that leads to overestimated t-statistics. Consequently, if we rely upon such inferences, we might mistakenly accept as significant those coefficients which, in fact, are not significant. (3) Statistical inferences on t- and F-statistic (which determine significance of 𝛽 -coefficients and coefficient of determination R2) are incorrect. As a consequence, prediction qualities of a model are worsened.

3. Consequences of autocorrelation Summing up: in presence of conditional autocorrelation: we can use and interpret 𝛽 -coefficients; but standard errors 𝑠𝑒 𝐻𝐶 𝛽 𝑗 are inconsistent; so we can’t construct confidence intervals for 𝛽 𝑗 and test hypothesizes about 𝛽 𝑗 .

4. Correction of autocorrelation: robust standard errors What to do in presence of autocorrelation? Correct standard errors! 𝑉𝑎𝑟 𝛽 |𝑋 = 𝑅𝑆𝑆 𝑛−𝑘 𝑋′𝑋 −1 𝑉𝑎𝑟 𝐻𝐶 𝛽 |𝑋 = 𝑋′𝑋 −1 𝜱 𝑋′𝑋 −1  We use Newey-West estimator of the covariance matrix (1987). The estimator can be used to improve the OLS-regression when the variables have heteroskedasticity or autocorrelation. 𝜱 = 𝑗=−𝑘 𝑘 𝑘− 𝑗 𝑘 𝑡 𝜀 𝑡 𝜀 𝑡+𝑗 𝑋 𝑡 ′ ∙ 𝑋 𝑡+𝑗 The idea is to replace usual standard errors s𝑒 𝛽 𝑗 for robust (heteroscedasticity and autocorrelation consistent) standard errors 𝑠𝑒 𝐻𝐴𝐶 𝛽 𝑗 , where 𝑠𝑒 𝐻𝐴𝐶 𝛽 𝑗 are roots of diagonal elements of a corresponding matrix.

4. Correction of autocorrelation: robust standard errors What problems have we solved? If in formula for t-statistic we insert 𝑠𝑒 𝐻𝐴𝐶 𝛽 𝑗 , then, when number of observations n is growing, then t-statistic will tend to normal distribution. 𝛽 𝑗 − 𝛽 𝑗 𝑠𝑒 𝐻𝐴𝐶 𝛽 𝑗 →𝑁(0,1) We need normality condition when we test hypothesis on significance of 𝛽 -coefficients. Now we can test hypothesis on significance of 𝛽 -coefficients and construct confidence intervals. Great!

4. Correction of autocorrelation: robust standard errors What problems we haven’t solved? Estimates of 𝛽 -coefficients haven’t changed and still not efficient.

4. Correction of autocorrelation: robust standard errors In practice: (1) Estimate a model as usual: model<-lm(data=data, y~x+z) (2) Compute robust covariance matrix (“sandwich” package) vcovHAC(model) (3) Use robust covariance matrix for hypothesis testing (“lmtest” package) coeftest(model,vcov.=vcovHAC)

4. Correction of autocorrelation: robust standard errors When is it advisable to use robust covariance matrix and robust standard errors? In cases when we suspect presence of autocorrelation and do not want to model its structure (structure of relationship between residuals). For example, in time series or in data where there is geographical closeness between observations.

5. Detecting autocorrelation 5.1. Graphical analysis of residuals Estimate a model with the help of OLS and build a graph of residuals. We plot a previous residual 𝜀 𝑡−1 along the X-axis and a current residual 𝜀 𝑡 along the Y-axis. 𝜀 𝑡 = 0.9𝜀 𝑡−1 + 𝑢 𝑡 Positive autocorrelation 𝜀 𝑡 = −0.9𝜀 𝑡−1 + 𝑢 𝑡 Negative autocorrelation 𝜀 𝑡 are independent No autocorrelation

5. Detecting autocorrelation 5.2. The Durbin-Watson test Assumptions: DW-test is applied only for the autocorrelation of the first order: 𝜀 𝑡 = 𝜌𝜀 𝑡−1 + 𝑢 𝑡 Normality of residuals 𝜀. Strong exogenity, 𝐸 𝜀 𝑡 |𝑋 =0 We test a hypothesis H0: 𝜌=0 (no autocorrelation).

5. Detecting autocorrelation 5.2. The Durbin-Watson test Algorithm of the Durbin-Watson test: Estimate a regression model, using OLS, and obtain residuals 𝜀 𝑖 . Calculate DW-statistic: 𝐷𝑊= 𝑖=2 𝑛 𝜀 𝑖 − 𝜀 𝑖−1 2 𝑖=1 𝑛 𝜀 𝑖 2

5. Detecting autocorrelation 5.2. The Durbin-Watson test Statistical inference on DW-test: 𝜌 – sample correlation of residuals, then 𝐷𝑊=2 1− 𝜌 : 0<𝐷𝑊<4 𝐷𝑊≈0 : strong positive autocorrelation 𝜌 ≈1 𝐷𝑊≈2 : no autocorrelation 𝜌 ≈0 𝐷𝑊≈4 : strong negative autocorrelation 𝜌 ≈−1

5. Detecting autocorrelation 5.2. The Durbin-Watson test In practice: R calculates exact p-values for DW-test; If p-value is less then level of significance, then hypothesis H0 on absence of autocorrelation is rejected; 𝑝_𝑣𝑎𝑙𝑢𝑒<𝛼 , H0 is rejected, there is autocorrelation in the model. H0: no autocorrelation. Or we can use critical value approach (tables of critical points of DW- statistic).

5. Detecting autocorrelation 5.3. The Breusch-Godfrey test Assumptions: BG-test is used for autocorrelation of any given order (𝜌). 𝜀 𝑡 = 𝛷 1 𝜀 𝑡−1 +…+ 𝛷 𝜌 𝜀 𝑡−𝜌 + 𝑢 𝑡 It does not require normality of residuals 𝜀. BG-test is correct even if strong exogenity condition (𝐸 𝜀 𝑡 |𝑋 =0) is violated; BG-test is asymptotical test, i.e. it is correct when n is big. We test a hypothesis H0: 𝛷 1 = 𝛷 2 =…= 𝛷 𝜌 =0 (no autocorrelation).

5. Detecting autocorrelation 5.3. The Breusch-Godfrey test Algorithm of the Breusch-Godfrey test: Estimate a regression model, using OLS, and obtain residuals 𝜀 𝑖 . Build an auxiliary regression of 𝜀 𝑖 on initial regressors ( 𝜀 𝑡−1 , 𝜀 𝑡−2 ,…, 𝜀 𝑡−𝜌 ) and calculate 𝑅 𝑎𝑢𝑥 2 . Calculate BG-statistic, where n is a number of observations, 𝜌 is an assumed order of autocorrelation: 𝐵𝐺= 𝑛−𝜌 𝑅 𝑎𝑢𝑥 2

5. Detecting autocorrelation 5.3. The Breusch-Godfrey test Statistical inference on BG-test: If H0 is true, then BG-statistic has 𝜒2-distribution with 𝜌 degrees of freedom. H0: 𝛷 1 = 𝛷 2 =…= 𝛷 𝜌 =0 (no autocorrelation) 𝐵𝐺= 𝑛−𝜌 𝑅 𝑎𝑢𝑥 2 ~ χ 𝜌 2 If BG-statistic > χ 𝑐𝑟𝑖𝑡𝑖𝑐𝑎𝑙 2 , then H0 is rejected. Even though DW-test is rather widespread, it is advisable to prefer BG-test to DW-test. Nowadays, analogues of DW-test are used in spatial econometrics.

5. Detecting autocorrelation Example of DW-test and BG-test: Model: 𝑰 𝒕 =−𝟏𝟐−𝟏.𝟎 𝒓 𝒕 +𝟎.𝟏𝟔 n=19, DW=1.32 Question: Is there autocorrelation in the model? What is the order of autocorrelation ( 𝜌 )? Assumption 1. Autocorrelation of the first order: 𝜀 𝑡 = 𝛾 1 𝜀 𝑡−1 + 𝑢 𝑡 Assumption 2. Autocorrelation of the second order: 𝜀 𝑡 = 𝛾 1 𝜀 𝑡−1 + 𝛾 2 𝜀 𝑡−2 + 𝑢 𝑡

Autocorrelation Lecture summary Autocorrelation: when the following OLS-assumption is not hold: Cov ε i , ε j |X =E ε i ε j =0 i≠j We can observe autocorrelation in time series and spatial data. In the most simple case, in order to correct autocorrelation it is sufficient to use special standard errors: seHAC. If we want to make an investigation of autocorrelation profoundly, we should keep in mind that it is a separate discipline (there are many special models, including the maximum likelihood method).

Homework Visit instructor’s web-page on Econometrics. www.alveronika.wordpress.com Homework “Autocorrelation” (exemplary exam test). Print out “Handout. Topic 5” for the next lesson.