1. Regression with Autocorrelated Errors
Let’s look at the relationship between two U.S. weekly interest rate series measured in percentages
r1(t) = The 1-year Treasury constant maturity rate
r3(t) = The 3-year Treasury constant maturity rate
From 1/5/1962 to 9/10/1999.
Consider the regression model for studying the structure of interest rates
r3(t)=a+br1(t)+e(t)
Spring 2004
K. Ensor, STAT 421

2. Spring 2004
K. Ensor, STAT 421

3. The two series are highly correlated.
Scatterplots between series simultaneous in time, and the change in each series.
The two series are highly correlated.
Spring 2004
K. Ensor, STAT 421

4. Try this model?
Splus summary results of ordinary least squares fit of r3(t)=a + b r1(t) + e(t)
Residual Standard Error = 0.538,
Multiple R-Square =
N = 1967, F-statistic = on 1 and 1965 df,
p-value = 0
coef std.err t.stat p.value
Inter X
Spring 2004
K. Ensor, STAT 421

5. Behavior of Residuals The residuals are nonstionary. Spring 2004
K. Ensor, STAT 421

6. Next step-
From a regression perspective the assumptions of our regression model are violated – long memory / nonstationarity of the residuals.
Let’s consider the change series of interest rates
c1(t)=(1-B)r11(t)
c3(t)=(1-B)r3(t)
Now regress c3 on c1.
Spring 2004
K. Ensor, STAT 421

7. Spring 2004
K. Ensor, STAT 421

8. Spring 2004
K. Ensor, STAT 421

9. c3(t)=a + b c1(t) + e(t) Regression results OLS fit of
Residual Standard Error = ,
Multiple R-Square =
N = 1966, F-statistic = on 1 and 1964 df,
p-value = 0
coef std.err t.stat p.value
Intercept X
Spring 2004
K. Ensor, STAT 421

10. Looking at the Residuals
There is a small bit (very small bit) of autocorrelation – violating our regression assumptions.
Spring 2004
K. Ensor, STAT 421

11. How to proceed?
The residuals from the regression fit exhibit dependence over the time lags.
Identify the time series model.
Refit Regression + time series model using MLE.
Spring 2004
K. Ensor, STAT 421

12. The correct model c3(t)=a + b c1(t) + e(t) with e(t) = a(t) + q a(t-1)
Parameter estimates: (a,b,q,s2)
Standard errors: ( , , )
R-squared = 85.4%
Spring 2004
K. Ensor, STAT 421

13. Regression + MA diagnostics
WE are ignoring the
Structure shown in the
Partials.
Spring 2004
K. Ensor, STAT 421

14. Summary – Regression with autocorrelated errors
Fit regression model
Check residuals for the presence of autocorrelation.
Rather than use the Durbin-Watson statistic commonly recommended in regression texts we use the more general Box-Pierce test.
If autocorrelation is present, identify the nature of the autocorrelation and simultaneously fit (via MLE) the regression parameters and the time series parameters (in Splus use arima.mle)
Note – if your software does not simultaneously fit the parameters and you do not have time to write your own code the interate the fitting of the TS parameters and regression parameters until convergence is reached.
Spring 2004
K. Ensor, STAT 421

15. Long memory processes No memory – ACF is zero
Short memory – stationary
exponential decay in the ACF
Non stationary – random walk
For any fixed lag the ACF will converge to one as the sample size increases to infinity
You will still observe a decay in the sample ACF because we do not have an infinite sample.
How would you design a simulation study to demonstrate this property?
Long memory – stationary
The “differencing” exponent is between –½ and ½.
Spring 2004
K. Ensor, STAT 421

16. Properties of a long memory process
If d<.5 – weakly stationary and has an infinite MA average
If d>-.5, then the process can be written as in infinite order AR
The ACF has a nice functional form with the first autocorrelation being d/(1-d).
Note the decay rate – page 73
Polynomial decay rate for d<.5 (not exponential)
The partial acf at lag k is given by d/(k-d)
A process is a fractional ARMA process (ARFIMA) if the fractionally differenced series follows an ARMA(p,q) process.
Spring 2004
K. Ensor, STAT 421