1. Lecture 8 ARIMA Forecasting II
BABS 502
Lecture 8
ARIMA Forecasting II

2. Content The Box-Jenkins Modeling Process Seasonal ARIMA Models
Concluding comments on ARIMA models
© Martin L. Puterman

3. Refitting if necessary Forecasting
The Box Jenkins Approach to forecasting with ARIMA models (Text Figure 8.10)
Identification
Fitting
Diagnostics
Refitting if necessary
Forecasting
© Martin L. Puterman

4. Identification What does the data look like? What patterns exist?
Is the data stationary?
Tools
Plots of data
PACF
ACF
© Martin L. Puterman

5. Model Fitting Propose trial model e.g. ARIMA(0,1,2)
Estimate model parameters using arima or (Arima) in R
See 8.7 in text for a discussion of the difference of these two functions
Obtain:
Parameter estimates
Test statistics
Goodness of fit measures
Residuals
Diagnostics
© Martin L. Puterman

6. Diagnostics Determines whether model fits data adequately.
The goal is to extract all information and ensure that residuals are white noise
Key measures
ACF of Residuals
PACF of Residuals
Ljung-Box-Pierce Q Statistic (Portmanteau Test)
Tests whether a set of residual autocorrelations is significantly different than zero.
Implemented with Box.test in R
If model deemed adequate, proceed with forecasting, otherwise try a new model.
© Martin L. Puterman

7. Comments on Model Adequacy Testing
From NCSS documentation: The Portmanteau Test (sometimes called the Box-Pierce-Ljung statistic) is used to determine if there is any pattern left in the residuals that may be modeled. This is accomplished by testing the significance of the autocorrelations up to a certain lag. In a private communication with Dr. Greta Ljung, we have learned that this test should only be used for lags between 13 and 24. The test is computed as :
where rj is the jth residual autocorrelation.
Under H0: All residual autocorrelations equal zero; Q(k) is distributed as a Chi-square with (K-p-q-P-Q) degrees of freedom where p,q,P and Q are the model orders.
© Martin L. Puterman

8. Forecasting with ARIMA models
ARIMA forecasting is done automatically in any statistical program.
You should try to figure out how the point forecasts is obtained through the equation for the model.
It helps to write out model equation
This is complicated with seasonal models, we will discuss this below.
In AR portion of models use past values in forecasts
In MA portion of models use past residuals in forecasts.
Prediction intervals are usually very wide; out of sample forecast errors might be more reliable.
© Martin L. Puterman

9. Google Share Price Forecasting
Series Monthly price (Jan 23, 2006 – March 3, 2008)
Model Regular(1,1,0) Seasonal (No seasonal parameters)
Observations 111
Root Mean Square
Model Estimation Section
Parameter Parameter Standard Prob
Name Estimate Error T-Value Level
AR(1)
Forecast of price
Row Date Forecast Lower Upper 95% Limit
Fitted Model
Xt+1-Xt = .238 (Xt – Xt-1) or Xt+1 = Xt (Xt – Xt-1)
One Step Ahead Forecast = * (432.7 – 471.2) = 423.5
© Martin L. Puterman

10. Seasonal ARIMA Models
The basic concept is to add extra terms to model that take into account a persistent seasonal pattern
For example, a AR model for monthly data may contain information from lag 12, lag 24, etc.
i.e. Yt = A1Yt-12 +A2 Yt et
This is referred to as an ARIMA(0,0,0)x(2,0,0)12 model
General form is ARIMA(p,d,q)x(ps,ds,qs)s
This combines both non-seasonal and seasonal terms
This provides a broader class of models.
The challenge is to select a model from a larger class.
© Martin L. Puterman

11. Wages Data
Observe data is non-stationary
© Martin L. Puterman

12. Differenced Wages Data
Autocorrelations of Wages (1,0,12,0,0)
Lag Correlation Lag Correlation Lag Correlation Lag Correlation
Significant if |Correlation|>
© Martin L. Puterman

13. Model Fitting ARIMA(0,1,3)x(0,0,1)12
Model Estimation Section
Parameter Standard Prob
Name Estimate Error T-Value Level
MA(1)
MA(2) E
MA(3)
SMA(1)
Ljung-Box Inadequate Model
© Martin L. Puterman

14. Model Fitting ARIMA(0,1,3)x(0,0,2)12
Model Estimation Section
Parameter Standard Prob
Name Estimate Error T-Value Level
MA(1)
MA(2) E
MA(3)
SMA(1)
SMA(2) E
© Martin L. Puterman

15. Model Fitting ARIMA(0,1,0)x(1,1,0)12
Model Estimation Section
Parameter Parameter Standard Prob
Name Estimate Error T-Value Level
SAR(1) E
© Martin L. Puterman

16. Model Comparison .0316 34.96 .0245 11.08 .0239 15.84 Model RMSE
Ljung-Box (24)
Residual ACF
(0,1,3)x(0,0,1)12
.0316
34.96
(0,1,3)x(0,0,2)12
.0245
11.08
(0,1,0)x(1,1,0)12
.0239
15.84
But we are concerned about forecasting and should compare models
out of sample (usually simpler models are better).
Also – forecasts from the last model looks most reasonable.
© Martin L. Puterman

17. Interpreting Seasonal Models
What does a ARIMA(1,0,0)x(1,0,0)12 model mean in terms of the data xt?
We use the backshift operator Bxt = xt-1, the identity operator Ixt = xt and the difference operator Dxt = (I – B)xt = xt – xt-1 to understand this.
An AR(1) model is written as
(I – a1B) xt = et
which becomes xt – a1xt-1 = et
which becomes xt = a1xt-1 + et
Note B2xt = B(Bxt) = Bxt-1 = xt-2.
An AR(2) model is written as
(I – a1B – a2B2) xt = et
An MA(1) model is written as
xt = (I – b1B) et = et – b1et-1
© Martin L. Puterman

18. What does a ARIMA(1,0,0)x(1,0,0)12 model mean in terms of the data xt?
It is written as: (I – a12B12)(I – a1B) xt = et Note the order of the terms on the left doesn’t matter. Above can be rewritten as (I – a1B)(I – a12B12) xt = et or (I – a1B - a12B12 + a1a12 B13) xt = et xt – a1 xt-1 – a12 xt-12 + a1a12xt-13 = et finally xt = a1 xt-1 + a12 xt-12 - a1a12xt-13 + et This is analogous to regressing xt on xt-1, xt-12, xt-13. And forecasts will be based on past or predicted values for these quantities.
© Martin L. Puterman

19. Concluding Comments
The ARIMA models are not designed for models with multiplicative seasonality. In such cases;
Use log or Box-Cox transforms.
De-seasonalize and use ARIMA on de-seasonalized data.
Models with persistent trends can be de-trended and ARIMA applied to the de-trended series.
Several automatic fitting programs do a good job fitting ARIMA models
Parsimony is desirable – use models with as few as terms as possible
AIC and BIC criterion penalize number of terms in the model
Theoretical result – any high order MA model can be written as a low order AR model and vice versa; e.g. an MA(6) can be closely represented by an AR(1) or AR(2) model
Key point – Above approach to model selection is based on in sample fitting
Need to compare all models on the basis of out-of sample forecasts on holdout data.
Simpler ARIMA models seem to work better out of sample even though they may not give the best fit.
Recall from early slides that fitting is different than forecasting.
ARIMA models forecasts can be pooled with those from one or more other models.
© Martin L. Puterman