1. BABS 502 Lecture 9 ARIMA Forecasting II March 23, 2009

2. © Martin L. Puterman 20082 Content The Box-Jenkins Modeling Process Seasonal ARIMA Models Concluding comments on ARIMA models

3. © Martin L. Puterman 20083 The Box Jenkins Approach to forecasting with ARIMA models Identification Fitting Diagnostics Refitting if necessary Forecasting

4. © Martin L. Puterman 20084 Identification What does the data look like? What patterns exist? –Is the data stationary? Tools –Plots of data –PACF –ACF

5. © Martin L. Puterman 20085 Model Fitting Trial model is proposed –e.g. ARIMA(0,1,2) Model parameters are estimated using statistical software –Output includes Parameter estimates Test statistics Goodness of fit measures Residuals Diagnostics

6. © Martin L. Puterman 20086 Diagnostics Determines whether model fits data adequately. –The goal is to extract all information and ensure that residuals are white noise Key measures –ACF of Residuals –PACF of Residuals –Ljung-Box-Pierce Q Statistic (Portmanteau Test) Tests whether a set of residual autocorrelations is significantly different than zero. See next slide for details If model deemed adequate, proceed with forecasting, otherwise try a new model.

7. © Martin L. Puterman 20087 Comments on Model Adequacy Testing (NCSS Documentation) The Portmanteau Test (sometimes called the Box-Pierce-Ljung statistic) is used to determine if there is any pattern left in the residuals that may be modeled. This is accomplished by testing the significance of the autocorrelations up to a certain lag. In a private communication with Dr. Greta Ljung, we have learned that this test should only be used for lags between 13 and 24. The test is computed as : where r j is the jth residual autocorrelation. Under H 0 : All residual autocorrelations equal zero; Q(k) is distributed as a Chi-square with (K-p-q-P-Q) degrees of freedom where p,q,P and Q are the model orders.

8. © Martin L. Puterman 20088 Forecasting with ARIMA models ARIMA forecasting is done automatically in any statistical program. You should try to figure out how this works in terms of the equation for the model. –It helps to write out model equation This is complicated with seasonal models Using the Backshift operator B k x t = x t-k helps. –In AR portion of models use past values in forecasts –In MA portion of models use past residuals in forecasts. Prediction intervals are usually very wide; out of sample forecast errors might be more reliable.

9. © Martin L. Puterman 20089 Google Share Price Forecasting SeriesMonthly price (Jan 23, 2006 – March 3, 2008) ModelRegular(1,1,0) Seasonal (No seasonal parameters) Observations111 Root Mean Square20.12511 Model Estimation Section ParameterParameterStandardProb NameEstimateErrorT-ValueLevel AR(1)0.2383635.0092.59780.009382 Forecast of price RowDateForecastLower Upper 95% Limit 1122094423.5360.7486.3 1132095421.3340.4502.3 1142096420.8324.8516.8 1152097420.7311.6529.8 1162098420.7299.9541.4 Fitted Model X t+1 -X t =.238 (X t – X t-1 ) or X t+1 = X t +.238 (X t – X t-1 ) One Step Ahead Forecast = 432.70 +.238 * (432.7 – 471.2) = 423.5

10. © Martin L. Puterman 200810 Seasonal ARIMA Models Basic concept is to add extra terms to model that take into account a persistent seasonal pattern For example, a AR model for monthly data may contain information from lag 12, lag 24, etc. –i.e. Y t = A 1 Y t-12 +A 2 Y t-24 + e t –This is referred to as an ARIMA(0,0,0)x(2,0,0) 12 model General form is ARIMA(p,d,q)x(ps,ds,qs) s –This combines both non-seasonal and seasonal terms This provides a broader class of models so the challenge is to select a model from a larger class.

11. © Martin L. Puterman 200811 Wages Data Observe data is non-stationary

12. © Martin L. Puterman 200812 Differenced Wages Data Autocorrelations of Wages (1,0,12,0,0) LagCorrelationLagCorrelationLagCorrelation LagCorrelation 1-0.055496110.062967210.200640310.013874 2-0.004269120.506937220.016009320.152615 30.29882613-0.056564230.008538330.077908 40.108858140.041622240.494130340.013874 50.073639150.28708625-0.024546350.118463 60.121665160.001067260.058698360.328709 70.048026170.088581270.20064037-0.086446 80.069370180.092850280.003202380.028815 90.218783190.010672290.036286390.205977 100.044824200.115261300.09178240-0.066169 Significant if |Correlation|> 0.237356

13. © Martin L. Puterman 200813 Model Fitting ARIMA(0,1,3)x(0,0,1) 12 Model Estimation Section ParameterStandardProb NameEstimateErrorT-ValueLevel MA(1)0.13900650.12007611.15770.247006 MA(2)1.547035E-020.12026380.12860.897645 MA(3)-0.20834030.1170662-1.77970.075128 SMA(1)-0.54271890.1019158-5.32520.000000 Ljung-Box 242034.960.020343Inadequate Model

14. © Martin L. Puterman 200814 Model Fitting ARIMA(0,1,3)x(0,0,2) 12 Model Estimation Section ParameterStandardProb NameEstimateErrorT-ValueLevel MA(1)0.21341330.11505361.85490.06361 MA(2)7.882232E-020.11616950.67850.49744 MA(3)-0.33586050.1119487-3.0000.00269 SMA(1)-0.42825750.1181367-3.6250.00028 SMA(2)-0.85555236.011709E-02-14.230.00000

15. © Martin L. Puterman 200815 Model Fitting ARIMA(0,1,0)x(1,1,0) 12 Model Estimation Section ParameterParameterStandardProb NameEstimateErrorT-ValueLevel SAR(1)-0.54955768.447082E-02-6.50590.000000

16. © Martin L. Puterman 200816 Model Comparison ModelRMSELjung- Box (24) Residual ACF (0,1,3)x(0,0,1) 12.031634.96 (0,1,3)x(0,0,2) 12.024511.08 (0,1,0)x(1,1,0) 12.023915.84 But we are concerned about forecasting and should compare models out of sample (usually simpler models are better). Also – forecasts from the last model looks most reasonable.

17. © Martin L. Puterman 200817 Concluding Comments The ARIMA models are not designed for models with multiplicative seasonality. In such cases; –Use log transforms. –De-seasonalize and use ARIMA on de-seasonalized data. Models with persistent trends can be de-trended and ARIMA applied to the de- trended series. Several automatic fitting programs do a good job fitting ARIMA models Parsimony is desirable – use models with as few as terms as possible –AIC and BIC criterion penalize number of terms in the model –Theoretical result – any high order MA model can be written as a low order AR model and vice versa; e.g. an MA(6) can be closely represented by an AR(1) or AR(2) model Key point – Above approach to model selection is based on in sample fitting Need to compare all models on the basis of out-of sample forecasts on holdout data. –Simpler ARIMA models seem to work better out of sample even though they may not give the best fit. –Recall from early slides that fitting is different than forecasting. ARIMA models forecast can be pooled with those from other models.