1. A comparison of automatic model selection procedures for seasonal adjustment Cathy Jones

2. Motivation Automatic forecasting procedures are required for seasonal adjustment of official statistics time series Current seasonal adjustment software offers two automatic procedures Aim of this work is to compare the performance of these procedures in the context of seasonal adjustment

3. Overview Time series background What is seasonal adjustment? X-13ARIMA-SEATS ARIMA models Forecasting Methods for automatic model selection Results Future work

4. Time Series Background Data source: ONS

5. Seasonal Adjustment What is seasonal adjustment? “process of removing from a time series variations associated with the time of year and/or the arrangement of the calendar” Why seasonally adjust? primary interest of users is movements in time series removes predictable variation from time series in order to aid interpretation

6. Seasonal Adjustment Data source: ONS

7. X-13ARIMA-SEATS Chosen as the seasonal adjustment software for use in official statistics (agreed by the Statistical Policy and Standards Committee in 2012) Produced by US Census Bureau RegARIMA modelling used to ‘clean’ series Can seasonally adjust with X-11 algorithm or SEATS

8. RegARIMA models Time series regression models with ARIMA errors used to deal with autocorrelation Used to forecast/backcast reduces revisions to seasonally adjusted series caused by asymmetric moving averages used in the X-11 method also used to forecast some components of National Accounts ‘Cleans’ the time series estimation and removal of outliers, level shifts etc

9. End point problem Data source: ONS

10. Forecasting Data source: ONS

11. Why selecting a good model is important Forecasts needed to deal with end point problem (asymmetric averages give an implied forecast) Good forecasts minimise revisions to the seasonally adjusted estimates Selection and estimation of regressors

12. Why is automatic model selection needed? ONS seasonally adjust thousands of time series -last year TSAB reviewed over 13,500 series Manual selection of ARIMA models is very time consuming- we’d never get through them all

13. Automatic model selection X-13ARIMA-SEATS provides two automatic model selection routines Pickmdl Automdl Pickmdl was used in older versions of the software (from X-11-ARIMA onwards) Automdl is based on routine from TRAMO (available from X-12-ARIMA onwards)

14. Pickmdl Chooses the first model in the following list that satisfies a number of tests (0,1,1)(0,1,1) s (0,1,2)(0,1,1) s (2,1,0)(0,1,1) s (0,2,2)(0,1,1) s (2,1,2)(0,1,1) s The tests are: testing that the average absolute percent error of forecasted values are within certain limits test that the residuals are not correlated no sign of over differencing

15. Automdl fits a (0,1,1)(0,1,1) s model identification of differencing orders by empirical unit root tests iterative procedure to determine ARMA model orders (maximum orders are set by default and can be changed) identified model is compared to default final model checks

16. Data Monthly GDP 176 monthly series 40 quarterly series Data spans from January 1995 to October 2014 Automatic detection of calendar effects and outliers Removed series that included seasonal breaks

17. Results Stability Forecast errors Forecast differences Differences in seasonal adjustment estimates

18. Stability of model selected 5 Years 7 years 9 years 11 years Full series........................

19. Stability of model selected 5 Years 7 years 9 years 11 years Full series........................

20. Stability of model selected 5 Years 7 years 9 years 11 years Full series........................

21. Stability of the model selected

22. Forecasts Data source: ONS

23. Average absolute percentage error in within-sample forecast values

24. Automdl lower 30% of the time Pickmdl lower 28% of the time Same 42% of the time On average, automdl's error is roughly 8% lower than pickmdl's

25. Average absolute percentage error in within-sample forecast values Last three years

26. Average absolute percentage error in within-sample forecast values Last three years Automdl lower 35% of the time Pickmdl lower 23% of the time Same 42% of the time On average, automdl's error is roughly 7% lower than pickmdl's

27. Forecast results F = Failures (%) AFD = Average Absolute Forecast Difference (%) BFP = Best Forecast Performance (%) LMV = Lowest Model Variance (%) Monthly AutomdlPickmdl F027.6 AFD11.54 BFP4753 LMV6136 Quarterly AutomdlPickmdl F05 AFD10.2112.07 BFP5842 LMV6832

28. Seasonal adjustment estimates Data source: ONS

29. Seasonal adjustment estimates Pickmdl produced better adjustment in 6% of cases Automdl was better in 33% of cases 41% were exactly the same 20% showed very little difference Data source: ONS

30. Revision analysis Span 1 Span 2 Span 3 Span 4 Span 5

31. Revision analysis Span 1 Span 2 Span 3 Span 4 Span 5

32. Revision analysis Span 1 Span 2 Span 3 Span 4 Span 5

33. Revision analysis Span 1 Span 2 Span 3 Span 4 Span 5

34. Seasonal adjustment revisions

35. Conclusions Pickmdl performs better when considering model stability Automdl has a lower average absolute percentage error in within-sample forecast values Out-of-sample forecasts performance: Pickmdl appears slightly better for monthly series Automdl appears much better for quarterly series Little difference in seasonal adjustment revisions between methods

36. Future work SEATS Series with differing volatility Simulated series How different model selection impacts on regressor identification and estimation Errors on shorter spans Sliding spans stability Using current regressors

37. Thanks for listening!