1. Eurostat Seasonal Adjustment

2. Topics Motivation and theoretical background (Øyvind Langsrud) Seasonal adjustment step-by-step (László Sajtos) (A few) issues on seasonal adjustment (László Sajtos)

3. Presented by Øyvind Langsrud Statistics Norway

4. Time series with seasonal and non-seasonal variation

5. Removing the seasonal variation

6. Removing also the non-seasonal variation

7. Monthly time series example Trend and seasonality can be seen –How to find it by computation?

8. Quick and dirty calculation of trend by ordinary linear regression: y = a + b*time + e time = 2000.000, 2000.083, 2000.167, 2000.250, 2000.333, 2000.417, 2000.500, 2000.583, 2000.667, 2000.750, 2000.833, 2000.917, 2001.000, 2001.083, …... a = -6619.731 b = 3.351223

9. Including seasonality in "the dirty model" y = a + b*time + c month + e

10. a = -6468.505 b = 3.275956 c = mnd0 mnd2 mnd3 mnd4 mnd5 mnd6 -9.19620250 -16.59062737 -6.79790939 -8.51090569 -1.18890200 6.33881598 mnd7 mnd8 mnd9 mnd10 mnd11 mnd12 1.84439111 4.62139480 -2.56494236 -0.04409251 1.53598811 30.55299181 Transforming to seasonal adjustment language a + b*time → T t c month → S t e → I t y t = T t + S t + I t

11. Trend from "the dirty model" y t = T t + S t + I t

12. Seasonality from "the dirty model" y t = T t + S t + I t

13. Seasonal adjustment by "the dirty model" y t = T t + S t + I t

14. Question to the audience: What is wrong with this ordinary regression approach ?

15. Irregular component by "the dirty model" y t = T t + S t + I t

16. In practise a multiplicative model is used: y t = T t × S t × I t y t is not the original series but a series that is corrected for holiday and trading day effects (calendar adjusted) y t = T t × S t × I t

17. Note that the seasonal factors vary slightly along time

18. y t = T t × S t × I t This time the irregular component looks more as true noise Note that correlated neighbour values is allowed (autocorrelation)

19. y t = T t × S t × I t This is seasonally adjusted data as published by Statistics Norway

20. Multiplicative model: y t = T t × S t × I t Additive model: y t = T t + S t + I t How to calculate T t, S t, and I t from y t ? This is done by filtering techniques –One element of this methodology is how to calculate the trend from seasonally adjusted data –This is a question of smoothing a noisy series

21. 2000-2014

22. 2007-2012

23. Smoothing by averaging P t = (Y t-1 + Y t + Y t+1 )/3

24. Also called filtering P t = (Y t-2 + Y t-1 + Y t + Y t+1 + Y t+2 )/5 The filter is [1,1,1,1,1]/5

25. Here the filter length is 9

26. Filtering can be performed twice 3x3 filter –3-term moving average of a 3-term moving average –The final filter is [1,2,3,2,1]/9 – P t = (Y t-2 + 2Y t-1 + 3Y t + 2Y t+1 + Y t+2 )/9 2x12 filter –[1/2,1,1,1,1,1,1,1,1,1,1,1,1/2]/12 –Also called a centred 12-term moving average –Question to the audience:  Why is this filter of special interest?

27. Henderson filters Finding filters with good properties is an interesting topic … Hederson (1916) introduces the so-called Henderson filters X-12-ARIMA uses this type of filter to calculate the trend The filter length determines the degree of smoothing

32. Question to the audience: Why does the filtered series stop in 2009?

33. Non-available observations at the end: Two solutions Asymmetric filters –Asymmetric variant of Henderson  [-0.034,0.116,0.383,0.534,0,0,0]  Can be used at the last observation Forecasts in place of the unobserved values –The “starting series” for the X12-ARIMA decompositions is a calendar adjusted series which is based on reg-ARIMA modelling –The reg-ARIMA modelling can also be used to produced forecasts – X12-ARIMA uses these forecasts in trend calculations

34. Finding the seasonal component by filtering From a series with the trend removed we make 12 series –January-values, February-values, … Each of these series is smoothed by filtering Altogether these smoothed series are the seasonal component

35. The X12-ARIMA algorithm The decomposition is made by several iterative steps –Seasonal component from series with trend removed –Trend from series with seasonal component removed Initial estimate of trend using the 2x12 moving average One element is downweighting of observations with an extreme irregular component

36. X12-ARIMA or SEATS Both method can be viewed as filtering techniques X12-ARIMA –A non-parametric method –No model assumed SEATS –The components are assumed to follow ARIMA models –The filters are derived from modelling –Possible to do inference and to make forecasts with confidence intervals –So why the name X12-ARIMA when this method is the one that is not based on ARIMA?  Answer on the next slide

37. Calendar adjustment by reg-ARIMA modelling Seasonal ARIMA model –Correlated errors (autocorrelation) –Differencing the series makes the model quite good without explicit parameters for trend and seasonality –Need to decide the type of ARIMA model: ARIMA(p,d,q)(P,D,Q) Regression parameters in the model –Calendar effects: Trading day, Moving holyday, … –Outliers and level shifts Here y can be a log-transformed and leap-year adjusted variant of the original data "The dirty model" mentioned earlier:

38.  This slide is “stolen” from https://www.scss.tcd.ie/Rozenn.Dahyot/ST7005/15SeasonalARIMA.pdf  Here B is the backshift operator: BY t =Y t-1  ARIMA(0,1,1)(0,1,1)  Most common model  Airline model

39. Example of regression variables in reg-ARIMA modelling Easter –2000 and 2001: Easter in April –2008: Easter in March –2002: 4 of 5 Norwegian Easter days in March Trading day –Six parameters needed to model seven days –Mon: Number of Mondays minus Number of Sundays Easter Mon Tue Wed Thu Fri Sat Jan 2000 0.0000000 0 -1 -1 -1 -1 0 Feb 2000 0.0000000 0 1 0 0 0 0 Mar 2000 -0.2571429 0 0 1 1 1 0 Apr 2000 0.2571429 -1 -1 -1 -1 -1 0 May 2000 0.0000000 1 1 1 0 0 0 Jun 2000 0.0000000 0 0 0 1 1 0 Jul 2000 0.0000000 0 -1 -1 -1 -1 0 Aug 2000 0.0000000 0 1 1 1 0 0 Sep 2000 0.0000000 0 0 0 0 1 1 Oct 2000 0.0000000 0 0 -1 -1 -1 -1 Nov 2000 0.0000000 0 0 1 1 0 0 Dec 2000 0.0000000 -1 -1 -1 -1 0 0 Jan 2001 0.0000000 1 1 1 0 0 0 Feb 2001 0.0000000 0 0 0 0 0 0 Mar 2001 -0.2571429 0 0 0 1 1 1 Apr 2001 0.2571429 0 -1 -1 -1 -1 -1 May 2001 0.0000000 0 1 1 1 0 0 Jun 2001 0.0000000 0 0 0 0 1 1 Jul 2001 0.0000000 0 0 -1 -1 -1 -1 Aug 2001 0.0000000 0 0 1 1 1 0 Sep 2001 0.0000000 -1 -1 -1 -1 -1 0 Oct 2001 0.0000000 1 1 1 0 0 0 Nov 2001 0.0000000 0 0 0 1 1 0 Dec 2001 0.0000000 0 -1 -1 -1 -1 0 Jan 2002 0.0000000 0 1 1 1 0 0 Feb 2002 0.0000000 0 0 0 0 0 0 Mar 2002 0.5428571 -1 -1 -1 -1 0 0 Apr 2002 -0.5428571 1 1 0 0 0 0 May 2002 0.0000000 0 0 1 1 1 0 : : : Mar 2008 0.7428571 0 -1 -1 -1 -1 0 Apr 2008 -0.7428571 0 1 1 0 0 0 May 2008 0.0000000 0 0 0 1 1 1 Jun 2008 0.0000000 0 -1 -1 -1 -1 -1 Jul 2008 0.0000000 0 1 1 1 0 0 Aug 2008 0.0000000 -1 -1 -1 -1 0 0 Sep 2008 0.0000000 1 1 0 0 0 0 Oct 2008 0.0000000 0 0 1 1 1 0 Nov 2008 0.0000000 -1 -1 -1 -1 -1 0 Dec 2008 0.0000000 1 1 1 0 0 0

40. Trading day: Separate effect of each day or common effect of all weekdays? Question to the audience: –Why exactly equal t-values? Regression Model -------------------------------------------------------------- Parameter Standard Variable Estimate Error t-value -------------------------------------------------------------- Trading Day Mon -0.0019 0.00193 -1.00 Tue 0.0064 0.00194 3.31 Wed 0.0018 0.00190 0.94 Thu -0.0016 0.00195 -0.81 Fri 0.0138 0.00188 7.37 Sat 0.0034 0.00193 1.73 *Sun (derived) -0.0219 0.00196 -11.16 Regression Model -------------------------------------------------------------- Parameter Standard Variable Estimate Error t-value -------------------------------------------------------------- Trading Day Weekday 0.0036 0.00053 6.87 **Sat/Sun (derived) -0.0090 0.00131 -6.87

41. Outliers An extreme observation caused by a special event can be problematic –Can influence the modelling in a negative way  Parameter estimates  Forecasts  Decomposition Solution –Include the outlier as a dummy variable in the reg-ARIMA modelling  ….0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0…. –The outlier is included in the irregular component after modelling  The observation is still included in seasonally adjusted data  But has no effect on the trend  Question to the audience: Examples of special events?

43. Level shift is handled similar to outliers –Regression variable: ….0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1…. –Level shift is included in the trend

44. Presented by László Sajtos Hungarian Central Statistical Office

45. Topics Seasonal adjustment step-by-step (A few) issues on seasonal adjustment

46. Seasonal adjustment step-by-step

47. Seasonal adjustment step-by-step: structure Input data STEPS with check points Preliminary results Output data If results are acceptable Not acceptable results

48. Basic conditions Length of time series (enough long to be seasonally adjusted?)  Monthly datasets: at least 3-year long  Quarterly datasets: at least 4-year long At least 5-7-year long time series is optimal! Expert information Collecting expert data from the sections about datasets (potential outliers, methodological changes, changes in exterior factors (e.g. law), connections to other time series and sectors) Time series analysis (STEP 0)

49. Graphical analysis via basic and sophisticated graphs Plotted raw dataset Spectral analysis: autocorrelogram and auto-regressive spectrum Identifying and explaining missing observations and outliers Correction of data faults Test for seasonality Graphical analysis, test for seasonality (STEP 1)

50. Seasonality Seems additive Data : Hungarian monthly retail volume index, food Probably outliers Graphical analysis, an example (2000-2013)

51. Automatic test Graphical analysis Software tools Verification Type of transformation (STEP 2)

52. Determining factors which may affect (regressors)+national holidays Non-significance or absence Little significance Keep Significance Elimination Consideration based on professional reasons Elimination Calendar adjustment (STEP 3)

53. Outlier treatment (Step 4) Automatic outlier testing Software tools Verifying the results STEP 1 Keep it Significant Monitoring Stability Available expert information Less significant, but professionally reasonable Not significant Eliminate it Consideration based on professional reasons

54. Airline model Software tools Not satisfying results Good results Keep model Manual settings Automatic choice recommended Other low ordered models Reducing the order of the model ARIMA model (Step 5)

55. Decomposition (Step 6) Software tools Eliminating deterministic effects Decomposition Multiplicative Log-additiv e Additive

56. Quality diagnostics (Step 7) 1.Model adequacy on residuals: Ljung-Box test Box-Pierce test 2.Seasonality: based on spectral graphics 3. Stability analysis: sliding spans Documentation required!

57. Manual settings (Step 8) In case of: Detailed analysis Quality diagnostics are not auspicious Further outlier correction Other advanced settings (e.g. confidence intervals) Manual settings Quality diagnostics Dissemination satisfying Manual settings not (STEP 9)

58. EXAMPLE (IN DEMETRA 2.04 SOFTWARE) HUNGARIAN INDUSTRIAL TIME SERIES

59. Automated module

60. Open the input database

61. The list of time series

62. Selection of time series output

63. Save of output

64. Diagnostic, outlier %

65. Adjustment without fixed models

66. Setting the method and trading day regressor

67. Setting the country specific holidays

68. The results Manual settings required Quality diagnostics

69. (A few) issues on seasonal adjustment

70. Issues in Memobust book Consistency issuesData presentation RevisionIssues on chained indices Treatment of the crisisDocumentation Communication with users

71. Revision SA data Unadjusted data Reasons: Data arrival after deadline Erroneous data etc. What to do: Data review Reasons : New information are available Better estimation required. What to do: Estimating new model, new seasonal factors

72. Revision strategies Goal: preserving accuracy, taking new information into consideration while avoiding large changes reliability and stability Strategies: Extreme types Current Concurrent Alternative types Partial concurrent Controlled current Extreme types Alternative types

73. Horizon of revision Practices: ESS Guideline: 3-4 years before the beginning of the revision period Statistics Denmark: at least 13 months back in time Question: How many months of data should be revised?

74. Consistency issues Issues Linkages in economy and among time series;expectations of users; errors; etc. Temporal constraints E.g.Annual and infra-annual series Cross-sectional constraints E.g.Total industrial and segmental series Time consistency issue Aggregation consistency issue

75. Time consistency issues

76. Benchmarking Benchmark: typically annual data Aim: Providing time consistency, the techniques operate with the sum of modified sub-annual series Benchmarking Pro-rating method Denton method

77. Pro-rating method

78. Denton method How it works: Based on quadratic optimalization Advantages: The method can be developed, specificated More reliable results (smaller discontinuities compared with pro-rating)

79. Aggregation consistency Indirect SA Direct SA

80. Methods to achieve aggregation consistency Only direct or indirect seasonal adjustment Pro-rating Denton method Regression based models

81. Thank you for your attention! Questions?