Eurostat Seasonal Adjustment

Topics Motivation and theoretical background (Øyvind Langsrud) Seasonal adjustment step-by-step (László Sajtos) (A few) issues on seasonal adjustment (László Sajtos)

Presented by Øyvind Langsrud Statistics Norway

Time series with seasonal and non-seasonal variation

Removing the seasonal variation

Removing also the non-seasonal variation

Monthly time series example Trend and seasonality can be seen –How to find it by computation?

Quick and dirty calculation of trend by ordinary linear regression: y = a + b*time + e time = , , , , , , , , , , , , , , …... a = b =

Including seasonality in "the dirty model" y = a + b*time + c month + e

a = b = c = mnd0 mnd2 mnd3 mnd4 mnd5 mnd mnd7 mnd8 mnd9 mnd10 mnd11 mnd Transforming to seasonal adjustment language a + b*time → T t c month → S t e → I t y t = T t + S t + I t

Trend from "the dirty model" y t = T t + S t + I t

Seasonality from "the dirty model" y t = T t + S t + I t

Seasonal adjustment by "the dirty model" y t = T t + S t + I t

Question to the audience: What is wrong with this ordinary regression approach ?

Irregular component by "the dirty model" y t = T t + S t + I t

In practise a multiplicative model is used: y t = T t × S t × I t y t is not the original series but a series that is corrected for holiday and trading day effects (calendar adjusted) y t = T t × S t × I t

Note that the seasonal factors vary slightly along time

y t = T t × S t × I t This time the irregular component looks more as true noise Note that correlated neighbour values is allowed (autocorrelation)

y t = T t × S t × I t This is seasonally adjusted data as published by Statistics Norway

Multiplicative model: y t = T t × S t × I t Additive model: y t = T t + S t + I t How to calculate T t, S t, and I t from y t ? This is done by filtering techniques –One element of this methodology is how to calculate the trend from seasonally adjusted data –This is a question of smoothing a noisy series

Smoothing by averaging P t = (Y t-1 + Y t + Y t+1 )/3

Also called filtering P t = (Y t-2 + Y t-1 + Y t + Y t+1 + Y t+2 )/5 The filter is [1,1,1,1,1]/5

Here the filter length is 9

Filtering can be performed twice 3x3 filter –3-term moving average of a 3-term moving average –The final filter is [1,2,3,2,1]/9 – P t = (Y t-2 + 2Y t-1 + 3Y t + 2Y t+1 + Y t+2 )/9 2x12 filter –[1/2,1,1,1,1,1,1,1,1,1,1,1,1/2]/12 –Also called a centred 12-term moving average –Question to the audience:  Why is this filter of special interest?

Henderson filters Finding filters with good properties is an interesting topic … Hederson (1916) introduces the so-called Henderson filters X-12-ARIMA uses this type of filter to calculate the trend The filter length determines the degree of smoothing

Question to the audience: Why does the filtered series stop in 2009?

Non-available observations at the end: Two solutions Asymmetric filters –Asymmetric variant of Henderson  [-0.034,0.116,0.383,0.534,0,0,0]  Can be used at the last observation Forecasts in place of the unobserved values –The “starting series” for the X12-ARIMA decompositions is a calendar adjusted series which is based on reg-ARIMA modelling –The reg-ARIMA modelling can also be used to produced forecasts – X12-ARIMA uses these forecasts in trend calculations

Finding the seasonal component by filtering From a series with the trend removed we make 12 series –January-values, February-values, … Each of these series is smoothed by filtering Altogether these smoothed series are the seasonal component

The X12-ARIMA algorithm The decomposition is made by several iterative steps –Seasonal component from series with trend removed –Trend from series with seasonal component removed Initial estimate of trend using the 2x12 moving average One element is downweighting of observations with an extreme irregular component

X12-ARIMA or SEATS Both method can be viewed as filtering techniques X12-ARIMA –A non-parametric method –No model assumed SEATS –The components are assumed to follow ARIMA models –The filters are derived from modelling –Possible to do inference and to make forecasts with confidence intervals –So why the name X12-ARIMA when this method is the one that is not based on ARIMA?  Answer on the next slide

Calendar adjustment by reg-ARIMA modelling Seasonal ARIMA model –Correlated errors (autocorrelation) –Differencing the series makes the model quite good without explicit parameters for trend and seasonality –Need to decide the type of ARIMA model: ARIMA(p,d,q)(P,D,Q) Regression parameters in the model –Calendar effects: Trading day, Moving holyday, … –Outliers and level shifts Here y can be a log-transformed and leap-year adjusted variant of the original data "The dirty model" mentioned earlier:

 This slide is “stolen” from  Here B is the backshift operator: BY t =Y t-1  ARIMA(0,1,1)(0,1,1)  Most common model  Airline model

Example of regression variables in reg-ARIMA modelling Easter –2000 and 2001: Easter in April –2008: Easter in March –2002: 4 of 5 Norwegian Easter days in March Trading day –Six parameters needed to model seven days –Mon: Number of Mondays minus Number of Sundays Easter Mon Tue Wed Thu Fri Sat Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May Jun Jul Aug Sep Oct Nov Dec Jan Feb Mar Apr May : : : Mar Apr May Jun Jul Aug Sep Oct Nov Dec

Trading day: Separate effect of each day or common effect of all weekdays? Question to the audience: –Why exactly equal t-values? Regression Model Parameter Standard Variable Estimate Error t-value Trading Day Mon Tue Wed Thu Fri Sat *Sun (derived) Regression Model Parameter Standard Variable Estimate Error t-value Trading Day Weekday **Sat/Sun (derived)

Outliers An extreme observation caused by a special event can be problematic –Can influence the modelling in a negative way  Parameter estimates  Forecasts  Decomposition Solution –Include the outlier as a dummy variable in the reg-ARIMA modelling  ….0,0,0,0,0,0,0,0,0,1,0,0,0,0,0,0,0,0…. –The outlier is included in the irregular component after modelling  The observation is still included in seasonally adjusted data  But has no effect on the trend  Question to the audience: Examples of special events?

Level shift is handled similar to outliers –Regression variable: ….0,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1…. –Level shift is included in the trend

Presented by László Sajtos Hungarian Central Statistical Office

Topics Seasonal adjustment step-by-step (A few) issues on seasonal adjustment

Seasonal adjustment step-by-step

Seasonal adjustment step-by-step: structure Input data STEPS with check points Preliminary results Output data If results are acceptable Not acceptable results

Basic conditions Length of time series (enough long to be seasonally adjusted?)  Monthly datasets: at least 3-year long  Quarterly datasets: at least 4-year long At least 5-7-year long time series is optimal! Expert information Collecting expert data from the sections about datasets (potential outliers, methodological changes, changes in exterior factors (e.g. law), connections to other time series and sectors) Time series analysis (STEP 0)

Graphical analysis via basic and sophisticated graphs Plotted raw dataset Spectral analysis: autocorrelogram and auto-regressive spectrum Identifying and explaining missing observations and outliers Correction of data faults Test for seasonality Graphical analysis, test for seasonality (STEP 1)

Seasonality Seems additive Data : Hungarian monthly retail volume index, food Probably outliers Graphical analysis, an example ( )

Automatic test Graphical analysis Software tools Verification Type of transformation (STEP 2)

Determining factors which may affect (regressors)+national holidays Non-significance or absence Little significance Keep Significance Elimination Consideration based on professional reasons Elimination Calendar adjustment (STEP 3)

Outlier treatment (Step 4) Automatic outlier testing Software tools Verifying the results STEP 1 Keep it Significant Monitoring Stability Available expert information Less significant, but professionally reasonable Not significant Eliminate it Consideration based on professional reasons

Airline model Software tools Not satisfying results Good results Keep model Manual settings Automatic choice recommended Other low ordered models Reducing the order of the model ARIMA model (Step 5)

Decomposition (Step 6) Software tools Eliminating deterministic effects Decomposition Multiplicative Log-additiv e Additive

Quality diagnostics (Step 7) 1.Model adequacy on residuals: Ljung-Box test Box-Pierce test 2.Seasonality: based on spectral graphics 3. Stability analysis: sliding spans Documentation required!

Manual settings (Step 8) In case of: Detailed analysis Quality diagnostics are not auspicious Further outlier correction Other advanced settings (e.g. confidence intervals) Manual settings Quality diagnostics Dissemination satisfying Manual settings not (STEP 9)

EXAMPLE (IN DEMETRA 2.04 SOFTWARE) HUNGARIAN INDUSTRIAL TIME SERIES

Automated module

Open the input database

The list of time series

Selection of time series output

Save of output

Diagnostic, outlier %

Adjustment without fixed models

Setting the method and trading day regressor

Setting the country specific holidays

The results Manual settings required Quality diagnostics

(A few) issues on seasonal adjustment

Issues in Memobust book Consistency issuesData presentation RevisionIssues on chained indices Treatment of the crisisDocumentation Communication with users

Revision SA data Unadjusted data Reasons: Data arrival after deadline Erroneous data etc. What to do: Data review Reasons : New information are available Better estimation required. What to do: Estimating new model, new seasonal factors

Revision strategies Goal: preserving accuracy, taking new information into consideration while avoiding large changes reliability and stability Strategies: Extreme types Current Concurrent Alternative types Partial concurrent Controlled current Extreme types Alternative types

Horizon of revision Practices: ESS Guideline: 3-4 years before the beginning of the revision period Statistics Denmark: at least 13 months back in time Question: How many months of data should be revised?

Consistency issues Issues Linkages in economy and among time series;expectations of users; errors; etc. Temporal constraints E.g.Annual and infra-annual series Cross-sectional constraints E.g.Total industrial and segmental series Time consistency issue Aggregation consistency issue

Time consistency issues

Benchmarking Benchmark: typically annual data Aim: Providing time consistency, the techniques operate with the sum of modified sub-annual series Benchmarking Pro-rating method Denton method

Pro-rating method

Denton method How it works: Based on quadratic optimalization Advantages: The method can be developed, specificated More reliable results (smaller discontinuities compared with pro-rating)

Aggregation consistency Indirect SA Direct SA

Methods to achieve aggregation consistency Only direct or indirect seasonal adjustment Pro-rating Denton method Regression based models

Thank you for your attention! Questions?