1. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Impact and timing of revisions for seasonally adjusted series relative to those for the corresponding raw series Richard McKenzie, Zakia Adam & Michela Gamba OECD OECD / Eurostat taskforce on performing revisions analysis for sub-annual economic statistics

2. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Item VI of task-force Terms of Reference “….. be able to break down the results of a revisions analysis study to quantify the impact of different (homogenous groups of) sources of revision. As a minimum the guidelines must describe how the effects of seasonal adjustment on revisions can be separated and quantified in relation to other reasons.

3. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Description of OECD study Compare revisions analysis for raw and SA series for selected countries, IIP & Retail trade –Size of raw and SA revisions at different intervals –Occurrence of revisions to raw and SA series –Bias in revisions to raw series relative to SA series –Revisions of raw and SA series for year-on-year growth rates –Provide a tool which would allow countries to perform similar studies with their data

4. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Countries chosen Balance of EU and non-EU countries Include some countries where seasonal adjustment is performed by OECD (forward factor) Most importantly, need vintage series for all monthly snapshots with few missing updates –Last criteria presented the biggest problem due to lack of availability of vintages of raw series

5. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Index of industrial production United States, Japan, Korea, Belgium, Czech Republic, Finland, France, Netherlands, Poland Finland, Belgium, Netherlands – raw data is in fact working day adjusted Belgium, Czech Republic, Poland – SA done by OECD (forward factor method)

6. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Retail Trade Volume Canada, Japan, Korea, Mexico, Norway, Sweden SA done by OECD for Mexico

7. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Size of raw vs SA revisions for month- on-previous-month growth rates Consider relative absolute mean revision (RMAR) Review for different revision intervals (is the relationship stable across time?)

12. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Size of raw vs SA revisions for month- on-previous-month growth rates Revisions of raw consistently smaller than SA by similar magnitude for different revisions intervals –Ratio differs across countries, WDA is evident (larger revisions relative to true raw series) Similar picture for Retail Trade, but revisions to raw series even smaller in proportion of those to SA Overwhelming evidence that majority of revisions to SA series is from process of seasonal adjustment ….. But month on month growth rates of raw series are of little use to users / analysts …..

13. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Occurrence of revisions for raw and SA series How many months after first published values do we expect data to be revised? –Raw series should only be affected by late data, fixing errors etc. (especially if direct from one survey with no models etc.) –Revisions to SA series will depend on revisions to raw series and process of seasonal adjustment Revisions in longer term may also affect raw series (e.g. rebases, methodological changes, benchmarking etc.)

16. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Occurrence of revisions for raw and SA series IIP –occurrence of revisions similar for raw and SA for USA, Netherlands, Korea, Japan –But raw less often revised than SA for Belgium, Czech Republic, Finland, France and Poland Retail trade –Occurrence of revisions similar for Korea, raw a bit less for Japan and raw much less for Canada, Mexico, Norway & Sweden

19. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Occurrence of revisions: raw vs SA Some evidence that revisions to raw series less likely the further from first published data than for SA More evident for Retail than IIP and likelihood of revisions to raw for Retail trade is lower –Reflecting again that Retail trade data more likely to be compiled from single survey in most countries Further work needed to understand the impact SA method has on occurrence and size of revisions OECD tool now developed to break down revisions on a month by month basis

20. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Are revisions statistically significant? Ideally revisions should centre around zero over time (i.e. equally likely to be + or - with mean revision close to 0) –If this is not true then reasons causing this tendency in the compilation process should be found Should not expect a bias to exist in the raw series but not in the SA and vice versa

21. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Mean revision statistically significant for raw but not SA (& vice versa) IIP –Some evidence found for Belgium and Korea but very much on the margin Retail trade –Some evidence for Canada but mean revision still very small –For Korea, raw series mean revision significant (at 10% and 5% level) for all revisions intervals but not for SA >Mean revisions smaller but growing with length of interval (SA is in same direction and close to SIG)

22. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Which short term measures should we advise users to focus on? Are first estimates of month-on-previous-month growth rates of SA series for IIP and Retail Trade reliable enough to enable informed decision making? –Not in most countries! First estimates of MoM growth rates revised by 2/3 initial value on average after one year (95% countries revised by more than 2/5) –Shows the value that revisions analysis can provide to users Previous study suggested that users should focus on year- on-year growth rates for short-term analysis as these are more robust with respect to revisions

23. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Revisions to year-on-year growth rates Which series should year-on-year growth rates be calculated on? –OECD Data presentation handbook recommends YoY growth rates to be calculated from raw or WDA series if available What about robustness of raw / WDA and SA YoY growth rates with respect to revisions?

27. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Revisions to year-on-year growth rates: Conclusion / Future work Some weak evidence to suggest that YoY are more robust with respect to revisions for raw series compared to SA –Thus supports recommendations in the handbook –More empirical research would be interesting for users, particularly for WDA series

28. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Conclusions / Future work Relative mean absolute revisions (RMAR) for month-on-month growth rates are much lower for raw relative to SA series –Process of seasonal adjustment accounts for more than half of RMAR for the SA series, sometimes up to 90% and indeed still more than 50% in comparison to WDA series –…… but mean average revision is still generally higher for RAW series and month-on-month growth rates for raw series are not much use for short-term analysis

29. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Conclusions / Future work Likelihood of revisions to raw series (MoM growth rates) drops considerably after the first month but tends to stabilise at lower level in following months Likelihood of short-term revisions to SA series should depend on method used but more work needed to assess the relative impacts New OECD tool allows analysis of revisions to raw series on a month-on-month basis up to 6 months from first published value

30. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 Future work Disseminate results on OECD website, prepare paper and discuss implications for taskforce recommendations Use detailed revisions triangles to try and identify and quantify other reasons for revisions? OECD also conducting a study to look at the impact of improvements in timeliness on revisions

31. OECD Short-Term Economic Statistics Working PartyJune 25-27 2007 THE END