1. Eurostat Enhancing dissemination: aspects related to sets of policy indicators ESTP course - MIP Luxembourg 1-3 December 2015 R. Ruggeri Cannata

2. Eurostat Outline Release policy Revision analysis Data coverage

3. Eurostat Looking at the set of indicators: release policies ESS guidelines on revisions policy for PEEIs, Annex: Principles for a common revision policy for European Statistics Principle 4:Routine and annual revisions Routine and annual revisions should be published in the framework of well defined, synchronised and regularly updated release/revision calendars at national and European level. Releases of European and national data aggregates should be synchronized as far as possible.

4. Eurostat Looking at the set of indicators: data from different domains Impact on the MIP set of indicators No aggregate (EU and EA) Different domains: different release policies FA: always up to date GGGD: as published in the EDP notification

5. Eurostat Looking at the set of indicators: release policies – the issue of ratios For most of the MIP indicators: numerator and denominator stemming from different domains (% GDP, deflator) Denominator value is taken at the date of the release of the numerator (Numerator policy) Ratio updated whenever numerator or denominator is updated (Freshest information policy) Values are "fixed" at the moment of the official release (EDP policy) Numerator's domain policy

6. Eurostat Revisions

7. Eurostat Harmonization of revision policies: CoP Principle 6 "Impartiality and objectivity" 6.6: Advance notice is given on major revisions or changes in methodologies Principle 8 "Appropriate Statistical Procedures" 8.6: Revisions follow standard, well-established and transparent procedures Principle 12 "Accuracy and Reliability" 12.3: Revisions are regularly analysed in order to improve statistical processes

8. Eurostat ESS guidelines on revision policy for PEEIs The definition of a revision policy at ESS level is an essential step towards further harmonisation of infra-annual statistics, especially Principal European Economic Indicators (PEEIs) It follows the vision of an integrated European statistical system where the releases and the revisions of European statistics are coordinated and synchronised as far as possible, and transparency for the user is maximised via clear communication of revision policies and practices

9. Eurostat Are revision good or bad? Revisions have to be considered a normal phenomenon to increase progressively the quality and in particular the accuracy of the data. Revision policy should be recognized as an important aspect of good governance in statistics Good governance in statistics, in turn, is part of public sector transparency and accountability more broadly

10. Eurostat Definition of revisions Revisions are broadly defined as any change in a value of a statistic released to the public. They can occur either when new observations become available and some past values are modified or when the current and/or some past values are modified Data are generally revised in order to incorporate new, improved information. Therefore, revisions are inevitable whenever statistics are produced that report promptly on economic developments despite the fact that some relevant information is still outstanding

11. Eurostat Should a revision policy be publicly available? A public revision policy gives users confidence, providing in advance explanations for the reasons for revisions and the nature of them An ESS revision policy facilitates the comparison of practices adopted among domains and countries It promotes a common language when discussing revisions, also improves, as a matter of fact, the quality of the documentation Common policy action among member countries promotes consistency of data at the national and international levels

12. Eurostat Revision analysis for data users Aspects important for the evaluation of revision impact on the overall data quality: Accuracy Reliability Stability number of revisions within a given unit of time

13. Eurostat Is revision analysis a tool for the users only? Producers use revision analysis (size and direction) to identify the presence of problems in estimations and, where possible, to modify the statistical production process Producers use revision studies to identify particular methods that could be improved or in order to achieve greater accuracy or timeliness

14. Eurostat Revision analysis for data producers Revision analysis for data producers should tackle the problem of the consequences of the revision process on the quality of aggregated data It can be applied to tracking potential sources of systematic data anomalies and identifying areas of data collection and compilation process which should be improved

15. Eurostat Alternative views on vintages Compare a vintage to another and summarize the differences between the two columns (“vertical” or “classical” view) Look to the revisions of a specific reference period (“horizontal view”) Focus on the diagonals looking for example to the differences between the first release and the second release

16. Eurostat Descriptive revision analysis MAR: measures the average size of the revisions without providing an indication of directional bias Associated with a 90% range Arithmetic average (or revisions mean) reveals whether revisions are systematic or not and gives an indication on the average level of revision being close to zero Positive mean may indicate that on average earlier releases have been underestimated Test for significance

17. Eurostat Descriptive revision analysis - cont. RMAR useful when making comparisons in the size of revision across different indicators Revisions Variability: standard deviation of revisions as indicator of revisions volatility minimum and maximum revision

18. Eurostat Descriptive revision analysis - cont. News or noise: Revisions add new available information (news) Revisions arise because of measurement errors and inefficiencies in the preliminary estimates (noise) Testing the correlation between R t = L t - P t and the estimates L t and P t Noise: R t are significantly correlated with P t and uncorrelated with L t News: R t are significantly correlated with L t and uncorrelated with P t

19. Eurostat Revision analysis to improve the production process Hoven (2008): Dutch estimates of GDP volume growth Ciammola et al. (2008): Italian index of industrial production Both propose a top-down approach to identify the specific area(s)/domain(s) “responsible” for large or biased revisions

20. Eurostat Looking at set of indicators: revision analysis Basic/ core measures: targeting users that require quick, easy to understand information Additional/ advanced measures: targeting users that require more in-depth analysis Sophisticated / special user measures: information for detailed research purposes

21. Eurostat Summary of statistics for analysis of revisions (1) 1.Mean absolute revision 2.Median absolute revision 3.Arithmetic average or mean revision 4.Statistical significance of the mean revision 5.Median revision: 6.% of positive revisions: 7.% of negative revisions: 8.% of zero revisions: 9.Adjusted t-statistic for significance of mean revision 10.Critical values of t statistic for significance of mean revision 11.Standard deviation of revision 12.Root mean square revision 13.Quartile deviation 21

22. Eurostat Summary of statistics for analysis of revisions (2) 14.Minimum revision 15.Maximum revision 16.Range of revision 17.Skewness 18.% sign (later) = sign (earlier): 19.Acceleration / deceleration 20.Relative mean absolute revision: 21.Average absolute value of first published estimate 22.Correlation between revision and earlier estimate (test if revisions are „noise ‟ ) 23.Correlation between revision and later estimate (test if revisions are „news ‟ ) 24.Serial correlation of revisions: 25.Decomposition of the Mean Squared Revision 22

23. Eurostat Data coverage

24. Eurostat Looking at set of indicators: data coverage Policy makers need an as complete as possible picture of the economy Statistics are subject to events which could disrupt time series length adoption of new classifications (ESA 2010, BPM6) changes in the production process

25. Eurostat Approval process

26. Eurostat Characterising Back-calculation Horizon of backcalculation Choice of the model and of the method Robustness In official statistics: high degree of automation needed to minimise personal judgment Validation of results Backcalculation applications House price index: MT, ES, LT, LV, CY Disaggregation Unemployment rate: HR

27. Eurostat Application - HPI Aim: 10 years coverage for the Eurostat 2010=100 annual series using quarterly ones 1.Check on correlation between official and auxiliary series on the overlapping period 2.Choose the most correlated series as proxy 3.Data transformation: log-transformed and first order differentiated 4.OLS regression run on the overlapping period on delta logs 5.Estimated parameters are used to backcalculate 6.Calculate the annual series (AVG of 4 quarters)

28. Eurostat Latvian quarterly House price index (2010=100)

29. Eurostat Disaggregation: HR Unemployment rate HR data included in the MIP scoreboard since accession in 7/2013 Unemployment yearly values derived from monthly ones Monthly values produced by Eurostat from quarterly LFS + monthly number of unemployed persons HR LFS available twice a year for 2000-2006 Eurostat disaggregated the quarterly data to produce monthly values by components after using a SARIMA model to produce missing quarters

30. Eurostat HR Unemployment rate (%)

31. Eurostat Useful Links ESS guidelines on revisions http://epp.eurostat.ec.europa.eu/cache/ITY_OFFPUB/KS-RA-13-016/EN/KS- RA-13-016-EN.PDF OECD/Eurostat Task Force on “Performing Revisions Analysis for Sub-Annual Economic Statistics” http://www.oecd.org/std/40315546.pdf