Quality Criteria Initial Ideas

Establishing the Basics ESS recommend quality reports should comment on 9 areas: Relevance Accuracy Timeliness Accessibility & Clarity Coherence & Comparability Trade off between output quality components Assessment of user needs and perceptions Performance, cost and respondent burden Confidentiality, transparency and security

Scope We’re focusing on quality of the flash estimate not the quality of the GDP estimate We’re also looking at setting the quality criteria that need to be met before first publication of a European flash estimate For this reason it seems sensible for the horizontal working group to focus on: Accuracy Timeliness Despite limiting the scope it is recognised that the other dimensions are still important and should be addressed by the wider working group before first publication

Accuracy Our initial thinking is the accuracy of the flash estimate is best measured by analysing revisions performance Comparisons against estimates at t+45, t+60, and t+90 could be used to assess quality T-test’s could be used to determine if flash estimates exhibit any bias or the revision is significantly different from 0 We could also look at the correlation between flash GDP and later estimates

Accuracy – Some Questions What are we accessing the accuracy of? The European aggregate? Country level estimates? What is our lead indicator of accuracy? T-test result Average Absolute Revision Mean Revision Correlation What level of revision is acceptable? European aggregate +/- 0.1 percentage point? Country level aggregate +/- 0.2 percentage points?

Accuracy – Some Thoughts Analysing at a country level would be helpful in the development phase for flash and also helpful for users once published Countries may want to breakdown to lower levels to see which components are performing better/worse The horizontal group will aim to produce guidance on what countries can do Horizontal group will also make recommendations on which criteria should be met before first publication of flash

Timeliness A key criteria that needs to be assessed before publication of flash GDP Need to ensure the t+30 deadline can be consistently met by a large proportion of member states How many quarters of meeting this deadline do we need before we go live with flash?

Work Plan By the next working group we will: Further ahead: Produce guidance on what countries can do to measure accuracy Propose a set of accuracy criteria that should be met before publication of flash (for approval by the wider group) Propose a criteria to be met on timeliness Further ahead: Produce a template for monitoring accuracy Aid in the collection of data for monitoring Provide updates on progress towards targets