1. Quality Criteria
Initial Ideas

2. Establishing the Basics
ESS recommend quality reports should comment on 9 areas:
Relevance
Accuracy
Timeliness
Accessibility & Clarity
Coherence & Comparability
Trade off between output quality components
Assessment of user needs and perceptions
Performance, cost and respondent burden
Confidentiality, transparency and security

3. Scope
We’re focusing on quality of the flash estimate not the quality of the GDP estimate
We’re also looking at setting the quality criteria that need to be met before first publication of a European flash estimate
For this reason it seems sensible for the horizontal working group to focus on:
Accuracy
Timeliness
Despite limiting the scope it is recognised that the other dimensions are still important and should be addressed by the wider working group before first publication

4. Accuracy
Our initial thinking is the accuracy of the flash estimate is best measured by analysing revisions performance
Comparisons against estimates at t+45, t+60, and t+90 could be used to assess quality
T-test’s could be used to determine if flash estimates exhibit any bias or the revision is significantly different from 0
We could also look at the correlation between flash GDP and later estimates

5. Accuracy – Some Questions
What are we accessing the accuracy of?
The European aggregate?
Country level estimates?
What is our lead indicator of accuracy?
T-test result
Average Absolute Revision
Mean Revision
Correlation
What level of revision is acceptable?
European aggregate +/- 0.1 percentage point?
Country level aggregate +/- 0.2 percentage points?

6. Accuracy – Some Thoughts
Analysing at a country level would be helpful in the development phase for flash and also helpful for users once published
Countries may want to breakdown to lower levels to see which components are performing better/worse
The horizontal group will aim to produce guidance on what countries can do
Horizontal group will also make recommendations on which criteria should be met before first publication of flash

7. Timeliness
A key criteria that needs to be assessed before publication of flash GDP
Need to ensure the t+30 deadline can be consistently met by a large proportion of member states
How many quarters of meeting this deadline do we need before we go live with flash?

8. Work Plan By the next working group we will: Further ahead:
Produce guidance on what countries can do to measure accuracy
Propose a set of accuracy criteria that should be met before publication of flash (for approval by the wider group)
Propose a criteria to be met on timeliness
Further ahead:
Produce a template for monitoring accuracy
Aid in the collection of data for monitoring
Provide updates on progress towards targets