Quality Criteria Near Final

Establishing the Basics ESS recommend quality reports should comment on 9 areas: Relevance Accuracy Timeliness Accessibility & clarity Coherence & comparability Trade off between output quality components Assessment of user needs and perceptions Performance, cost and respondent burden Confidentiality, transparency and security

Scope We’re focusing on quality of the flash estimate, not the quality of the GDP estimate We’re also looking at setting the quality criteria that need to be met before first publication of a EU28 and EA18 flash estimate For this reason the horizontal working group have focused on: Accuracy Timeliness At the June 2014 task force meeting it was also suggested we add a communication plan criteria

Near Final Paper A near final paper has been presented to the task force This paper was agreed in principal and presented to the wider National Accounts Working Group in May 2015 NAWG did not suggest any particular change to the paper or criteria As such the quality group would like to present the paper and four sign off criteria to the task force for final or almost final approval

Criterion 1 – Bias Criteria The aim of this criterion is to test whether the flash estimate provides an unbiased estimate of what T+45 growth will be: T+30 should be an unbiased estimate of GDP growth at T+45, with an average revision between -0.05 and 0.05, and no more than 66.7% of revisions in the same direction This is probably the most desirable property of a ‘flash’ estimate of a statistic Our main difficulty will be testing this with only 16 quarters of data available (8 real, 8 retrospective). Statistical tests were ruled out due to the limited number of quarters being available (see Annex A)

Criterion 2 – Scale/Size This criterion compares the size of revision between T+30 & T+45, and T+30 & T+65 The average absolute revision for T+30 GDP growth should be within 0.10 ppt compared with T+45 GDP growth estimate, and 0.13 ppt compared with T+65 GDP growth estimate. As with criteria one, using statistical tests was ruled out due to insufficient number of data points When setting this criteria consideration was given to the revisions performance of the US Flash estimate (presented in paper) There was also discussion around the fact that the EU28 estimate is a developing statistic and that one could assume revisions may improve as members states method evolve

Criterion 3 – Timeliness This criterion is designed to ensure enough member states can deliver to deadline consistently to produce quality results at the EU28 and EA18 level For the final two quarters of 2015 Eurostat needs to receive 70% of last year’s EU Member States GDP by the T+29 deadline This criteria is set to safeguard against the publication of a T+30 flash before it has been proven the timeliness can be adhered to on a consistent basis In addition to this criterion there should also be no known reasons why the criteria wouldn’t be met for each quarter of 2016 and beyond.

Criterion 4 – Communication This criterion is designed to ensure users are well informed about the estimate and it’s strengths and weaknesses Key milestones of the communication plan should be met or plans in place to meet before publication of the T+30 flash estimate This criteria is set to safeguard against the publication of a T+30 flash before users understand how it should be interpreted Outstanding Work Linked to the communication working group

Decision Time There has been a large amount of discussion around quality criteria since the second working group meeting Various issues have been raised and considered when developing the 4 criteria The objective for the quality working group was to propose quality acceptance criteria that would be used to inform a ‘go/no go’ decision for publication of a T+30 flash estimate With the task force’s approval these four criteria would become the ‘go/no go’ quality criteria

Questions?