Prodcom Working Group Item 08.2 - Quality reporting and indicators 18th -19th September 2014

CURRENT SITUATION: NO REGULAR quality reporting from MS 2. Very general provision in PRODCOM, under Art.6 of Regulation 912/2004: ”[…] Member States are also to provide Eurostat with the necessary information about their survey methods, samples and coverage for the purpose of documentation of compliance with the principles of the PRODCOM methodology” Item 14_08 Prodcom Working Group 18th -19th September 2014

IN THE PAST: 2003 & 2007 - Quality reports have been provided on voluntary basis 2012 - Eurostat presented the National Reference Metadata Editor (NRME) in the Working Group. 4 countries (DK, IE, PT, UK) agreed to test the reporting tool in 2013 2013 – Denmark presented their experience with NRME to the WG held in October. CONCLUSION: the application was EASY TO USE Item 14_08 Prodcom Working Group 18th -19th September 2014

Current context of FRIBS 6 preliminary provisions, out of which: Member States shall take all measures necessary to ensure the quality of the statistics transmitted and of the business registers according to common quality criteria. For the purpose of this Regulation, the quality criteria defined in Article 12(1) of Regulation (EC) 223/2009 shall apply. The modalities, structure and periodicity of the quality reports shall be adopted by the Commission (Eurostat) in the form of implementing acts. Those implementing acts shall be adopted in accordance with the examination procedure referred to Article 13(2). Mention : The 6 provisions appear in preliminary version of FRIBS legislation 2 – Code of Practice (quality criteria) 3 – In principiu exista flexibilitate si implementarea va depinde de la domeniu la domeniu, dar va trebui sa introducem un standard de raportare in Regular Reporting Item 14_08 Prodcom Working Group 18th -19th September 2014

The quality criteria & indicators (1) (defined in Article 12(1) of Regulation (EC) 223/2009) RELEVANCE R1. User satisfaction index R2. Rate of available statistics ACCURACY A1. Coefficient of variation A2. Unit response rate (un-weighted/weighted) A3. Item response rate (un-weighted/weighted) A4. Imputation rate and ratio A5. Over-coverage and misclassification rates A6. Geographical under-coverage ratio A7. Average size of revisions Item 14_08 Prodcom Working Group 18th -19th September 2014

The quality criteria & indicators (2) (defined in Article 12(1) of Regulation (EC) 223/2009) TIMELINESS AND PUNCTUALITY T1. Punctuality of time schedule of effective publication T2. Time lag between the end of reference period and the date of first results T3. Time lag between the end of reference period and the date of the final results   ACCESSIBILITY AND CLARITY AC1. Number of publications disseminated and/ or sold AC2. Number of accesses to databases AC3. Rate of completeness of metadata information for released statistics. Item 14_08 Prodcom Working Group 18th -19th September 2014

The quality criteria & indicators (3) (defined in Article 12(1) of Regulation (EC) 223/2009) COHERENCE AND COMPARABILITY C1. Length of comparable time-series C2. Number of comparable time-series C3. Rate of differences in concepts and measurement from European norms C4. Asymmetries for statistics mirror flows CH1. Rate of statistics that satisfies the requirements for the main secondary use Item 14_08 Prodcom Working Group 18th -19th September 2014

REPORTING STANDARDS (introduced in Nov 2013 by SIMS) ESMS: the Euro SDMX Metadata Structure - is the basic ESS structure for reference metadata. ESQRS: the ESS Standard for Quality Reports Structure - is the harmonized report structure for ESS quality reports Item 14_08 Prodcom Working Group 18th -19th September 2014

ESMS structure Item XX Prodcom Working Group 18th -19th September 2014 The ESMS structure is not corresponding exactly with quality criteria, but with a slight adjustment it covers all of them The proposed indicators ( which can be found in handbook / task force – as source) constitutes a good starting point in building of Prodcom indicators, depending on specificities of our activity Need for quality reporting in PRODCOM. The guiding lines are given by: - The ESMS structure is not corresponding exactly with quality criteria, but with a slight adjustment it covers all of them the quality dimensions in the code of practice & mentioned indicators Additional work done for the development of indicators corresponding to the different quality dimensions (handbook…task force); this can be a starting point to complement or substitute the current compliance score which is a composite indicator that is however difficult to interpret. Other criteria important for PRODCOM should be considered: e.g. the 90% representativeness threshold or other criteria in the regulation. In case that indicators cannot be computed, some qualitative information on the methodology is necessary for the quality and compliance assessment. Item XX Prodcom Working Group 18th -19th September 2014

RECOMMENDATION OF REPORTING (technical manual annexed to the ESSC document of November 2013) “It is recommended that all statistical processes in the ESS should at least have a basic quality report in the form of short or user quality report (ESMS)”. Prodcom Working Group 18th -19th September 2014

RECOMMENDATION OF REPORTING (DIME-IT Directors Joint meeting, March 2014) “All statistical processes producing, exchanging and disseminating European statistics should be accompanied by general reference metadata – including basic quality information according to the ESS quality criteria – in using the ESMS structure.” Item 14_08 Prodcom Working Group 18th -19th September 2014

EUROSTAT PROPOSALS: To use ESMS as quality reporting standard (filled template attached in annex 2 of document) MS to send their annual quality report recommendable till 15th of November of every year (this deadline could be later reviewed in the terms of FRIBS) A limited number of quality indicators should be computed – or at least qualitative information where indicators can not be computed Item XX Prodcom Working Group 18th -19th September 2014

EUROSTAT PROPOSAL: Member States voluntary for tests for 2103 data Using the given structure and the guidelines provided as Annex (the guidelines could be refined during the testing period) Eurostat will provide its support and feedback When FRIBS will be final, this reporting will become compulsory Written consultation for quality indicators and compliance assessment, if necessary Item XX Prodcom Working Group 18th -19th September 2014

THANK YOU Prodcom Working Group 18th -19th September 2014