M. Henrard, B5 N. Buysse and H. Linden, B6 Eurostat The ESS Metadata Handler and the National Reference Metadata Editor Working Group on Regional and Urban Statistics 27- 28 September 2010, Luxembourg M. Henrard, B5 N. Buysse and H. Linden, B6 Eurostat

Outline The ESS Metadata-Handler and one of its main components The National Reference Metadata Editor (National RME) The standard structures for metadata reporting Euro SDMX Metadata Structure (ESMS) ESS Standard for Quality Reports structure (ESQRS) Impact on the statistical business processes What is next? Cross-domain concepts in the SDMX framework describe metadata concepts relevant to several statistical domains. SDMX recommends use of the concepts outlined below whenever feasible to promote re-usability and exchange of statistical information and their related metadata between national and international organizations. Whenever used, these concepts should conform to the specified names, roles, and representations defined in the SDMX Content-Oriented Guidelines. CDC promote: reusability exchange of data and related metadata Dynamic asset: initial growth, then relative stabilisation expected continuos maintenance 2 2

The ESS Metadata-Handler is the backbone of a metadata-driven harmonised statistical business process at Eurostat and in the European Statistical System; provides one central interface for accessing different types of metadata by Eurostat staff or by ESS members; provides services to other IT applications within the ESS; is the primary source for different types of harmonised metadata to be used within Eurostat and the ESS. Cross-domain concepts in the SDMX framework describe metadata concepts relevant to several statistical domains. SDMX recommends use of the concepts outlined below whenever feasible to promote re-usability and exchange of statistical information and their related metadata between national and international organizations. Whenever used, these concepts should conform to the specified names, roles, and representations defined in the SDMX Content-Oriented Guidelines. CDC promote: reusability exchange of data and related metadata Dynamic asset: initial growth, then relative stabilisation expected continuos maintenance 3 3

The ESS Metadata-Handler and its main components Input from national metadata Metadata from the Eurostat Domain manager Eurostat as main administrator ESS – Metadata Handler Euro SDMX Registry National RME EMIS CODED RAMON Cross-domain concepts in the SDMX framework describe metadata concepts relevant to several statistical domains. SDMX recommends use of the concepts outlined below whenever feasible to promote re-usability and exchange of statistical information and their related metadata between national and international organizations. Whenever used, these concepts should conform to the specified names, roles, and representations defined in the SDMX Content-Oriented Guidelines. CDC promote: reusability exchange of data and related metadata Dynamic asset: initial growth, then relative stabilisation expected continuos maintenance Common user Interface Output produced for the Eurostat Web Other output for Eurostat or external users 4 4

The National Reference Metadata Editor as part of the Eurostat-MH is the component dealing with national reference metadata; will accommodate the current ESS standards for reference metadata such as the ESMS or the upcoming ESQRS; is addressed to the national producers of reference metadata, who will be able to compile their domain-specific metadata on-line and transmit them to Eurostat; is the IT tool stimulating the harmonization of reference metadata within the ESS; will be made available to the ESS in 4th quarter of 2010.

Business process related to a national reference metadata file

The Euro SDMX Metadata Structure (ESMS) Is the standard format for reference metadata in Eurostat and within the ESS; Is made of 21 concepts selected from the list of 62 SDMX cross-domain concepts to be used within the ESS; Is now fully used at Eurostat with more than 300 files disseminated on the website; Is the format to be used also for the reporting of national reference metadata files to Eurostat (Commission Recommendation 2009/498/EC of June 2009) Is fully SDMX compliant

The Euro SDMX Metadata Structure (ESMS) The 21 concepts and sub-concepts of the ESMS : Cross-domain concepts in the SDMX framework describe metadata concepts relevant to several statistical domains. SDMX recommends use of the concepts outlined below whenever feasible to promote re-usability and exchange of statistical information and their related metadata between national and international organizations. Whenever used, these concepts should conform to the specified names, roles, and representations defined in the SDMX Content-Oriented Guidelines. CDC promote: reusability exchange of data and related metadata Dynamic asset: initial growth, then relative stabilisation expected continuos maintenance 8 8

The ESS Standard for Quality Reports Structure (ESQRS) Within the European Statistical System (ESS) reporting on statistical data quality exists in many statistical domains….

The ESS Standard for Quality Reports Structure (ESQRS) Progress made since 2008: 01/2009: release of the new version of the ESS quality reporting documents: ESS Standard for Quality Reports (ESQR) ESS Handbook for Quality Reports (EHQR)  Detailed requirements following the European Statistics Code of Practice ESS Quality and Performance Indicators (QPI’s) defined 03/09: EP/Council Regulation 223/2009  Article 12 defining the quality criteria to be reported

The ESS Standard for Quality Reports Structure (ESQRS) Based on the ESQR (ESS Standard for Quality Report), a new report structure - the ESS Standard for Quality Reports Structure: ESQRS - was created for harmonising the reporting on statistical data quality within the ESS. The ESQRS is using the main statistical data quality criteria as listed in EP/Council Regulation 223/2009 and as being part of the ESMS and details them further : Relevance Accuracy Timeliness and Punctuality Accessibility and Clarity Comparability Coherence A subset of the Quality Performance Indicators (QPI’s) is covered in the new ESQRS

The relation between the ESS standards ESMS and ESQRS ESMS is more oriented to the USERS of statistics  to understand the statistical data released there is no need for too detailed information on data quality 21 SDMX cross domain concepts used ESQRS is more oriented to the PRODUCERS of statistics to monitor the quality of the statistics produced in detail concentrating on the main quality concepts (being also part of the ESS Statistics Regulation No 223/2009) However, there is information on data quality which is common to both ESMS and ESQRS.

ESMS and ESQRS – example of correspondence

Impact on the statistical business processes ESMS and ESQRS are to be integrated into the upcoming National Reference Metadata Editor for production, exchange and dissemination of reference metadata in the ESS, allowing: + More AUTOMATIC PRODUCTION of the reference metadata in the ESS. + Information collected ONLY ONCE and reused for different purposes (ESQRS -> ESMS; NSI->Eurostat; Eurostat-> IMF/OECD). + More harmonised and better availability of metadata on quality. + Full SDMX compliance (metadata creation, exchange and dissemination).  The harmonisation work also considerably contributes to the improvement and integration of the statistical business processes as defined in the Commission Communication 404/2009. Cross-domain concepts in the SDMX framework describe metadata concepts relevant to several statistical domains. SDMX recommends use of the concepts outlined below whenever feasible to promote re-usability and exchange of statistical information and their related metadata between national and international organizations. Whenever used, these concepts should conform to the specified names, roles, and representations defined in the SDMX Content-Oriented Guidelines. CDC promote: reusability exchange of data and related metadata Dynamic asset: initial growth, then relative stabilisation expected continuos maintenance  14 14

Piloting is starting now (in autumn 2010): What is next ? ~ 30/35 statistical domains in the ESS collect national reference metadata; ~ 15/20 statistical domains in the ESS collect national quality reports; Piloting is starting now (in autumn 2010): Mapping of existing national reference metadata with the new Standards; Conversion of existing national reference metadata into the RME; Completion and transmission of national reference metadata files; Validation of national reference metadata files; Dissemination of national reference metadata files (if decided so).