Interoperability of metadata systems: Follow-up actions Item 3.2 Standards Working Group meeting 2019 Luca Gramaglia Eurostat, Unit B5

Introduction This presentation will: Describe the progress made on improving the interoperability of metadata and quality reporting in the ESS. Show the principles Eurostat would like to apply in the future when implementing SIMS in new domains. Describe some actions Eurostat will take to improve communication with national ESS-MH administrators

Progress on interoperability improvements

Introduction The 2018 Standards Working Group saw a discussion on the various issues affecting the interoperability od metadata exchange in the ESS. The discussion outlined some of the most notable obstacles to interoperability of metadata exchange, and suggested actions to improve the situation. The document presented at the Working Group is attached below. The purpose of this presentation is to give an overview of the progress on the actions agreed at the last Standards Working Group.

European Interoperability Framework

Legal interoperability Improvement action Progress Develop a common approach for the inclusion of metadata and quality reporting in legal acts Eurostat is considering amending or replacing Recommendation 2009/498/EC on the use of ESMS for reference metadata by a recommendation on the use of SIMS for metadata and quality reporting. While the recommendation itself would be non-binding, it could be referenced in or used as a basis for legal acts on quality reporting, thus guaranteeing a common approach in legal acts. As the issuance of a recommendation takes time, Eurostat's quality team is taking a more proactive approach on ensuring that legal acts on quality reporting are aligned to SIMS.

Organisational interoperability Improvement action Progress Clearly indicate in the ESS-MH which metadata flows are currently actively used for national metadata and quality reporting This is a pending improvement action in the ESS-MH Establish a link between the inventory of datasets in EDAMIS and the inventory of metadata flows in the ESS-MH This link has been established. The new EDAMIS 4 will allow to store this link and to make it visible in EDAMIS and eventually in the ESS-MH. However, due to delays in the EDAMIS 4 developments, this feature is not yet available. Give Member States the possibility to provide information at different levels of granularity

Semantic interoperability Improvement action Progress Expand use of domain-specific guidelines The new principles for SIMS implementation (see next section) realise these improvements Replace HTML templates by sub-concepts (phase-out of grouping functionality needed for full implementation) Allow Member States to submit a single report containing all SIMS concepts in case both ESMS and ESQRS are needed

Technical interoperability Improvement action Progress Phase-out the grouping functionality This is a pending improvement action in the ESS-MH Automate the transmission of metadata and quality report transmission via the use of EDAMIS Work on this is underway. A description of the envisaged workflow is available in the document below. There are however some dependencies between the introduction of this functionality and EDAMIS 4, which means that delays in the EDAMIS 4 developments impact its availability.

Summary Since the 2018 Standards Working Group meeting, the areas that saw the most concrete advancement were the ones related to semantic interoperability. This is due to the fact that they require changes in implementation practices rather than in the supporting IT systems. The actions that require changes in the supporting IT systems (ESS-MH and EDAMIS) have progressed more slowly. Upcoming ESS-MH improvements (in order of priority): Better overview of active metadataflows Alternative to grouping for tabular data Allowing provision of information at different levels of granularity

Principles

Principles Direct implementation of SIMS: If both ESMS and ESQRS are required for a particular domain, a single SIMS template will be implemented in the ESS-MH. This will guarantee that he principle of "once-for-all-purposes" reporting is respected. SIMS concepts remain pure: No introduction of customisation via HTML templates in standard SIMS concepts. Any customisation must happen at the level of sub-concepts. This guarantees that information compiled at national level for the standard SIMS concepts can be directly reused at European level.

Principles Clear hierarchy of customisation options: As a general rule, customisation should be avoided as much as possible. When customisation is needed the following hierarchy of options can be considered in descending order of preference: Insertion of additional guidelines, which contextualise and further define the generic SIMS guidelines in a specific domain. Insertion of sub-concepts at the lowest hierarchical level of the SIMS structure. Sub-concepts allow for the clear separation of "standard" information and "domain-specific" information. HTML templates (currently unavoidable for tabular information). As stated before, HTML templates can only be inserted at the levels of domain-specific sub-concepts, and cannot be inserted in standard concepts.

Principles No "grouping" functionality: The grouping functionality in the ESS-MH was introduced to replace the need for HTML templates for tabular data. It has however failed to do that and has introduced issues in terms of interoperability and maintainability. No new implementation of the grouping functionality will be done (but current implementations may need to be supported until a suitable alternative is found).

In practice: the ESAW domain Eurostat applied the principles outlined in the previous slides in the ESAW domain (European Safety and Accidents at Work). At the end of 2018, the situation in the ESAW domain was as follows: Eurostat domain managers collected information covering the full set of SIMS concepts via Excel. This was because they did not want Member States to enter twice the same information for ESQRS and ESMS. The information related to the ESMS concepts was manually transferred to the ESS-MH. In addition, Eurostat collected detailed tables with information on the coverage and completeness of the statistics (e.g. coverage by sector).

In practice: the ESAW domain One single template based on SIMS (not ESMS or ESQRS) was created in the ESS-MH, to avoid duplication of information. "Direct implementation of SIMS" principle No customisation whatsoever was added to the standard SIMS concepts. "SIMS concepts remain pure" principle The tables on coverage and completeness were added as HTML tables within sub-concepts added at the lowest level of the SIMS structure. "Clear hierarchy of customisation options" principle "No grouping functionality" principle

In practice: the ESAW domain Before: Sample Excel file used by Eurostat to collect SIMS-compliant information from Member States Sample Excel file with additional tabular information After:

Communication with national ESS-MH administrators

Requests from national ESS-MH administrators Additional information on metadataflows: Which metadataflows are active for national metadata and quality reporting Deadlines for the transmission of reports Links to the covered EDAMIS datasets Clear information on whether domain-specific guidelines are present Notifications when national files are validated and published All these changes are being considered as improvements to the ESS-MH

Requests from national ESS-MH administrators Additional information on ongoing projects and upcoming changes: Status of ongoing implementation projects Status of requested ESS-MH improvements The publicly accessible Confluence space will contain all this information and be regularly updated: https://webgate.ec.europa.eu/fpfis/wikis/display/RMSDE/Reference+Metadata+and+Standards+for+Data+Exchange+team+Home

Thank you for your attention! Any questions?