1. Interoperability of metadata systems: Follow-up actions
Item 3.2
Standards Working Group meeting 2019
Luca Gramaglia
Eurostat, Unit B5

2. Introduction This presentation will:
Describe the progress made on improving the interoperability of metadata and quality reporting in the ESS.
Show the principles Eurostat would like to apply in the future when implementing SIMS in new domains.
Describe some actions Eurostat will take to improve communication with national ESS-MH administrators

3. Progress on interoperability improvements

4. Introduction
The 2018 Standards Working Group saw a discussion on the various issues affecting the interoperability od metadata exchange in the ESS.
The discussion outlined some of the most notable obstacles to interoperability of metadata exchange, and suggested actions to improve the situation. The document presented at the Working Group is attached below. The purpose of this presentation is to give an overview of the progress on the actions agreed at the last Standards Working Group.

5. European Interoperability Framework

6. Legal interoperability
Improvement action
Progress
Develop a common approach for the inclusion of metadata and quality reporting in legal acts
Eurostat is considering amending or replacing Recommendation 2009/498/EC on the use of ESMS for reference metadata by a recommendation on the use of SIMS for metadata and quality reporting.
While the recommendation itself would be non-binding, it could be referenced in or used as a basis for legal acts on quality reporting, thus guaranteeing a common approach in legal acts.
As the issuance of a recommendation takes time, Eurostat's quality team is taking a more proactive approach on ensuring that legal acts on quality reporting are aligned to SIMS.

7. Organisational interoperability
Improvement action
Progress
Clearly indicate in the ESS-MH which metadata flows are currently actively used for national metadata and quality reporting
This is a pending improvement action in the ESS-MH
Establish a link between the inventory of datasets in EDAMIS and the inventory of metadata flows in the ESS-MH
This link has been established. The new EDAMIS 4 will allow to store this link and to make it visible in EDAMIS and eventually in the ESS-MH. However, due to delays in the EDAMIS 4 developments, this feature is not yet available.
Give Member States the possibility to provide information at different levels of granularity

8. Semantic interoperability
Improvement action
Progress
Expand use of domain-specific guidelines
The new principles for SIMS implementation (see next section) realise these improvements
Replace HTML templates by sub-concepts
(phase-out of grouping functionality needed for full implementation)
Allow Member States to submit a single report containing all SIMS concepts in case both ESMS and ESQRS are needed

9. Technical interoperability
Improvement action
Progress
Phase-out the grouping functionality
This is a pending improvement action in the ESS-MH
Automate the transmission of metadata and quality report transmission via the use of EDAMIS
Work on this is underway. A description of the envisaged workflow is available in the document below.
There are however some dependencies between the introduction of this functionality and EDAMIS 4, which means that delays in the EDAMIS 4 developments impact its availability.

10. Summary
Since the 2018 Standards Working Group meeting, the areas that saw the most concrete advancement were the ones related to semantic interoperability. This is due to the fact that they require changes in implementation practices rather than in the supporting IT systems.
The actions that require changes in the supporting IT systems (ESS-MH and EDAMIS) have progressed more slowly.
Upcoming ESS-MH improvements (in order of priority):
Better overview of active metadataflows
Alternative to grouping for tabular data
Allowing provision of information at different levels of granularity

11. Principles

12. Principles
Direct implementation of SIMS: If both ESMS and ESQRS are required for a particular domain, a single SIMS template will be implemented in the ESS-MH. This will guarantee that he principle of "once-for-all-purposes" reporting is respected.
SIMS concepts remain pure: No introduction of customisation via HTML templates in standard SIMS concepts. Any customisation must happen at the level of sub-concepts. This guarantees that information compiled at national level for the standard SIMS concepts can be directly reused at European level.

13. Principles Clear hierarchy of customisation options:
As a general rule, customisation should be avoided as much as possible.
When customisation is needed the following hierarchy of options can be considered in descending order of preference:
Insertion of additional guidelines, which contextualise and further define the generic SIMS guidelines in a specific domain.
Insertion of sub-concepts at the lowest hierarchical level of the SIMS structure. Sub-concepts allow for the clear separation of "standard" information and "domain-specific" information.
HTML templates (currently unavoidable for tabular information). As stated before, HTML templates can only be inserted at the levels of domain-specific sub-concepts, and cannot be inserted in standard concepts.

14. Principles
No "grouping" functionality: The grouping functionality in the ESS-MH was introduced to replace the need for HTML templates for tabular data. It has however failed to do that and has introduced issues in terms of interoperability and maintainability. No new implementation of the grouping functionality will be done (but current implementations may need to be supported until a suitable alternative is found).

15. In practice: the ESAW domain
Eurostat applied the principles outlined in the previous slides in the ESAW domain (European Safety and Accidents at Work).
At the end of 2018, the situation in the ESAW domain was as follows:
Eurostat domain managers collected information covering the full set of SIMS concepts via Excel. This was because they did not want Member States to enter twice the same information for ESQRS and ESMS. The information related to the ESMS concepts was manually transferred to the ESS-MH.
In addition, Eurostat collected detailed tables with information on the coverage and completeness of the statistics (e.g. coverage by sector).

16. In practice: the ESAW domain
One single template based on SIMS (not ESMS or ESQRS) was created in the ESS-MH, to avoid duplication of information.
"Direct implementation of SIMS" principle
No customisation whatsoever was added to the standard SIMS concepts.
"SIMS concepts remain pure" principle
The tables on coverage and completeness were added as HTML tables within sub-concepts added at the lowest level of the SIMS structure.
"Clear hierarchy of customisation options" principle
"No grouping functionality" principle

17. In practice: the ESAW domain
Before:
Sample Excel file used by Eurostat to collect SIMS-compliant information from Member States
Sample Excel file with additional tabular information
After:

18. Communication with national ESS-MH administrators

19. Requests from national ESS-MH administrators
Additional information on metadataflows:
Which metadataflows are active for national metadata and quality reporting
Deadlines for the transmission of reports
Links to the covered EDAMIS datasets
Clear information on whether domain-specific guidelines are present
Notifications when national files are validated and published
All these changes are being considered as improvements to the ESS-MH

20. Requests from national ESS-MH administrators
Additional information on ongoing projects and upcoming changes:
Status of ongoing implementation projects
Status of requested ESS-MH improvements
The publicly accessible Confluence space will contain all this information and be regularly updated:

21. Thank you for your attention! Any questions?