1. SDMX: Background and purpose Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, 1-2 March 2017

2001: time to assess the way exchanges were made New developments offering opportunities to overcome issues

WHAT ARE THE ISSUES? Data being collected in multiple ways (surveys, files, web queries, metadata etc.); Data being transmitted in various formats (paper, excel, flat files etc.) Data being transmitted in various media (email, CD-ROMs, file uploads etc.); Data being stored in various places (USB, hard drive, servers, cloud etc.);

WHAT ARE THE ISSUES? Multiple organisations collecting similar or same data; Similar concepts in wording can have a different content; Increasing burden on organisations (collection, maintenance and managing); The intensive and manual nature of data collection; Errors and inconsistencies;

- Increasing demand for data (ease of use of Internet); WHAT WERE THE NEW DEVELOPMENTS? AN IDEA GERMINATES Faster exchanges possible; More frequent / bigger exchanges; - Increasing demand for data (ease of use of Internet); Growing types of information exchange; - between businesses - between businesses and customers - between individuals

WHAT OPPORTUNITIES? Afforded by new technological developments, opportunities: For the process to be more efficient; To improve trust and reliability; To improve web dissemination;

WHAT OPPORTUNITIES? Simplification (streamlining data flows, central management); Standardisation (software tools, data sharing); Harmonisation (data structures, concepts and code lists);

' A pessimist sees the difficulty in every opportunity; an optimist sees the opportunity in every difficulty.' Winston Churchill

So what is SDMX? The 'Statistical Data and Metadata eXchange' is an international initiative aimed at developing and employing more efficient processes for the exchange and sharing of statistical data and metadata among international organisations and member countries. It consists of technical and statistical standards, guidelines, an IT service infrastructure and IT tools.

What is the business case for SDMX? SDMX is a global response: 7 international organisations as sponsors, in collaboration with countries throughout the world; SDMX is an ISO IS standard (17369): - a document, established by consensus; - approved by a recognised body; - providing rules and guidelines; - for common and repeated use; - for optimum degree of order; - viewed as safe, reliable, good quality.

SDMX improves timeliness: - faster access to data; - move towards automation. SDMX improves accessibility: - bilateral, gateway and data-sharing; - push and pull modes; SDMX improves interpretability: - standardises structural metadata (the identifiers and descriptors of data); - standardises reference metadata (the content and quality of data);

SDMX improves coherence: - uses cross domain concepts; - uses shared code lists; - uses content oriented guidelines; - reuse across domains and agencies - aims for single figure dissemination. SDMX can reduce data errors: - some automated validation; - agreed structures for transmission; - time saved on conversion, mapping; - less manual intervention.

SDMX can reduce the reporting burden: - pre-validated content; - automated publication; - possible 'pull' by collecting agencies. SDMX can reduce IT development and maintenance costs: - open source approach; - no licensing costs; - shared toolbox; - improved interoperability between systems and applications.

SDMX is well suited to supporting a data sharing process: reporting every number only once.

SDMX is about changing from a multiple, diverse and complex exchange system, to a common, harmonised and standardised exchange system.

What are the downsides? SDMX is not investment free: it means training; it means changes. SDMX is not a magic wand: it is suited to aggregated data but not microdata. SDMX is dynamic: software versions are updated to increase functionalities and overcome bugs.

KEY messages: SDMX responds to a business need; SDMX improves quality in data and metadata exchanges; SDMX is an international standard based on shared experiences; SDMX offers cost-efficiencies.