Interoperable data formats: SDMX Building the solution for NRPs: innovative approaches and technologies Interoperable data formats: SDMX New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Statistical Data and Metadata eXchange SDMX is an international initiative that aims at standardising and modernising (“industrialising”) the mechanisms and processes for the exchange of statistical data and metadata among international organisations and their member countries. Launched in 2002, SDMX is governed by a Sponsors Committee and a Secretariat. The Technical and Statistical Working Groups manage the definition and updating of the technical standard and publish statistical guidelines New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Statistical Data and Metadata eXchange SDMX is an ISO standard (ISO 17369) designed to describe statistical data and metadata, normalise their exchange, and improve their efficient sharing across statistical and similar organisations. It provides an integrated approach to facilitating statistical data and metadata exchange, enabling interoperable implementations within and between systems concerned with the exchange, reporting and dissemination of statistical data and their related meta-information. 1.           What is SDMX? SDMX, which stands for Statistical Data and Metadata eXchange, is an ISO standard designed to describe statistical data and metadata, normalise their exchange, and improve their efficient sharing across statistical and similar organisations. It provides an integrated approach to facilitating statistical data and metadata exchange, enabling interoperable implementations within and between systems concerned with the exchange, reporting and dissemination of statistical data and their related meta-information. It consists of: technical standards (including the Information Model) statistical guidelines an IT architecture and tools But SDMX is not just a format for data exchange. Taken together, the technical standards, the statistical guidelines and the IT architecture and tools can support improved business processes for any statistical organisation as well as the harmonisation and standardisation of statistical metadata. The first version of the SDMX technical standard (1.0) was finalised in 2004 and approved as ISO (“International Standard”) 17369 in January 2013. Version 2.0 was approved in 2005. Version 2.1 was completed and published in May 2011. The Information Model which forms the core of SDMX has been developed to support statistics as collected and used by governmental and international statistical organisations, and this model is also applicable to other organisational contexts involving statistical data and related metadata. The statistical guidelines aim at providing general statistical governance as well as common (“cross-domain”) concepts and code lists, a common classification of statistical domains and a common terminology. The first set of guidelines was published in January 2009. Many IT tools have been developed to support the use and implementation of SDMX. Most of these tools are of an open source nature, so that they can be used as components for building IT systems in statistical organisations. Examples of such tools are the SDMX Registry, the Data Structure Wizard (DSW) and the SDMX Reference Infrastructure (SDMX-RI). The work processes of SDMX are fully transparent: public consultations are conducted when major revisions are envisaged. The first global SDMX data exchanges were implemented in 2013 by the seven sponsor organisations and covered National Accounts, Balance of Payments and Foreign Direct Investment. Many additional data exchanges are presently under development, both between international organisations and between international organisations and their constituencies. New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Why SDMX? ISO standard (ISO 17369) Number of organizations using SDMX has risen significantly in the last 5 years Mature technology According to Gartner’s Hype Cycle chart presented in the Global Conference 2017, SDMX is in the «slope of enlightenment» towards the «plateau of productivity» SDMX Information Model helps streamlining your data model Tools available to «use» SDMX with low investment Global artefacts facilitates data exchange agreements Several formats: e.g. SDMX-ML, SDMX-json, SDMX-csv New York, 22-24 January 2018 National platforms for SDG reporting

Technical Specifications Statistical Data and Metadata eXchange Technical Specifications The SDMX Information Model Guidelines to Harmonise Content Content-oriented Guidelines (COG) Tools IT Architectures for data exchange SDMX compliant tools SDMX is not just a data transmission format… New York, 22-24 January 2018 National platforms for SDG reporting 5 5

SDMX Information Model Statistical Data and Metadata eXchange Image source: Metadata Technology New York, 22-24 January 2018 National platforms for SDG reporting

«DOING» SDMX vs. «USING» SDMX MITH: SDMX is too complex! SDMX looks complex and intimidating if you try to understand all its detailed documentation, but… Have you ever read the IEEE 802.11 documentation? and so… Do you have difficulties to use WiFi? You don’t need to develop an SDMX interface to use SDMX as you are not supposed to build your own wireless Ethernet adapter to use WiFi New York, 22-24 January 2018 National platforms for SDG reporting

Information Model (“metamodel”) Statistical Data and Metadata eXchange Dimensions (ex: country, variable/topic, year) Data Structure Definition (DSD) Code lists Dataset Structure Structural Metadata Attributes (ex: unit of measure) Metadata about an individual value, a time series or a group of time series Identify/Describe Data Provides a way of modelling data, metadata and exchange processes New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Deriving a Data Structure from a table VARIABLE AGE ATTRIBUTES OBS_VALUE COUNTRY SEX TIME PERIOD DIMENSIONS ATTRIBUTES MEASURE New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Conclusions Statistical Data and Metadata eXchange The Information Model which forms the core of SDMX has been developed to support statistics as collected and used by statistical organisations The statistical guidelines aim at providing general statistical governance as well as common (“cross-domain”) concepts and code lists Many IT tools have been developed to support the use and implementation of SDMX, most of them are of an open source nature and can be used as components for building IT systems in statistical organisations. Taken together, the technical standards, the statistical guidelines and the IT architecture and tools can support improved business processes for any statistical organisation as well as the harmonisation and standardisation of statistical metadata. The Information Model which forms the core of SDMX has been developed to support statistics as collected and used by governmental and international statistical organisations, and this model is also applicable to other organisational contexts involving statistical data and related metadata. The statistical guidelines aim at providing general statistical governance as well as common (“cross-domain”) concepts and code lists, a common classification of statistical domains and a common terminology. The first set of guidelines was published in January 2009. Many IT tools have been developed to support the use and implementation of SDMX. Most of these tools are of an open source nature, so that they can be used as components for building IT systems in statistical organisations. Examples of such tools are the SDMX Registry, the Data Structure Wizard (DSW) and the SDMX Reference Infrastructure (SDMX-RI). New York, 22-24 January 2018 National platforms for SDG reporting

National platforms for SDG reporting Conclusions Everyday we use different tools and appliances to be more efficient in our activities. We learn how to use them with a quick reading of a guide; or we simply know how to use them, because it is like any other similar tool (standard). But for most of these tools, we don’t have any idea on how they have been designed, or how they work. You can use SDMX-based tools and applications in the same way, and they will certainly make your «work» more efficiently. (especially if your work is data reporting!) The Information Model which forms the core of SDMX has been developed to support statistics as collected and used by governmental and international statistical organisations, and this model is also applicable to other organisational contexts involving statistical data and related metadata. The statistical guidelines aim at providing general statistical governance as well as common (“cross-domain”) concepts and code lists, a common classification of statistical domains and a common terminology. The first set of guidelines was published in January 2009. Many IT tools have been developed to support the use and implementation of SDMX. Most of these tools are of an open source nature, so that they can be used as components for building IT systems in statistical organisations. Examples of such tools are the SDMX Registry, the Data Structure Wizard (DSW) and the SDMX Reference Infrastructure (SDMX-RI). New York, 22-24 January 2018 National platforms for SDG reporting

Thank you ! Edgardo Greising Head of Microdata and Knowledge Management Unit STATISTICS - ILO greising@ilo.org