1. SDMX : General introduction H. Linden, Eurostat, Unit B5
STATISTICAL DATA AND METADATA EXCHANGE
SDMX : General introduction H. Linden, Eurostat, Unit B5
this is meant to be a presentation of about 15 – 20 minutes, giving you a down-to-earth summary of the project within the context of SDMX

2. What is SDMX and why is it so important?
Current situation :
 Lacking harmonisation causes extra costs and inconvenience !!

3. What is SDMX and why is it so important?
Solution: A common and harmonised phone charger
Benefits :
For producers
One charger  cheaper prod. costs
One package cheaper transport costs
For clients
One charger serves all  convenience
Lower prod. Costs  lower phone prices
Possible charging from computers
For everyone
Less chargers needed  environmental benefits

4. What is SDMX and why is it so important?
Current situation in the ESS :
Different types of data and metadata files exchanged
 Conversion of formats needed
No standardisation of the statistical contents
 Renaming, mappings needed
Problem of correspondence between variables, codes, etc.
The way forward: creation of technical, IT and statistical
standards to be used for official data and metadata

5. What is SDMX and why is it so important?
SDMX is carried out by seven international organizations: BIS, ECB, Eurostat, IMF, OECD, UN, World Bank;
These organisations have joined their forces with the results of
creating the SDMX technical and statistical standards and guidelines together with an IT service infrastructure and IT tools to be used for the efficient exchange and sharing of statistical data and metadata.
These technical and statistical standards and guidelines are used and implemented more and more by statistical organisations around the world. 5

6. SDMX within the European Statistical System
In 03/2009, the Eurostat top management confirmed SDMX and its implementation within the ESS in stating:
“As a first step before making SDMX compulsory for all domains in Eurostat, the use of SDMX would be made compulsory for all new or considerably changed datasets and reference metadata sets.”
The SDMX technical and statistical standards and guidelines can be regarded as one of the main enablers for implementing the new Eurostat vision based on the Commission Communication (COM 404/2009).

7. SDMX – the main components
The SDMX technical standards (v. 2.1 revised in August 2011)
SDMX information model
Data and metadata messages and query formats
Registry services
The SDMX guidelines to harmonise the statistical contents
the SDMX Content-oriented Guidelines
The SDMX IT architecture
the “push”, “pull” and “hub” approaches
The SDMX IT tools
IT tools produced by sponsoring organisations or others which are openly available
The SDMX IT tools are accessible through the SDMX IT tool database on the SDMX website
IM and syntax specifications form what is named “technical standards”
Version 1.0 specification of the technical standards has been approved by the International Organization for Standardization (ISO) as a Technical Specification (ISO/TS 17369: 2005 SDMX).
The Version 2.0 specification (November 2005) broadens the framework to support wider coverage of metadata exchange as well as a more fully articulated architecture for data and metadata exchange. Steps are also being taken to bring this work forward within the context of ISO, assuring that SDMX technical standards build on other recognized standards and providing the basis for interoperability with them.
The development of the content-oriented guidelines is being undertaken outside the ISO framework. This should facilitate steps by the SDMX sponsoring institutions to broadly involve content-oriented efforts of a wide range of experts that are already working in many subject-matter domains within the global statistical community. 7

8. SDMX components
The SDMX Content-oriented Guidelines (released in 01/2009):
Annex 1: The SDMX cross-domain concepts:
List of statistical concepts relevant to statistical domains to be used within the SDMX technical standards
Annex 2: The SDMX cross-domain code lists:
Statistical code lists relevant to statistical domains to be used within the SDMX technical standards
Annex 3: The statistical subject matter domains
List of subject matter domains (e.g. demography statistics, national accounts…)
Annex 4: The Metadata Common Vocabulary
Metadata cross-domain statistical terminology used above
SDMX is not only IT oriented, but also contains statistical standards; it is not only a data transmission format. 8

9. ESS SDMX implementation: process
Specs
Codes B5
Draft DSD
Metadata
& Codes B6
Pilot
Project
MS & B
Process
SDMX
DSD
Define
B3 [B5 / B4]
Parallel
Stage MS
Working
Group
MS [B]
Preparation
Compliance
Implementation
Production
B4 [B3 / B2] 9

10. Benefits of SDMX
SDMX is crucial for rendering the production method of EU statistics more efficient → it implements the new Eurostat Vision
The use of the SDMX standards and guidelines facilitates the exchange of data and metadata between organisations (international and national) and their comparability and consistency
The use of SDMX compliant European standards for reference and quality related metadata decrease production costs within the ESS;
SDMX provides standard formats for the exchange of data/metadata from end-to-end of the statistical business process;
SDMX provides common IT tools created by the different organisations which are freely available.

11. Costs of SDMX
Creation and maintenance of the Data Structure Definitions used in each statistical domain for data exchange
Adaptation to the standard terminology for structural metadata (code lists)
Adaptation to the European Standards for reference and quality related metadata reports
Discussions/agreements between Member States and international organisations are needed
Possible re-adaptation of the data production and/or dissemination systems needed

12. SDMX – latest progress
More emphasis on the implementation of SDMX within the ESS
Accelerated implementation in statistical domains (more and more Data Structure Definitions created)
Harmonisation of structural metadata (more and more harmonised code lists available) and reference metadata (full use of the ESMS; implementation of the ESQRS; possible future standard on process related metadata; …)
SDMX is also being implemented into IT applications (eDamis, EBB, ESS-MH, etc.)
Many training and other capacity building actions organised (for IT staff, statisticians…)

13. Summarising
SDMX is global; it provides technical and statistical standards, IT tools and an IT infrastructure.
SDMX implements the Eurostat Vision for statistical business process harmonisation.
SDMX implementation in statistical domains is progressing (e.g. the ESMS is fully implemented at Eurostat).
The SDMX implementation has some costs for production units, but the benefits should be larger over time as data and metadata exchange with Member States and other organisations will be facilitated.
For more info, please visit: or the SDMX info space
Single contact point :

14. QUESTIONS ?