1. SDMX Training
Bank Indonesia
16-18 September 2015
Yogyakarta, Indonesia
SDMX implementation aspects: getting started, practicalities and maintenance process

2. Checklist
This checklist is taken from a draft of a document under development by the sdmx statistical working group
Please note that the document is still under development and review by the swg

3. Inspiration taken from the following documents
How to Model a Statistical Domain for Data Exchange in SDMX (under drafting): These guidelines outline general principles on how to design and create SDMX artefacts in a statistical domain, following a step-by-step approach based on the SDMX information model. The document is also providing recommendations on the number of Data Structure Definitions (DSDs) for a subject-matter domain, following the approach that the decision on the number of DSDs needed for describing a statistical domain should come after - and as a consequence of - the discussion on the data model and the architectural approach to be followed in the data collection exercise. Contact point : SDMX Statistical Working Group at
Guidelines for the Creation and Management of SDMX Code Lists (under revision): These guidelines are intended to support the creation of Codelists to be used all along the statistical business process and in particular when SDMX is implemented in statistical domains. They are strongly recommended for use when SDMX-compliant Data Structure Definitions (DSDs) are built-up and implemented in statistical domains. Contact point : SDMX Statistical Working Group at
SDMX Glossary (under revision): Previously known as the Metadata Common Vocabulary (MCV) the revised SDMX Glossary will be more specifically SDMX-oriented and will cover a) SDMX cross-domain Concepts (i.e. standard concepts which should be used across statistical domains wherever possible to enhance exchange and comparison possibilities), b) the technical terminology from the Implementation Model and other techncial specifications, and c) various important statistical concepts. Contact point : SDMX Statistical Working Group at
Global and/or local registries: See under Section 2. above.
Examples of Global DSD matrices: Examples of Global DSD matrices as well as overviews of artefacts used in a Global DSD can be found in existing Global DSDs (e.g. for National Accounts: Balance of Payments and Foreign Direct Investment:
Handbook for Statisticians on how to use SDMX: This document was drafted in 2012 in the framework of an ESSnet contract sponsored by Eurostat. This handbook is aimed at people working in the production services but it is also intended to those metadata experts that are responsible for variable and classification definitions used for data dissemination. It is also addressed to those readers already having some knowledge of the SDMX standard and that want to have only a hands-on approach on data modelling. Any depth investigation on SDMX is left to the official documentation. [Link to be added]

4. Lifecycle

5. 1.3 Assign roles and responsibilities (preliminary)
1. Initiate
1.1 Acquire a basic understanding of SDMX standards and guidelines, and implementation tools
1.3 Assign roles and responsibilities (preliminary)
1.2 Identify stakeholders (within organisation, across NSIs and at international level)
1.4 Draft a project charter

6. 2.3 Specify general context of data exchange
2. Plan and Organize
2.3 Specify general context of data exchange
2.2 Assign roles and responsibilities
2.1 Organize kick-off meeting
2.4 Develop work breakdown and project schedule
2.5 Miscellaneous planning and organizational issues

7. 3.1 List and describe data flows to be covered
3. Design
3.6 Design derived variables and transformation formulae
3.1 List and describe data flows to be covered
3.2 Elaborate concept scheme
3.3 Build DSD matrix
3.4 Optimize matrix
3.7 Design associated Metadata Structure Definitions (MSDs)
3.5 List artefacts to be developed

8. 4.2 Develop SDMX artefacts
4 Develop and Test
4.1 Determine fitness for purpose of existing implementations with a view to saving time and resources, and reusing as much as possible existing material (e.g. Concept Schemes, Code Lists)
4.2 Develop SDMX artefacts
4.3 Develop associated Metadata Structure Definitions (MSDs)
4.4 Draft documentation
4.5 Test
4.6 Make corrections
4.7 Review documentation

9. 5.1 Upload information to output systems
5 Implement
5.1 Upload information to output systems
5.2 Formalize maintenance agreements
5.3 Launch information campaign
5.4 Provide support, training and capacity building

10. 6.1 Gather evaluation inputs
6 Evaluate
6.1 Gather evaluation inputs
6.2 Conduct evaluation
6.3 Agree an action plan
6.4 Implement action plan

11. 1. Initiate
1.1 Acquire a basic understanding of SDMX standards and guidelines, and implementation tools
1.3 Assign roles and responsibilities (preliminary)
1.2 Identify stakeholders (within organisation, across NSIs and at international level)
1.4 Draft a project charter
One effective way of learning about SDMX and its benefits is to implement a small scale pilot study on a statistical domain using the available global Data Structure Definitions and suites of implementation tools

12. Consult existing documentation
1. Initiate
1.1 Acquire a basic understanding of SDMX standards and guidelines, and implementation tools
Consult existing documentation
One effective way of learning about SDMX and its benefits is to implement a small scale pilot study on a statistical domain using the available global Data Structure Definitions and suites of implementation tools
Attend webinars and training sessions, capacity building sessions, international conferences
Install, test and get used to basic tools to produce XML files

13. 1.3 Assign roles and responsibilities (preliminary)
1. Initiate
1.3 Assign roles and responsibilities (preliminary)
Agree on a project manager or project management cell (general project management, organization of meetings, calendar keeper, monitoring and control, reporting, etc.)
Assign roles and responsibilities of all other stakeholders
1.4 Draft a project charter
1.2 Identify stakeholders (within organisation, across NSIs and at international level)
Senior management (Experience has shown that it is essential that senior management drives SDMX)
Subject-matter specialists
Experts in SDMX standards and technical implementation
Experts in code lists, classifications and methodology
Coordination / quality unit
Data dissemination unit
IT specialists

14. 2.1 Organize kick-off meeting
2. Plan and Organize
2.1 Organize kick-off meeting
Define project scope
Set expectations of all key project stakeholders
Identify project risks
Discuss project plans
2.2 Assign roles and responsibilities (final)
2.3 Specify general context of data exchange
Domains: Single, multi
Purpose: Single, multi
Type of data:
data and metadata, metadata only
microdata, macrodata
one-dimensional data set, multi-dimensional data set, time series
Level of data exchange
within organization
between national organizations
between national and international organizations
between international organizations

15. 2.3 Specify general context of data exchange
2. Plan and Organize
2.3 Specify general context of data exchange
Type of data recipient: human, machine
Location of data recipient
Role in data exchange
collection
provider
DSD maintenance
end user
Process pattern: bilateral, gateway, data sharing
Data and metadata exchange mode: Push, pull
Data and metadata architecture
data warehouse (repository) architecture
hub architecture
data producer architecture
registry
Frequency of data exchange

16. 2.4 Develop work breakdown and project schedule
2. Plan and Organize
2.4 Develop work breakdown and project schedule
Break down the project into smaller and more manageable components such as deliverables, work packages, activites, and tasks
Project schedule:
identify dependencies between tasks
assign resources for each task
identify task start and end dates
work out the overall project duration
Distribute plans to stakeholders and discuss them

17. 3.1 List and describe data flows to be covered
3. Design
3.1 List and describe data flows to be covered
3.2 Elaborate concept schemes
List all concepts needed for the domain observed
Map/align chosen concepts to SDMX concepts
Consolidate concepts
Group concepts: Cross-domain concepts, shared consepts, specific concepts
Add default representation to concepts (drawing on existing concept schemes)
3.3 Build DSD matrix
Put data flows in relation to the concept scheme (i.e. which concept is used in which data flow)
If SDMX implementations have already taken place in neighbouring domains, consider synergy and integration opportunities (e.g. extension of existing DSDs instead of development of new ones)

18. Fill blanks in the matrix with default codes
3. Design
3.4 Optimize matrix
Fill blanks in the matrix with default codes
3.5 List artefacts to be developed
3.6 Design derived variables and transformation formulae
Check which SDMX official artefacts are already readily available (e.g. cross-domain concepts, cross-domain code lists)
3.7 Design associated Metadata Structure Definitions (MSDs)

19. 4.2 Develop SDMX artefacts
4. Develop and Test
4.1 Determine fitness for purpose of existing implementations with a view to saving time and resources, and reusing as much as possible existing material (e.g. concept schemes, code lists)
4.2 Develop SDMX artefacts
Derive DSDs from the matrix: 1 DSd per set of dimensionality
Optimize the number of DSDs by single-fixing data flows to a code where required
Develop domain-specific concepts and code lists
Create dataflows
Create constraint artefacts
4.3 Develop associated Metadata Structure Definitions (MSDs)
4.4 Draft documentation
4.5 Test
4.6 Make corrections
4.7 Review documentation

20. 5.1 Upload information to output systems
5. Implement
5.1 Upload information to output systems
Upload artefacts to the appropriate registry
Resolve issues
Validate and sign off
Disseminate documentation
5.2 Formalize maintenance agreements
Draft agreement on the governance of the SDMX artefacts
Provide overview of all artefacts relating to a particular statistical domain
Provide details about the maintenance of artefacts over time
Mention any other maintenance aspects relevant to te domain concerned
5.3 Launch information campaign
8.2 Conduct evaluation (GSBPM 8.2)
Post announcement on ad hoc website(s)
Make announcements at meetings

21. 5.4 Provide support, training and capacity building
5. Implement
5.4 Provide support, training and capacity building
8.2 Conduct evaluation (GSBPM 8.2)
Organise training sessions (e.g. webinars)
Provide helpdesk (e.g. functional address, forum)

22. 6.1 Gather evaluation inputs
6. Evaluate
6.1 Gather evaluation inputs
Determine persons / team to conduct evaluation
Gather inputs required for evaluation
6.2 Conduct evaluation
Conduct detailed analysis and evaluation of all gathered inputs
Produce report detailing finding, and recommendations for improvement
6.3 Agree on action plan
8.2 Conduct evaluation (GSBPM 8.2)
Present evaluation report to appropriate boards for discussion
Agree on action plan for either implementing or amending the proposed recommendations
Set up metrics to monitor the success and benefits derived from implementing recommendations
6.4 Implement action plan

23. Proof of Concept
The lead unit within an organisation and / or lead agency within the data exchage community would also take responsibility for organising a proof of concept pilot study that would demonstrate the benefits of SDMX implementation.
Which unit / area within the organisation would be the primary contact and main driver for the pilot implementation project - e.g. IT unit; training area; subject matter area; data dissemination or coordination units?
The key and most time consuming process for SDMX implementation is the harmonisation of structural metadata that will facilitate the exchange of data.

24. Reference Metadata Determine
what metadata currently exists;
whether it is structured and uses a standard metadata template that covers a number of domains – refer below;
whether the national structural metadata concepts are consistent with those used in the SDMX Content-Oriented Guidelines, i.e. for the Cross Domain Concepts located in the SDMX Glossary;
where it is stored. In many countries metadata are frequently stored and disseminated in Word files or via PDFs. 
Options for such a template are the use of: 
the template used for the IMF’s Special Data Dissemination Standard (SDDS);
Eurostat’s ESMS or Single Integrated Metadata Structure; or
a national metadata template.

25. Questions?