1. Item 7.3 (b) SDMX for UOE data collection
Luca Gramaglia and Javier Alcántara

4. Target objectives Robust statistical modelling of UOE data collection
Streamlined validation
Transparent validation procedures
1 data provider = 1 data receiver = 1 feedback
Efficient data sharing between Eurostat-OECD-UIS
Lower IT development costs through sharing of IT tools

5. Web Services
Common
Structures
Common
Validation
Rules
Shared IT tools

6. Statistical Data and Metadata EXchange
World Bank
UNSD
SDMX
ISO TS 17369

7. SDMX is a standard to… Describe and model data: Exchange data:
Describes the concepts underlying a dataset and the relationships between them
Common vocabulary and modelling principles across statistical domains
Exchange data:
SDMX allows the construction of an IT infrastructure for data exchange
Common language…
…within and across statistical domains

8. Describing data using SDMX: the DSD
Identify the concepts involved in the datasets and their role:
Dimensions: Identify and describe the data
Attributes: Provide additional information about data. May be attached at different levels (observation, dataset)
Measure: Represents the phenomenon or phenomena to be measured

9. DIMENSIONS ATTRIBUTES MEASURES FREQUENCY COUNTRY TOURISM TOPIC TIME
OBS_VALUE
OBS_STATUS
DIMENSIONS
ATTRIBUTES
MEASURES

10. Exchanging data using SDMX
Data files must be structured according to the DSD
The preferred transmission format is SDMX-ML
XML syntax to express the DSD
Built-in capacity to work with XML Web Services
Automatisation of query/transmission of data
Standard access to centrally managed SDMX artefacts

11. How can SDMX help in UOE?
SDMX offers a standard conceptual description of the data
SDMX offers the capacity to streamline the exchange of data using Web Services
SDMX offers a language for data description on which validation rules can be defined
SDMX allows the construction of interoperable standards-based tools
Common
Structures
Web Services
Common Validation Rules
Shared IT tools
SDMX is not an end in itself: it is a means to an end

12. Implementing SDMX for UOE
Achieving the envisaged streamlined business process will require work by OECD-Eurostat-UIS on several fields (e.g. definition of common validation rules, IT infrastructure developments)
SDMX will be the backbone of this process. The first steps are:
Description of the UOE questionnaire using the SDMX conceptual model
Implementation of SDMX for data transmission

13. Implementing SDMX for UOE
Introduction of ISCED 2011: opportunity to begin modernisation process for UOE data collection
OECD-Eurostat-UIS developed an SDMX conceptual framework for the UOE questionnaire: 2 DSDs cover UOE data collection
2014 UOE Excel questionnaire are SDMX-compliant (can be converted to an SDMX file)

14. Implementing SDMX for UOE
UIS, OECD and Eurostat plan to gradually abandon Excel questionnaires in favour of SDMX compliant transmissions.
We would like to do this move with you as smoothly as possible.
As a 1st step, we will share for your review the proposed Data Structure Definitions for the UOE data collection.

15. Implementing SDMX for UOE
This proposed first version of the DSD has been embedded in the Excel questionnaires of this year's 2014 data collection as an experiment.
These questionnaires will be converted by us into SDMX-ML files, which will then be uploaded into our database (no action needed from your part).
Up to November 2014, countries are welcome to provide feedback on the DSD to be used for the data collection.

16. Implementing SDMX for UOE
In January 2015, we will update this DSD, if needed, and will come up with an implementation program.
Our goal would be to have full SDMX compliant transmission in 2016/2017, abandoning the Excel questionnaires.
The hardest part from your side is to go from Excel files to a flat file. We're here to support you!

18. Example of SDMX-ML file
Example of csv file with SDMX codes from the DSD

19. Pilot exercise for the 2014 UOE collection
Countries are encouraged to join a pilot exercise to send fully or partially this year 2014's data collection using SDMX compliant transmissions. This could take place once the Excel questionnaires have been sent. The sooner you start the readier you will be for the gradual phase-out of the Excel questionnaires!

20. Summary of the implementation process
Start in 2014, planned completion 2016/2017
Development of necessary infrastructure
Participants will be supported through specific bilateral contacts
Possible implementation of ad-hoc on-line training sessions
Support for SDMX implementation always available at