Item 7.3 (b) SDMX for UOE data collection Luca Gramaglia and Javier Alcántara

Target objectives Robust statistical modelling of UOE data collection Streamlined validation Transparent validation procedures 1 data provider = 1 data receiver = 1 feedback Efficient data sharing between Eurostat-OECD-UIS Lower IT development costs through sharing of IT tools

Web Services Common Structures Common Validation Rules Shared IT tools

Statistical Data and Metadata EXchange World Bank UNSD SDMX ISO TS 17369

SDMX is a standard to… Describe and model data: Exchange data: Describes the concepts underlying a dataset and the relationships between them Common vocabulary and modelling principles across statistical domains Exchange data: SDMX allows the construction of an IT infrastructure for data exchange Common language… …within and across statistical domains

Describing data using SDMX: the DSD Identify the concepts involved in the datasets and their role: Dimensions: Identify and describe the data Attributes: Provide additional information about data. May be attached at different levels (observation, dataset) Measure: Represents the phenomenon or phenomena to be measured

DIMENSIONS ATTRIBUTES MEASURES FREQUENCY COUNTRY TOURISM TOPIC TIME OBS_VALUE OBS_STATUS DIMENSIONS ATTRIBUTES MEASURES

Exchanging data using SDMX Data files must be structured according to the DSD The preferred transmission format is SDMX-ML XML syntax to express the DSD Built-in capacity to work with XML Web Services Automatisation of query/transmission of data Standard access to centrally managed SDMX artefacts

How can SDMX help in UOE? SDMX offers a standard conceptual description of the data SDMX offers the capacity to streamline the exchange of data using Web Services SDMX offers a language for data description on which validation rules can be defined SDMX allows the construction of interoperable standards-based tools Common Structures Web Services Common Validation Rules Shared IT tools SDMX is not an end in itself: it is a means to an end

Implementing SDMX for UOE Achieving the envisaged streamlined business process will require work by OECD-Eurostat-UIS on several fields (e.g. definition of common validation rules, IT infrastructure developments) SDMX will be the backbone of this process. The first steps are: Description of the UOE questionnaire using the SDMX conceptual model Implementation of SDMX for data transmission

Implementing SDMX for UOE Introduction of ISCED 2011: opportunity to begin modernisation process for UOE data collection OECD-Eurostat-UIS developed an SDMX conceptual framework for the UOE questionnaire: 2 DSDs cover UOE data collection 2014 UOE Excel questionnaire are SDMX-compliant (can be converted to an SDMX file)

Implementing SDMX for UOE UIS, OECD and Eurostat plan to gradually abandon Excel questionnaires in favour of SDMX compliant transmissions. We would like to do this move with you as smoothly as possible. As a 1st step, we will share for your review the proposed Data Structure Definitions for the UOE 2015 data collection.

Implementing SDMX for UOE This proposed first version of the DSD has been embedded in the Excel questionnaires of this year's 2014 data collection as an experiment. These questionnaires will be converted by us into SDMX-ML files, which will then be uploaded into our database (no action needed from your part). Up to November 2014, countries are welcome to provide feedback on the DSD to be used for the 2015 data collection.

Implementing SDMX for UOE In January 2015, we will update this DSD, if needed, and will come up with an implementation program. Our goal would be to have full SDMX compliant transmission in 2016/2017, abandoning the Excel questionnaires. The hardest part from your side is to go from Excel files to a flat file. We're here to support you!

Example of SDMX-ML file Example of csv file with SDMX codes from the DSD

Pilot exercise for the 2014 UOE collection Countries are encouraged to join a pilot exercise to send fully or partially this year 2014's data collection using SDMX compliant transmissions. This could take place once the Excel questionnaires have been sent. The sooner you start the readier you will be for the gradual phase-out of the Excel questionnaires!

Summary of the implementation process Start in 2014, planned completion 2016/2017 Development of necessary infrastructure Participants will be supported through specific bilateral contacts Possible implementation of ad-hoc on-line training sessions Support for SDMX implementation always available at ESTAT-SUPPORT-SDMX@ec.europa.eu