13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management of statistical data and metadata BP-12-15

13-Jul-07 Name of the presentation Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

13-Jul-07 Name of the presentation Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

13-Jul-07 Name of the presentation SDMX Information model Content- oriented guidelines IT infrastructure for exchange and sharing The SDMX Components

13-Jul-07 Name of the presentation SDMX Information model The SDMX Components Describe statistics in a standard way Main object: Data Structure Definition (DSD) –Statistical concepts (Dimensions and attributes) –Code lists Central Management –SDMX Registry Allows IT tools to validate –Structure of data –Codes used

13-Jul-07 Name of the presentation Content- oriented guidelines The SDMX Components Cross Domain Concepts (Annex 1) –Reporting area –Currency of denomination –Frequency of collection Statistical Domains (Annex 3) –UNECE framework: –Demographic and social –Economic –Environmental and multi-domain Cross Domain Code Lists (Annex 2) –Country, currency, frequency,… Metadata Common Vocabulary (Annex 4) –Statistical terminology –Term, definition, context, source

13-Jul-07 Name of the presentation IT infrastructure for exchange and sharing The SDMX Components PUSH architecture Initiated by data provider: –Generates data file –Transmits it to receiver –Example: EDAMIS HUB architecture –Special implementation of pull –Distributed data warehouse –Data only transmitted on (end user) request –Example: Census Hub PULL architecture Initiated by data receiver: –Provider opens web service –Receiver picks up data –Example: SDMX Reference Infrastructure

13-Jul-07 Name of the presentation SDMX Information model Content- oriented guidelines IT infrastructure for exchange and sharing...not just a transmission format The SDMX Components

13-Jul-07 Name of the presentation World Bank UNSD SDMX ISO TS Statistical Data and Metadata EXchange

13-Jul-07 Name of the presentation Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

13-Jul-07 Name of the presentation 11 Eurostat SDMX Implementation Process

13-Jul-07 Name of the presentation Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

13-Jul-07 Name of the presentation Research & Development Health Education Crop National Accounts Balance of Payments Census Aviation Fisheries Waste Some Eurostat implementations (data exchange)

13-Jul-07 Name of the presentation Project Status BOP (Mar 2012) Roadmap approved, project on time for EU purposes Technical work ongoing Deliverables defined –Domain concepts – high level inventory Status: delivered and approved –Detailed analysis and coding Status: ongoing, eta June/July 2012 Shared concepts with NA done first by both groups Pilot implementations planned 2013

13-Jul-07 Name of the presentation Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

13-Jul-07 Name of the presentation SDMX Implementation in Member States Tools offeredAction Web FormsNo costs for organisation Manual work for senders (type, copy/paste) SDMX Converter (sender’s PC) Installation on sender’s PC Manual work for senders (convert) SDMX Converter (batch mode) Installation on server SDMX Reference Infrastructure Installation on server Mapping of database to DSDs Own SDMX Implementation Local development in organisations

13-Jul-07 Name of the presentation SDMX Implementation in Member States Example SDMX Converter 1.Generate CSV or GES 2.Take DSD and data file 3.Produce SDMX-ML All steps each time Example SDMX-RI 1. Take DSD 2. Link to database 3. Produce SDMX-ML Step 1 & 2 only once Only step 3 each time

13-Jul-07 Name of the presentation SDMX Implementation in Member States Central management of structures and code lists  SDMX Registry Notification on changes possible Web service interaction possible –Downloading structures and code lists from registry –Opening pull mode for data to international organisations

13-Jul-07 Name of the presentation Data Validation Process Before/During Transmission (EDAMIS – “First Level”) –Performed in Member States –Format Check (SDMX-ML) –Code Check (SDMX DSD) After Transmission ( “Second Level”) –Performed in Eurostat –Detailed value check –Mirror check –…

13-Jul-07 Name of the presentation Support for data providers SDMX Reference Infrastructure –Installed in ~20 countries (NSIs) for Census Hub –Can be extended to other domains (Balance of Payments) 1st contact point Local Transmission Coordinators –In National Statistical Institutes –Experience with EDAMIS and SDMX 2nd contact point Eurostat support teams

13-Jul-07 Name of the presentation Answered? What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do? More information Official SDMX Web Site – Eurostat SDMX Info Space – SDMX Support Team