1. Implementation of SDMX for data and metadata exchange SDMX Basics Course 12 April 2013 Daniel Suranyi Eurostat B5 Management of statistical data and metadata

2. What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do? Questions

3. IT Infrastructure for exchange and sharing Content Oriented Guidelines SDMX Information Model The SDMX Components 3...not just a transmission format !

4. SDMX – opportunities 4 Streamline data flows Central management (SDMX Registry) Software tools Data sharing Data structures, concepts, code lists Across domains Across organisations

5. Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

6. 6 Eurostat SDMX Implementation Process 6

7. Questions 7 What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

8. JVS/LCI (Job Vacancy Statistics/ Labour Cost Index) Maritime STS (Short Term Statistics) TEC (Trade by Enterprise Characteristics) International Trade* National Accounts* R&D* (Research & Development) Balance of Payments* EGR IS (EuroGroups Register Identification Service) Aviation Fisheries* Orchards Pesticides Waste Eurostat project status (data exchange) * Other international organisations involved

9. Implementation projects – Some milestones DomainInternational actorsProduction phase FisheriesUN (FAO)2011 Balance of PaymentsSDMX Sponsors (IMF coordinating) 09/2014 National AccountsSDMX Sponsors (Eurostat coordinating) 09/2014 Research & Development OECD, UNESCO~2014 International TradeUNSD, OECDTo be defined

10. Census HICP/ Compliance monitoring Pesticides R&D (Research & Development) Waste (additional workflows) ESAW Tourism Waste COMEXT (3rd countries) COSAEA Education (Pilot phase) JVS/LCI (Job Vacancy Statistics/ Labour Cost Index) PRODCOM HICP LFS-Q EU-SILC FATS STS (Short Term Statistics) LFS-A Eurostat implementation projects (metadata exchange) * Other international organisations involved

11. Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

12. SDMX Implementation in Member States 12 Tools offeredAction Web FormsNo costs for organisation Manual work for senders (type, copy/paste) SDMX Converter (sender’s PC) Installation on sender’s PC Manual work for senders (convert) SDMX Converter (batch mode) Installation on server SDMX Reference Infrastructure Installation on server Mapping of database to DSDs Own SDMX Implementation Local development in organisations

13. SDMX Implementation in Member States 13  Example SDMX Converter 1.Generate CSV or GES 2.Take DSD and data file 3.Produce SDMX-ML  All steps each time  Example SDMX-RI (installed in ~20 NSIs for Census Project) 1.Take DSD 2.Link to database 3.Produce SDMX-ML  Step 1 & 2 only once  Only step 3 each time

14. Data Validation Process  Before/During Transmission (“First Level”) - Performed in Member States - Format Check (SDMX-ML) - Code Check (SDMX DSD)  After Transmission ( “Second Level”) - Performed in Eurostat - Detailed value check - Mirror check - …

15. Support for data providers SDMX Reference Infrastructure Installed in ~20 countries (NSIs) for Census Hub Can be extended to other domains 1st contact point - Local Transmission Coordinators In National Statistical Institutes Experience with EDAMIS and SDMX 2nd contact point - Eurostat support teams

16. Questions? 16 SDMX Support Team ESTAT-Support-SDMX@ec.europa.eu SDMX Website http://www.sdmx.org Eurostat SDMX Info Space https://webgate.ec.europa.eu/fpfis/mwikis/sdmx