Implementation of SDMX for data and metadata exchange SDMX Basics Course 12 April 2013 Daniel Suranyi Eurostat B5 Management of statistical data and metadata.

Slides:

Advertisements

Similar presentations

Eurostat Progress report SDMX-NA ESTAT, Unit C.2.

Advertisements

Introduction to SDMX Seminar Eurostat/ECLAC 02 October 2012 August Götzfried Head of Unit, Eurostat B5 Management of statistical data and metadata.

Better data quality through global data and metadata sharing

SDMX – AN OECD PERSPECTIVE Paul Schreyer OECD CCSA Special Session, September 2014 Rome.

Eurostat Unit B3 – Statistical Information Technologies Data transmission tools and services 15/05/ eDAMIS The standard solution for transmitting.

Global SDMX Implementation Experience from on-going projects Daniel Suranyi, Eurostat Project Officer for SDMX implementation SDMX Expert Group,

Overview of SDMX: Statistical Data and Metadata eXchange Technical and Content Standards for Statistical Data Ann McPhail, Division Chief Statistics Department,

Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX Basics Training – 2012 IT architectures for data exchange SDMX-RI and the Hub approach.

B A C K G R O U N D B R I E F I N G A N D N E X T S T E P S METIS Geneva, February 2004 Statistical Data and Metadata Exchange Initiative.

Eurostat Global Implementation of SDMX in National Accounts Committee for the Coordination of Statistical Activities 22 nd Session 4-6 September Ankara,

13-Jul-07 Implementation of SDMX for data and metadata exchange Balance of Payments Working Group 2-3 April 2012 Daniel Suranyi Eurostat B5 Management.

1 Meeting on the Management of Statistical Information Systems (MSIS 2010) SDMX architecture for data sharing and interoperability Francesco Rizzo, ISTAT,

ESCWA SDMX Workshop Session: Reporting Scenarios.

Basics David Barraclough OECD SDMX Coordinator

Eurostat 1 7a. Practical use case 1: Pesticides Use Project Blanaru Cristina Eurostat Unit B5: “Central data and metadata services” SDMX Basics course,

Model and Representations

Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”

Eurostat 4. SDMX: Main objects for data exchange 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October.

SDMX IT Tools Introduction

SDMX and Metadata SDMX Basics Course 12 April 2013 Daniel Suranyi Eurostat B5 Management of statistical data and metadata.

2.An overview of SDMX (What is SDMX? Part I) 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.

1 5a. SDMX and reference metadata exchanges Bogdan ZDRENTU Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.

IS4STAT Vittorio Viaggi Eurostat

SDMX IN DATA COLLECTION AND DATA DISSEMINATION Workshop on Statistical Data Collection, Washington DC, 29 April - 1 May 2015.

SDMX IT Tools SDMX use in practice in NA

Eurostat 1.SDMX: Background and purpose 1 Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October 2015.

7b. SDMX practical use case: Census Hub

1 National Statistical Coordination Board Philippine Implementation Plan of SDMX Overview of the Philippine Implementation Plan of SDMX Model and Technology.

Implementation of SDMX for Balance of Payments Balance of Payments Working Group 9-10 April 2013 BP Daniel Suranyi Eurostat B5 Management of statistical.

CCSA session on SDMX implementation Werner Bier and Per Nymand-Andersen European Central Bank CCSA session on SDMX implementation, FAO headquarters, 11.

Eurostat 1 3.An overview of the SDMX implementation process Edward Cook Eurostat Unit B5: “Central data and metadata services” SDMX Basics course,

IT Directors’ Group Meeting October 2010 Sharing data validation tools in the ESS Christine WIRTZ – Head of Unit B3 Georges PONGAS – Unit B3 Daniel.

Eurostat Reporting every number only once: International data cooperation for National Accounts Walter Radermacher Director-General Eurostat.

15-16 December 2010 CGST Meeting 1 IT Developments TRIS 1 – TRIS 1 / TRIS 2 Item 7.1 on the agenda 1 TRIS = TRansport Information System.

Implementation of SDMX for data and metadata exchange SDMX Basics Course October 2012 Daniel Suranyi Eurostat B5 Management of statistical data and.

Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”

IAEA International Atomic Energy Agency Implementing SDMX for Energy Domain: From Discussion to Actual Implementation and Testing Andrii Gritsevskyi Oslo.

The evolution of the SDMX infrastructure and services

SDMX Opportunities MED Meeting 14 May 2013 Daniel Suranyi Eurostat B5

Usage of National Reference Metadata Editor (NRME)

SDMX: A brief introduction

SDMX Introduction and practical exercises

Disseminating statistics: Internet and Publications course

2. An overview of SDMX (What is SDMX? Part I)

Time Use Survey data processing and dissemination 17 July 2014

Eurostat – Units E2, B5 Cristina BLANARU

SDMX Implementation for PESTICIDES

2. An overview of SDMX (What is SDMX? Part I)

Workshop on ESA 2010 transmission programme – What and how?

Task Force on Annual Financial Accounts

Data Transmission Tools & Services EDAMIS, SDMX, Validation

Implementation of SDMX in the ESS

SDMX: an Overview Abdulla Gozalov UNSD.

SDMX Tools Overview and architecture

Statistical Information Technology

SDMX as basis for water data reporting

Sharing data validation activities in the ESS.

August Götzfried Eurostat unit B 4

ESS VIP ICT Project Task Force Meeting 5-6 March 2013.

SDMX : General introduction H. Linden, Eurostat, Unit B5

Working Group "Education and Training Statistics" April 2013

SDMX IT Tools SDMX use in practice in NA

Item 7.3 (b) SDMX for UOE data collection

SDMX Implementation The National Accounts use case

1. SDMX: Background and purpose

13. Knowing more about SDMX

EDIT data validation system Ewa Stacewicz EUROSTAT VALIDATION TEAM

14. SDMX: Global and regional implementation projects

Daniel Suranyi, Krassimir Ivanov

Presentation transcript:

Implementation of SDMX for data and metadata exchange SDMX Basics Course 12 April 2013 Daniel Suranyi Eurostat B5 Management of statistical data and metadata

What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do? Questions

IT Infrastructure for exchange and sharing Content Oriented Guidelines SDMX Information Model The SDMX Components 3...not just a transmission format !

SDMX – opportunities 4 Streamline data flows Central management (SDMX Registry) Software tools Data sharing Data structures, concepts, code lists Across domains Across organisations

Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

6 Eurostat SDMX Implementation Process 6

Questions 7 What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

JVS/LCI (Job Vacancy Statistics/ Labour Cost Index) Maritime STS (Short Term Statistics) TEC (Trade by Enterprise Characteristics) International Trade* National Accounts* R&D* (Research & Development) Balance of Payments* EGR IS (EuroGroups Register Identification Service) Aviation Fisheries* Orchards Pesticides Waste Eurostat project status (data exchange) * Other international organisations involved

Implementation projects – Some milestones DomainInternational actorsProduction phase FisheriesUN (FAO)2011 Balance of PaymentsSDMX Sponsors (IMF coordinating) 09/2014 National AccountsSDMX Sponsors (Eurostat coordinating) 09/2014 Research & Development OECD, UNESCO~2014 International TradeUNSD, OECDTo be defined

Census HICP/ Compliance monitoring Pesticides R&D (Research & Development) Waste (additional workflows) ESAW Tourism Waste COMEXT (3rd countries) COSAEA Education (Pilot phase) JVS/LCI (Job Vacancy Statistics/ Labour Cost Index) PRODCOM HICP LFS-Q EU-SILC FATS STS (Short Term Statistics) LFS-A Eurostat implementation projects (metadata exchange) * Other international organisations involved

Questions What is SDMX? Why use it? How do we get there? Where are we today? What can data providers do?

SDMX Implementation in Member States 12 Tools offeredAction Web FormsNo costs for organisation Manual work for senders (type, copy/paste) SDMX Converter (sender’s PC) Installation on sender’s PC Manual work for senders (convert) SDMX Converter (batch mode) Installation on server SDMX Reference Infrastructure Installation on server Mapping of database to DSDs Own SDMX Implementation Local development in organisations

SDMX Implementation in Member States 13  Example SDMX Converter 1.Generate CSV or GES 2.Take DSD and data file 3.Produce SDMX-ML  All steps each time  Example SDMX-RI (installed in ~20 NSIs for Census Project) 1.Take DSD 2.Link to database 3.Produce SDMX-ML  Step 1 & 2 only once  Only step 3 each time

Data Validation Process  Before/During Transmission (“First Level”) - Performed in Member States - Format Check (SDMX-ML) - Code Check (SDMX DSD)  After Transmission ( “Second Level”) - Performed in Eurostat - Detailed value check - Mirror check - …

Support for data providers SDMX Reference Infrastructure Installed in ~20 countries (NSIs) for Census Hub Can be extended to other domains 1st contact point - Local Transmission Coordinators In National Statistical Institutes Experience with EDAMIS and SDMX 2nd contact point - Eurostat support teams

Questions? 16 SDMX Support Team SDMX Website Eurostat SDMX Info Space