SDMX Implementation The National Accounts use case ESTP Training Course 8 & 9 April 2014 Fabien JACQUET Eurostat B5

Part I International Data Co-operation

Outline Background The data sharing model The necessary preconditions Conclusions

1. Background General objectives SDMX provides Reduce reporting burden on NSIs More efficient use of resources in International Organisation (IO) Ensure high quality and consistency of data of official statistics Improve global data exchange and dissemination SDMX provides technical/statistical standards new exchange modes (hubs) clear rules and responsibilities

European statistics: From national to Eurostat 2. The data sharing model Eurostat Data Validation EU Member state EU Member state EU Member state EU Member state European statistics: From national to Eurostat

2. The data sharing model U S E R EU countries OECD countries (non-EU countries only) Other contries (non-OECD countries only) Eurostat - ECB OECD IMF, UN, WB, ILO, BIS, other IOs U S E R Eurostat as international hub for European statistics

3. The necessary pre-conditions Internationally agreed statistical standards (e.g. SNA 2008, ESA 2010, BPM6, UOE questionnaire, manuals..) Internationally agreed SDMX data structures Global SDMX data structures for exchanging national data from national to international and for exchanging/disseminating from international to international organisation.

3. The necessary pre-conditions SDMX maintenance agreements Ownership and maintaining agency to be defined (normally between the international organisations) Internationally agreed data validation Data validation rules and SDMX data validation standards (beyond file structure validation) Streamlined data exchange processes Frequency and timeliness for data dissemination, data quality management etc.

4. Conclusions International data co-operation simplifies the national/international data/metadata exchange and dissemination processes; International data cooperation broadly uses SDMX; International data co-operation agreements are progressing in some statistical areas (e.g. National Accounts) Statistical integration needs to go hand-in-hand with technical integration and standardisation.

Part II SDMX and National Accounts

Excel questionnaires GESMES files BOP National Accounts Main Aggregates Sector Accounts Supply & Use Input / Output GFS FDI Excel questionnaires GESMES files

Why SDMX for National Accounts? Within Eurostat Within the ESS Internationally

Why SDMX for National Accounts? Within Eurostat Within the ESS Internationally Within Eurostat Within the ESS Internationally

National Accounts business needs Within Eurostat Within the ESS Internationally Main trigger: new methodology SNA 93  SNA 2008 ESA 95  ESA 2010 Business process reengineering Corporate production systems Integration of production chains Harmonised coding

National Accounts business needs Cross-domain integration Balance of Payments Foreign Direct Investment ESS Business process reengineering Standardisation Improved validation Reuse of tools Within Eurostat Within the ESS Internationally

National Accounts business needs Data quality 1 country  1 year  1 GDP Shared validation rules Across all organisations Lowering reporting burden 1 data flow from national to international International cooperation Inter-Agency Group on Economic and Financial Statistics = SDMX Sponsors Example: PGI Website http://www.principalglobalindicators.org Within Eurostat Within the ESS Internationally

Cross Domain Concepts and Code Lists Shared Concepts and Code Lists National Accounts SNA 2008 ESA 2010 Other NA related domains FDI statistics Other domains Balance of Payments Statistics Cross Domain Concepts and Code Lists Shared Concepts and Code Lists

Data Validation Process Before/During Transmission (“First Level”) - Covered by SDMX today - Format Check (SDMX-ML) - Code Check (SDMX DSD) After Transmission ( “Second Level”) - Not yet covered by SDMX - Detailed value check - Mirror check - …

SDMX Implementation in NA Project Cycle

General Approach Definition of domain concepts Grouping Coding cross-domain shared domain specific Coding Draft DSD Review

Reporting matrix  SDMX objects CL Constraints Data Flow Concept Scheme Code Lists Data Structure Definitions

Two review phases determined Step I: Content review Integrate and clarify the data structures and use – as much as possible - SDMX concepts and code lists or shared concepts and code lists; Involve all stakeholders on an early stage; Agree on the data structure matrix to be used for defining the DSDs (data structure definitions) used for data exchange; Step II. Technical review Translate the data structure matrix into SDMX DSDs and assure production and exchange capabilities;

Always needed: Data Structure Definitions (DSDs) DSD ID DSD Title Tables in transmission programs from 2014 MAIN Main aggregates ESA / OECD: 1, 3, 5, 20, 22 SEC Sector accounts and government finance ESA / OECD: 2, 6, 7, 8, 9,11, 26, 27, 28 ECB: MUFA, GFS EDP: 1, 2, 3, 4 OECD: 119, 7HH, 7II, PSD G20: Recommendation 15 SU Supply / use and input / output statistics ESA : 15, 16, 17, 18, 19 REG Regional statistics ESA : 10, 12, 13 PENS Pension schemes ESA: 29

Technical Review Phase – Contents Pilot package (April) DSD Guidelines DSD Matrix Pilot 1 issue log Questionnaire Samples for table 1 Excel package (May) Samples for most tables Related SDMX-ML files Tables missing (GFS): 2, 9, 11, 25, 27, 28 Excel 2 SDMX converter module, pilot version (on demand) Phase out in National Accounts

Technical Integration National statistical organisations International organisations National databases (Mapping) Source SDMX Data Structure Defintions & Data Flows National data sources Metadata driven process based on SDMX standards and tools Full automation of the data exchange possible

Data provider view: One goal – different possibilities Not applicable in National Accounts Webforms EDAMIS Single Entry Point SDMX-ML file Database export SDMX Converter Database SDMX Reference Infrastructure SDMX-ML file SDMX Web Service Same DSD  Same Format Excel sheets Phase out in National Accounts SDMX Converter SDMX-ML file

Conclusion SDMX statistical and technical standards & SDMX IT tools and IT infrastructure provide the "toolkit" for the harmonisation and integration of data structures, the rationalisation of national/international data flows, finally improving the data quality of official statistics.  Just give up a few of your old codes 