CASE STUDY: IMPLEMENTING SDMX EXCHANGE WITH MEMBER COUNTRIES IN SHORT-TERM ECONOMIC STATISTICS (STES)

OECD receives Short-term economics (STES) data from all member countries through established data flows Data flows originally set up as “Main Economic Indicators” flows. The main focus for this data has shifted to online dissemination, hence the new name (STES) STES data contains series from several domains (BOP, Labour force, Employment,...) Project Background

Project aim is to improve the STES data collection process: – Use a standard, open exchange format from all data providers – Reduce processing overhead: Reduce tool count, use of standard, existing tools. Reduce maintenance overhead – Improve quality in the data transmission by making validation of data relatively easy – Based on bilateral data flows, goal is not a Global DSD Project Background

Consulted with STES content and data collection experts Mapped existing flow concepts and codes to SDMX concepts and code lists Created the DSDs and added them to the STES public Clearspace site for easy distribution Sent the series lists and DSDs to the countries and offered technical assistance First draft of STES DSD

STES Frequency Reference area Subject Measure Unit Time period ATTRIBUTES Unit multiplier Observation status First Draft Concepts

Created DSD from existing flow’s series codes rather than taking opportunity to remodel Didn’t normalize the DSD’s concepts. For example Subject, Age and Sex were combined as Subject concept Tried to keep everything in 1 DSD – Most series don’t need Age and Sex dimensions, only required for Labour series – There is a logical split between Labour and the other series Issues in first draft DSD:1

Each country had specific Subjects, so created per- country DSDs containing shared + country-specific subjects – Many DSDs, maintenance nightmare – Non-standard use of SDMX using annotations to denote shared series and the country-specific series Things preventing compatibility with Eurostat registry – Missing ConceptSchemeVersion – KeyFamily ID was the same as ConceptScheme ID Used code lists external to DSD file – Data Structure Wizard tool didn’t support external code lists when creating xml-schema Issues in first draft DSD:2

Created DSD from existing flow’s series codes rather than taking opportunity to remodel Consulted with SDMX experts on how to remodel Didn’t normalize the DSD’s concepts. For example Subject, Age and Sex were combined into Subject concept Split the merged concepts into separate concepts (Subject, Age, Sex) Tried to keep everything in 1 DSD – Most series don’t need Age and Sex dimensions, only required for Labour series – There is a logical split between Labour and the other series Split STES DSD into 2 DSDs: STES and INFRA_ANNUAL_LABOR with different, relevant ConceptSchemes. Tidier, more coherent structure Resolve issues in first draft DSD:1

Created per country DSDs containing shared + country- specific subjects – Maintenance nightmare, many DSDs – Non-standard use of SDMX using annotations to denote shared series and the country-specific series Removed country-specific DSDs. All subjects merged into same code list. Specific series communicated separately Things preventing compatibility with Eurostat registry – Missing ConceptSchemeVersion – KeyFamily ID was same as ConceptScheme ID ConceptSchemeVersion added. KeyFamily ID and ConceptScheme ID made unique Used code lists external to DSD file – Data Structure Wizard tool didn’t support external code lists when creating xml-schema Moved code lists back into DSD file Resolve issues in first draft DSD:2

STESINFRA_ANNUAL_LABOUR Frequency Reference area Subject MeasureAge Sex Adjustment Unit Time period ATTRIBUTES Unit multiplier Observation status Second Draft Concepts

Need early consultation with SDMX standards experts before creation and dissemination of DSDs. If you are an expert yourself, a second opinion on the DSD is very useful. Don’t just use existing data flow schemas. Think about how to structure the data better, make reporting easier, and make DSD future maintenance relatively easy Use DSD best practices – guidelines from SDMX SWG Avoid merging of concepts and normalize them, at least to begin with Provide better, more timely support for participating providers. Ensure that resources are available to respond to questions in a timely manner Some lessons learned during first version of DSD

They are willing but don’t have resources The concern of future maintenance cost (one more system to support) See initial investment in SDMX as heavy burden (training, infrastructure, change for users) In some cases SDMX business case is not seen as strong enough for take-up Challenges faced by providers

Provide SDMX training courses and material. Promote SDMX-RI as a ready-to-go, free technical architecture to disseminate SDMX from existing infrastructure OECD will promote SDMX-RI as the main SDMX technical platform for dissemination and offer technical assistance for installing and use Possible solutions for these challenges

2 DSDs exist: STES and INFRA_ANNUAL_LABOUR. Downloadable from STES Clearspace site Marco Oksman is main contact for STES project We created a test mapping from our database to STES DSDs using SDMX-RI mapping store as proof-of-concept dissemination At various stages of implementation with several countries STES Meeting in this room immediately afterwards – you are welcome to join STES Project Status

STES Pilot status by country CountryStatus Czech RepublicData available now via SDMX web service (test) IsraelData available via files from web site ItalySome series available BrazilData available now via SDMX web service (test) MexicoImplementation ongoing South KoreaImplementation ongoing SlovakiaImplementation ongoing Bank of TurkeyImplementation ongoing SloveniaHelping them map their PC-Axis files. Support for implementation of SDMX-RI In dialogue with many other countries

Thank you STES Clearspace site: For more information contact Marco Oksman and David Barraclough