Upcoming changes to the DMX technical standard

Slides:



Advertisements
Similar presentations
Status on the Mapping of Metadata Standards
Advertisements

13 September 2012 SDMX Technical Working Group1 Report of the SDMX Technical Standards Working Group SDMX Expert Group Meeting, Paris, September 2012.
National Institute of Statistics, Geography and Informatics (INEGI) Implementation of SDMX in Mexico.
An Introductory Tutorial. Background and Purpose.
SDMX is a live project for INEGI We have some advances, but a lot of work ahead As INEGI gets proficiency in the use of SDMX new challenges are coming.
Background Data validation, a critical issue for the E.S.S.
Overview of SDMX: Statistical Data and Metadata eXchange Technical and Content Standards for Statistical Data Ann McPhail, Division Chief Statistics Department,
SDMX at the IMF Progress Report Expert Group on Statistical Data and Metadata Exchange (SDMX 2007), Geneva, May 8-11, 2007 Patrick Hinderdael, Economic.
 The HTML/HTTP combination data format and exchange protocol is the foundation pillar of the World Wide Web.  Evolution of the Web (Fig. 6.12) 6.3 Unified.
CHRIS NELSON METADATA TECHNOLOGY WORK SESSION ON STATISTICAL METADATA GENEVA 6-8 MAY 2013 Designing a Metadata Repository Metadata Technology Ltd.
Web Programming : Building Internet Applications Chris Bates CSE :
February, CONTEXT  CONSTITUTIONAL AMENDMENTS  Creation of the Statistical and Geographical Information System (SNIEG)  INEGI’s Autonomy (July.
SDMX Web Services the JSON version Sami Airo & Gerard Salou.
SOAP-based Web Services Telerik Software Academy Software Quality Assurance.
Model and Representations
Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,
Eurostat 4. SDMX: Main objects for data exchange 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October.
SDMX IT Tools Introduction
1 SDMX Global Conference September 2015 SDMX into the future VTL (Validation and Transformation Language) A new technical standard for enhancing.
Google maps engine and language presentation Ibrahim Motala.
3 June 2013 SDMX Technical Working Group Luxembourg 1 WP Item 6 Expressions and Calculations.
SDMX Basics course, March 2016 Eurostat SDMX Basics course, March Introducing the Roadmap Marco Pellegrino Eurostat Unit B5: “Data and.
IAEA International Atomic Energy Agency Implementing SDMX for Energy Domain: From Discussion to Actual Implementation and Testing Andrii Gritsevskyi Oslo.
United Nations Economic Commission for Europe Statistical Division CSPA: The Future of Statistical Production Steven Vale UNECE
Statistical Information Systems Introducing SIS tool .Stat
SDMX for SDG Indicators
WEB SERVICES.
Progress Update MSIS: Bratislava, April 2005
Exchanging Reference Metadata using SDMX
Interoperable data formats: SDMX
SDMX Information Model
SDMX for SDG Indicators
MSDs and combined metadata reporting
SDMX at the BIS – from the core to the boundaries
SISAI STATISTICAL INFORMATION SYSTEMS ARCHITECTURE AND INTEGRATION
11. The future of SDMX Introducing the SDMX Roadmap 2020
Measuring Data Quality and Compilation of Metadata
Cross-domain concepts
2. An overview of SDMX (What is SDMX? Part I)
2. An overview of SDMX (What is SDMX? Part I)
The Generic Statistical Information Model
Testing RESTful Web APIs
Updates from the SDMX Sponsors and SDMX Secretariat
SDMX implementation at Eurostat and in the Member States
LOD reference architecture
SDMX in the S-DWH Layered Architecture
Statistical Information Technology
3rd WGM Meeting 3 May 2018 Item 2.3 Possible standards for ESS Validation.
ESS VIP ICT Project Task Force Meeting 5-6 March 2013.
Prepared by Peter Boško, Luxembourg June 2012
Ioannis Xirouchakis / Unit B3
SODI Live Demonstration
NIEM Tool Strategy Next Steps for Movement
The role of metadata in census data dissemination
SDMX Roadmap 2020 Emanuele Baldacci
Reportnet 3.0 Database Feasibility Study – Approach
Web Programming : Building Internet Applications Chris Bates CSE :
5. SDMX: General input requirements
Presentation plan Accessing and Retrieving SDMX data
Validation Activities in the ESS What you will hear today…
Developing SDMX artefacts for data exchange, sharing and dissemination
14. SDMX: Global and regional implementation projects
SDMX: Frequently Asked Questions
SDMX IT Tools SDMX Registry
SDMX in AFRICA SDMX Roadmap th SDMX Global Conference
Palestinian Central Bureau of Statistics
SDMX Roadmap 2020: Achievements, status and future outlook
SDMX for Sustainable Development Goals Indicators: The Way Forward
A non-statistical data exchange scenario
Presentation transcript:

Upcoming changes to the DMX technical standard SDMX Global Conference 2-5 October Addis Ababa Prepared by: Juan Muñoz López SDMX Technical Working Group (SDMX-TWG)

VTL (Validation and Transformation Language) VTL lets incorporate the exchange of validation and transformation rules to the exchange of data and metadata, so you can assure data accuracy and correctness and calculate derived data. VTL 1.0 was published in 2015 A new improved version will be published by the end of 2017 After publishing this version the task team will continue making improvements and adding new operations Implementations of the language are planned to be developed Current Technical Specification documents can be found in: https://sdmx.org/?page_id=5096

Code List Extensions A Data Structure Definition (DSD) contains the Code Lists that describe possible values for the Dimensions. If you modify a code-list contained in a DSD you will need to define a new one. Some code lists are very dynamic, so managing DSD versions could be quite complicated. A task team composed by members from the TWG and from the SWG is working on implementing a way to let manage most common changes to code lists without needing to define a new DSD. This work will derive in some adjustments to the SDMX Information Model, but the task team is taking care to keep compatibility with current version

SDMX-JSON Format JSON which stands for JavaScript Object Notation, is a standard language (RFC 4627) based on JavaScript which is supported by the main Web Browsers (Chrome, Edge, Fire Fox, Internet Explorer, Opera, etc.) A SDMX-JSON format makes easier to create appealing visualizations of dataflows, improving the dissemination process The format specification has been publicly reviewed and it is in a final adjustments period to be delivered A draft version of the format can be found in: https://sdmx.org/wp-content/uploads/sdmx-json-data- message.pdf

SDMX-CSV Format Comma Separated Values (CSV) is a standard format (IETF RFC 4180) widely supported by almost all commercial tools used in the statistical community. An SDMX-CSV format will improve interoperability and make easier using the information for different purposes. As CSV is a light weight format it is easier to exchange large amounts of information, one required step for the exchange of microdata. SDMX Information Model addresses weakness of the CSV format. DSD provides the tools to maintain a well defined (and interpretable) dataflow. On September 15th 2017 the public review of the specification ended, and now the task team is working on improving it by applying received feedback. Current proposal can be accessed in: https://github.com/sdmx-twg/sdmx-csv

SDMX RESTful web services API specification (version 1.2.0) Web Services are essential for Machine to Machine (M2M) communication, they provide a way to directly connect applications and exchange information in an automated way SDMX supports two kinds of Web Services: SOAP which are based on XML messages RESTful which are based on HTTP requests The new version lets retrieve a single hierarchy within an hierarchical codelist, and several improvements to the documentation. Documentation, tutorial and list of changes can be obtained from: https://github.com/sdmx-twg/sdmx-rest

Technical priorities Attention to the SDMX 2020 roadmap actions Providing solutions to technical issues impeding SDMX implementation Extending usefulness of SDMX for different sceneries (new and emergent) Maintaining SDMX relevant in the context of new technologies and user needs: OpenData, Big Data, IoT, geo-referencing, etc. Fitting the standard into the statistical modernization projects and making it suitable to be inside the statistical standards ecosystem Improving technical documentation Maintaining a close relation with the conceptual development of the standard to ensure adequate support Maintaining technical integrity and compatibility of the technical standards

Contact: contact@sdmx.org