SDMX Converter Abdulla Gozalov, UNSD.

Slides:



Advertisements
Similar presentations
SDMX training session on basic principles, data structure definitions and data file implementation 29 November
Advertisements

Tutorial 11: Connecting to External Data
Working with Worksheet
Survey Data Management and Combined use of DDI and SDMX DDI and SDMX use case Labor Force Statistics.
Introduction to SPSS Edward A. Greenberg, PhD
Sdmx web services Strutural data
CountryData Technologies for Data Exchange SDMX Information Model: An Introduction.
Describing Statistical registers in SDMX and DDI: A Comparison Arofan Gregory Metadata Technology Eurostat, June 4-6, 2013 Luxembourg.
Francesco Rizzo (ISTAT - Italy) SDMX ISTAT FRAMEWORK GENEVE May 2007 OECD SDMX Expert Group.
Eurostat 1 7a. Practical use case 1: Pesticides Use Project Blanaru Cristina Eurostat Unit B5: “Central data and metadata services” SDMX Basics course,
Model and Representations
Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”
Eurostat 4. SDMX: Main objects for data exchange 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October.
Eurostat SDMX Reference Infrastructure: Tools demonstration November 2015 Eurostat Unit B3 – IT and standards for data and metadata exchange Jean-Francois.
SDMX IT Tools SDMX use in practice in NA
7b. SDMX practical use case: Census Hub
Eurostat November 2015 Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX IT Tools SDMX Converter Jean-Francois LEBLANC Christian.
Eurostat 6. SDMX: A non-technical overview of the SDMX architecture and IT tools 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services”
Eurostat Mapping Assistant May 2016 Eurostat, Unit B3 – IT solutions for statistical production Jean-Francois LEBLANC Christian SEBASTIAN 1.
Product Training Program
DTIAtlasFiberAnalyzer Tutorial
Microsoft FrontPage 2003 Illustrated Complete
4. SDMX: Main objects for data exchange
JDXpert Workday Integration
SDMX Information Model
Lesson 1: Introduction to Trifacta Wrangler
Lesson 1: Introduction to Trifacta Wrangler
TRAINING OF FOCAL POINTS ON THE CountrySTAT/FENIX SYSTEM
Exploring Microsoft® Access® 2016 Series Editor Mary Anne Poatsy
Lesson 1: Introduction to Trifacta Wrangler
Lesson 1: Introduction to Trifacta Wrangler
Practical use case of SDMX (1): Short-term Statistics (STS)
SDMX for SDGs What it means for you
SDMX Converter Raynald PALMIERI June 2015
SDMX: A brief introduction
Data collection of 2012: Data transmission standards and tools
Cross-domain concepts
2. An overview of SDMX (What is SDMX? Part I)
Eurostat – Units E2, B5 Cristina BLANARU
Lesson 1 – Chapter 1C Trifacta Interface Navigation
SDMX Tools Architecture
SDMX Reference Infrastructure
Statistical Data and Metadata eXchange (SDMX)
SDMX Information Model: An Introduction
Data Transmission Tools & Services EDAMIS, SDMX, Validation
Case Study ESTP Workshop on SDMX in National Accounts
Developing a Data Model
SDMX: an Overview Abdulla Gozalov UNSD.
SDMX Tools Overview and architecture
SDMX IT Tools SDMX Converter
SDMX as basis for water data reporting
Practical use cases of SDMX: Census Hub
ESS VIP ICT Project Task Force Meeting 5-6 March 2013.
Generate SDMX files.
SDMX IT Tools Data Structure Wizard
SDG Data Structure Definition
Item 7.3 (b) SDMX for UOE data collection
9. Practical use case 3: Pesticides Use Project
Lesson 14 Spreadsheet Software.
Eurostat Unit B3 – IT and standards for data and metadata exchange
Eurostat Unit B3 – IT and standards for data and metadata exchange
5. SDMX: General input requirements
Presentation plan Accessing and Retrieving SDMX data
Developing SDMX artefacts for data exchange, sharing and dissemination
SDMX: Frequently Asked Questions
Integrated Statistical Production System WITH GSBPM
SDG Data Structure Definition
“Argentina´s first steps in SDMX”
Presentation transcript:

SDMX Converter Abdulla Gozalov, UNSD

Software developed by Eurostat SDMX Converter Software developed by Eurostat Can be used to convert data from a variety of format into SDMX and vice versa Supports CSV, Excel, DSPL, and others Can be used to transform…. Non-SDMX data to SDMX SDMX to non-SDMX SDMX format to another SDMX format

SDMX Converter: Applications SDMX Converter is available as Desktop application with a Graphical User Interface Command-line application Web service Java library NEW: Web application Download from https://ec.europa.eu/eurostat/web/sdmx-infospace/sdmx-it-tools/sdmx-converter

Converting data to SDMX To transform data to SDMX using the SDMX Converter, you need Source data as CSV, DSPL, Excel, etc. A Data Structure Definition (DSD) according to which the SDMX dataset will be structured Mappings that show how the source data maps to the concepts of the Data Structure Definition As always in setting up SDMX exchange, configuring mappings takes the most time and effort

Using SDMX Converter with Excel Data and mappings can be placed into the same spreadsheet Additional information can be added to facilitate data entry E.g. code lists for validation and display of descriptions

Worksheet named Parameters contains mappings Worksheet names Worksheet named Parameters contains mappings Only one can be used at a time Shows how cells, rows, and columns map to the DSD dimensions and attributes Worksheet names starting with Val are ignored Can be used to store code lists or other ancillary information All other worksheets are considered to contain data and will be processed

Excel Mappings: the Parameters worksheet Mapping type Cell where data starts Concept Number of obs. columns Position or value Concept Role

Excel mappings worksheet Element: name of the DSD concept Type: role of the concept DIM: Dimension ATT: attribute DataStart: the first cell containing an observation NumColumns: number of observations per row

Column PosType: mapping or position type The following mapping/position types are supported: CELL ROW COLUMN FIX OBS_LEVEL MIXED SKIP

Mapping type: CELL The value for the entire worksheet is specified in the cell provided in the column Position E.g. if the spreadsheet is expected to only contain data for a single country, its code can be provided in a cell.

Mapping type: ROW Values for the concept are stored in the row specified in column Position

Mappings type: COLUMN Values for the concept are stored in the column specified in column Position

Mapping type: COLUMN (2) Also used with record-based representation, when row contains one record or observation, and each column holds values for one concept.

Mapping type: FIX Fixed value for the entire dataset is stored in the column Position and does not appear in the data spreadsheet E.g. if the data is always expected to be annual, frequency can be coded for the entire spreadsheet

Mapping type: MIXED The concept value is conditional Can be used to provide a default value “Use cell B3 for concept REF_AREA. If the cell is empty, use fixed value TH”

Mapping type: OBS_LEVEL Can be used in to specify attributes attached at the observation level relative to the cell containing the observation. E.g. when each row has multiple observations and their attributes. “For attribute OBS_STATUS, use cell that is 1 column to the right of the cell containing the observation value. If that cell is empty, use the value in cell H14.”

Mapping type: SKIP The concept value is not mapped Can only be used with optional attributes

Transcoding Transcoding refers to code mapping, when internal codes are different from DSD codes. With CSV and DSPL, you can configure transcodings in external files or specify them directly in the SDMX Converter. With Excel, SDMX Converter expects to find DSD codes in the spreadsheet.

THANK YOU!