1. Eurostat November 2015 Eurostat Unit B3 – IT and standards for data and metadata exchange SDMX IT Tools SDMX Converter Jean-Francois LEBLANC Christian SEBASTIAN

2. Eurostat Table of Contents 1.Objectives 2.What is the Converter 1.Design/Implementation 2.Minimum requirements 3.Supported formats 3.Interfaces 1.GUI 2.CLI 3.Common API 4.Web Services 2

3. Eurostat Table of Contents 4.SMDX Converter vs SDMX-RI 5.Where is the Converter 6.Hands-on exercise 3

4. Eurostat 1. Objectives Illustrate how to use the Converter List all the supported formats Explain its interfaces Clarify when it is recommended State its limitations 4

5. Eurostat 2. What is the Converter The SDMX converter is a JAVA application that converts between all the following formats* SDMX 2.0 o Generic, Compact, Utility And Cross-sectional. SDMX 2.1 o Generic Data, Generic TS, Structure-specific, Structure-specific TS GESMES o TS, 2.1, DSIS CSV, FLR MESSAGE GROUP (special SDMX 2.0 format) DSPL Excel *Limitations may apply. Please, check the User Manual. 5

6. Eurostat 2.1 Design/Implementation Steps to proceed a conversion Reading the input message o Parsing of the message o Populating the data model of the tool o (based on the SDMX information model) Reading the DSD o The DSD is retrieved from the Registry in order to complete the conversion o The DSD can be loaded from files so no connection is needed Writing the converted message o Uses the data model to write the output message in the target format 6

7. Eurostat 2.2 Minimum requirements 1.Input file 2.Output file (complete path) 3.Format for input and output files 4.Specify DSD 1.DSD file 2.Reference to a DSD file in the Registry 3.Reference to a Dataflow file in the Registry 7

8. Eurostat 2.3 Supported formats 8 OTHERS CSV FLR DSPL EXCEL SDMX 2.1 GENERIC DATA 2.1 GENERIC TS DATA 2.1 STRUCTURE SPECIFIC DATA 2.1 STRUCTURE SPECIFIC TS DATA 2.1 SDMX 2.0 GENERIC SDMX COMPACT SDMX UTILITY SDMX CROSS-SECTIONAL SDMX MESSAGE GROUP GESMES GESMES TS GESMES 2.1 GESMES DSIS Limitations may apply. Please, check the User Manual.

9. Eurostat 3. Interfaces Web service 9 User interface Command line API

10. Eurostat 3.1 GUI (Converter 5.1.0) 10 2.a Select the DSD in the local drive 2.b Identify a DSD to download from the SDMX Registry (configuration required) 2.c Identify a dataflow linked to the DSD to download from the SDMX Registry (configuration required) CSV parameters XML parameters for SDMX output formats 1. Selection of the input/output files and their format 6. SDMX (output) validation 2. If the local DSD includes multiple versions, we can specify the one derired 2. If the local DSD includes multiple versions, we can specify the one desired 3. SDMX header (.prop file) Only for flat and excel files 4. Mapping 5. CSV quotation 3. Excel parameter file

11. Eurostat 3.1.1 Example 11 2. Format1. Input and output files 3. DSD file or reference

12. Eurostat 3.1.2 Header 12

13. Eurostat 3.1.3 Change mapping 13

14. Eurostat 3.1.4 Transcoding 14

15. Eurostat 3.2 CLI Converter [Options] InputFile OutputFile InputFormat OutputFormat Converter Windows OS: converter.bat Unix OS: converter.sh Options -reg, -dsd_file, -dsd_id, -dsd_agency, -dsd_version, -df, -df_id, -df_version, -df_agency, -header_file, -date_format, -level, -mapping_file, -ordered_input, -trans_file, -delimiter, header_row For further information check the User Manual page 84. 15

16. Eurostat 3.2.1 Example in Windows 16

17. Eurostat 3.3 Common API 17

18. Eurostat 3.4 Web Service A web service also exists for the Converter Based on Java Can be installed on Tomcat Server or Weblogic A Test Client is provided to test the Web Service conversions (only available for Windows) 18

19. Eurostat 3.4.1 Wsdl 19

20. Eurostat 3.4.2 Web Service client 20

21. Eurostat 4. SMDX Converter vs SDMX-RI SDMX ConverterSDMX-RI Standalone applicationNeeds to be installed on a server File repositoryConnected to dissemination DB Generates SDMX files from input files Generates SDMX files from customized SDMX queries 21

22. Eurostat 5. Where to find the SDMX Converter You can download the latest version of the SDMX converter on CIRCABC https://circabc.europa.eu/w/browse/6b2323b6- 0d4e-43dc-bc7e-dd5e9ff958cc Available packages SDMX Converter Platform Independent SDMX Converter Web Service SDMX Converter installer for Windows 32-bit SDMX Converter Documentation 22

23. Eurostat 6. Hands-on exercise Conversion using the Common API in JAVA. Eclipse Java EE (or equivalent) SDMXSource code (www.sdmxsource.org)www.sdmxsource.org Input file and DSD 23

24. Eurostat SDMX Converter 24