1. Global Merchandise Trade SDMX Data Structure Definition Overview Markie Muryawan United Nations Statistics Division WCO IT Conference and Exhibition © 2015

2. General SDMX Information 2

3. What is it? What‘s SDMX? – SDMX = Statistical Data and Metadata Exchange – In layman’s term = standard data format for statistics – Standard in both statistical and technical aspect – Sponsored by 7 organizations: BIS - ECB - EUROSTAT - IMF - OECD - UN - World Bank SDMX consists of – Technical standards (including Information Model), – Statistical guidelines, and – An IT architecture and tools. 3

4. Why SDMX Why uses SDMX? – To reduce international reporting burden – To facilitate data processing – To increase efficiency of international data exchange – ISO Standard: 17369:2013 (SDMX v.2.1) In 2008, UN Statistical Commission (UNSC) recognized and supported the SDMX standards and guidelines – And SDMX sponsors report regularly to UNSC since 2002 (see the latest E/CN.3/2013/30 and E/CN.3/2015/33).E/CN.3/2013/30 E/CN.3/2015/33 4

5. Data Structure Definition (DSD) A DSD is a central component of the SDMX. It specifies: – i) a set of concepts which describe and identify the data; – ii) the attachment level of each of these concepts as well as their status (mandatory or conditional); and – iii) code lists which provide acceptable values for the concepts and their standard coding and formatting. 5

6. Latest SDMX Development - 2015 Three global data structure definitions: – System of National Accounts (SNA) – Balance of Payments (BOP) – Foreign Direct Investment (FDI) SDMX Global Registry https://registry.sdmx.org, consists of internationally agreed: https://registry.sdmx.org – Code list – Concept Scheme – Data structure definition 6

7. Latest SDMX Development (cont.) Ongoing development of SDMX data structure definition for statistics on – research and development – Education – short-term business – government finance and – international trade 7

8. Some example of SDMX implementations Eurostat collects census and national account data via SDMX The European System of Central Banks exchanges statistics on BOP in SDMX OECD receives short-term economic statistics through SDMX web services IMF Special Data Dissemination Standard Plus required submission of national data in SDMX A joint initiative between the AfDB and IMF provides a country- specific cloud-based data dissemination platform that converts data into SDMX BIS and some 60 central banks operate of its statistical data exchange based on SDMX UNSD established SDMX exchange of data linked to MDG, and it’s expected the similar approach for post-2015 indicators 8

9. SDMX technical development Draft specification for SDMX-JSON Design new validation and transformation language Working on common IT tools, http://www.sdmxtools.org/ e.g., Eurostat Reference Infrastructure http://www.sdmxtools.org/ 9

10. Now, Trade SDMX 10

11. SDMX-IMTS Working Group Established in 2013 [UN, OECD, Eurostat, ITC, UNCTAD] UNSD acts as secretariat for the Working Group The purpose is to specify uniform structures, concept definitions and code lists for the transmission of IMTS data and metadata Comply with latest technical specification (SDMX v.2.1) and follow the latest recommendations on IMTS (IMTS 2010) 11

12. UN Comtrade: Analysis of data flow 12 UN Comtrade Register 1 Eurostat Original commodity data Pre- process 4 Normalize 5 Aggregate 7 Publish 9 Tariffline data Processed data Published data Data Brokers Reported totals Supplementary data National Data Provider OPEC WTO IMF Down- load 2 Tariffline data user Published data user Convert 8 Original commodity data user Disseminate original data 3 Disseminate tariffline data 6 Disseminate published data 10

13. Latest update – May 2015 Participation of OECD, Eurostat, UN, ITC, UNCTAD Data flows analysed (previous slide) Concept scheme created for the whole domain. 31 concepts total!!! Most concepts coded, still work-in-progress, will be discussed during global consultation Concepts mapped to Data flows resulting in 3 groups of dimensionality. – DSD 1: All concepts, for data collection – DSD 2: Most concepts, for public dissemination – DSD 3: For aggregate data, and indices Concepts classified into 18 dimensions, 12 attributes and 1 measurement 13

14. Dimensions (total of 18) 14 FREQ TIME_PERIOD REF_AREA TRADE_FLOW COMMODITY_1 COMMODITY_1_CONF COMMODITY_2 COMMODITY_2_CONF COMMODITY_CUSTOM_BREAKDOWN COUNTERPART_AREA_1 COUNTERPART_AREA_1_CONF COUNTERPART_AREA_2 COUNTERPART_AREA_2_CONF TRANSPORT_MODE_BORDER CUSTOMS_PROC ECONOMIC_ACTIVITY TRANSFORMATION OBS_TYPE Annual/Quarterly/Monthly/etc. Reporting Period Reporting Country, mostly ISO country codes Exports/Imports/etc. HS-all editions, SITC all versions, BEC Confidentialized commodity code HS-all editions, SITC all versions, BEC Confidentialized commodity code Extension of commodity (tariff line level) Trading partner 1, mostly ISO country codes Confidentialized primary partner area Trading partner 2, mostly ISO country codes Confidentialized additional partner area Mode of transport Customs procedures codes, Revised Kyoto Economic Activity/Industry based on ISIC Transformation performed on OBS_VALUE CIF-type value, FOB-type value, net weight, gross weight, supplementary quantity, price index, unit value index, volume index

15. 15 TIME_PERIOD_START_DATE UNIT_MULT UNIT_MEASURE COMMENT_OBS TRADE_SYSTEM COMMODITY_CUSTOM_BREAKDOWN_CODE COMMODITY_CUSTOM_BREAKDOWN_DESC COUNTERPART_AREA_1_TYPE COUNTERPART_AREA_2_TYPE COUNTERPART_AREA_1_ANNOTATION COUNTERPART_AREA_2_ANNOTATION OBS_STATUS Reference Period Start Date Unit multiplier Unit of measure Comments to the observation value Trade System Custom commodity breakdown code Custom commodity breakdown description Type of primary partner area Type of additional partner area Annotation of primary partner area Annotation of additional partner area Observation Status Attributes (total of 12)

16. Some examples 16 DSD with basic data

17. 17

18. 18

19. Some examples 19 DSD with dual commodity classifications

20. 20

21. Some examples 21 DSD with full commodity breakdown – tariff line level

22. 22

23. Some examples 23 DSD with confidential data when country does NOT provide details for internal purposes

24. 24

25. 25

26. Some examples 26 DSD with confidential data when country does provide details for internal purposes

27. 27

28. 28

29. Simplified example of one single trade record 29 Implementation using SDMX-XML

30. 30 Codes used in this example: HS12_010121: "Pure-bred breeding horses" HS12_0101XX: "Live horses, asses, mules and hinnies // Confidential Item" SITC4_0015: "Horses, asses, mules & hinnies, live" ISIC4_014: "Animal production" _Z = "Not applicable" _X = "Not allocated / unspecified"

31. Roadmap 31 Completed Visualization/Analysis of data flows Definition of list of concepts to be covered for the reporting framework Review of existing Code lists in other domains Draft concept scheme for IMTS Draft code lists for IMTS Work in Progress Worldwide consultation with the countries (May-June 2015) Finalize a first global reporting framework for IMTS Gradual implementation of version 1 of global reporting framework for IMTS Agreement on maintenance and governance Further refinement of DSD, especially for DSD3

32. Global Consultation on IMTS DSD From 8 May 2015 – 30 June 2015 National agency responsible for IMTS Review of the following aspects: – Concepts (dimensions and attributes): How do the concepts defined apply to your practices? Is there any gaps? – Code lists: Are the code lists of all concepts comprehensive and appropriate? Is there a need to omit/include/modify any category? – Implementation of SDMX: What are the challenges and requirements? 32

33. Reflection Full implementation of SDMX would benefit everyone (e.g., harmonized coding for all data items) However, it’s required additional investment (people, time) and probably upgrade of IT existing system How to better align the work on Trade SDMX and WCO Data Model? 33

34. Thank you Please send your comments to muryawan@un.org