1. B A C K G R O U N D B R I E F I N G A N D N E X T S T E P S METIS Geneva, February 2004 Statistical Data and Metadata Exchange Initiative

2. 1  Challenges and Opportunities  What is SDMX?  Progress  Next steps Overview

3. 2  Exchange of statistical data is still complex, resource intensive, and expensive  Various international organisations have individual approaches for their constituencies  Uncertainties about how to proceed with new technologies (XML, web services …) Challenges

4. 3 International Organisations Regional Organisations accounts statistics Banks, Corporates Individual Households trans- actions accounts National Statistical Organisations accounts statistics 180 + Countries Internet, Search, Navigation www.z.org www.hub.org www.x.org www.y.org Statistical information building blocks

5. 4  Constituencies of international organisations have asked for common solutions  International organisations are in a position to expand their cooperation  IT technology seems to offer some ways forward that we should not ignore Opportunities

6. 5  Initiative of BIS, ECB, EUROSTAT, IMF, OECD, World Bank and UN  “... to focus on business practices in the field of statistical information that would allow more efficient processes for exchange and sharing of data and metadata within the current scope of our collective activities”  Global workshop at IMF, reports to UN Statistical Commission, liaison with international standards community and private sector initiatives  www.sdmx.org What is SDMX? (1)

7. 6

8. 7

9. 8

10. 9  Projects (shared funding and resources) launched in June 2002  batch data exchange  metadata common vocabulary  metadata repositories  a practical case study on emerging e-standards What is SDMX? (2)

11. 10  Overall goal: reality check on emerging standards by building prototype.  Closely coordinated work with other projects – exploring ways to validate practical impact of project deliverables.  Builds on experimentation taking place in different SDMX institutions.  Support from other institutions and central banks. Case Study

12. 11 Case Study: general approach  Selected the BIS-IMF-OECD-World Bank hub on external debt for the demo example  Reviewed available and emerging standards  Established experimental XML standards based on existing statistical exchange standards (i.e. GESMES/TS metadata “key family” concepts)  Built a metadata registry and populated it with links to where the data can be found  Built web services (registration, search, transformation, comparison)

13. 12 Case Study: key findings  Technology could be made to work better for statistical data exchange  Common information model underlies all statistical data  Common standards cut across different patterns of data exchange: bilateral, gateway, data sharing  Potential of (new) web technology is significant  XML is widely supported by software vendors, making it easier to use off-the-shelf tools to develop applications  registry-based architecture for metadata can facilitate navigation (i.e. data google)  use of web services enables automated data exchange  Possible to build on existing infrastructures, technologies and standards

14. 13 Example of information model: metadata “key family” Country Stock/Flow Unit Multiplier Unit Topic Time/Frequency

15. 14 Simple tool for creating a key family (Demo)

16. 15 Spreadsheet input to XML output (Demo)

17. 16 Patterns of Data Exchange  Bilateral  individual provider-to-consumer links  could be batch or web dissemination  variety of format standards  Gateway  many bilateral-to-central authority then on to others (bilaterally)  still many bilateral formats  typical in data bank exercises  Data sharing  statistical hubs through common standards and formats  data/metadata useable without prior agreement  dissemination could replace reporting

18. 17 GESMES/TS Reporting & Registry-Based Dissemination (1) Database EDI Gateway EDI Gateway Database GESMES/TS Data (1)Today, the sender renders the data out of the database as GESMES/TS EDIFACT, and sends it to a counter-party according to prior agreement. When received by the counter-party, it goes through a gateway and is passed into their internal systems.

19. 18 GESMES/TS Reporting & Registry-Based Dissemination (2) Database EDI & SDMX-ML Gateway EDI & SDMX-ML Gateway Database Transmission of GESMES/TS & SDMX-ML Data SDMX-ML Gateway Website Registration SDMX-ML Gateway Notification Retrieval of SDMX-ML Data Registry

20. 19 Possible registry-based infrastructure Registry Classification Scheme Key family Dataset Registry Services - Registration Statistics User ConsumerPublisher IMF Website World Bank Website OECD Website BIS Website Central Bank Website Key family metadata Search, Retrieve, Transform Dataset

21. 20 Web Services for statistical information exchange and sharing Web Service Service/Data Provider Registry Web Service Service/Data ConsumerService/Data Locator

22. 21

23. 22

24. 23

25. 24

26. 25

27. 26 Selecting Data Category to Search (Demo)

28. 27 Results of Registry Search – Comparison View (Demo)

29. 28  Reduced reporting burden – via common formats adopted by international organisations for data and metadata exchange  More user-friendly access when publishing national data and metadata on the web - via global standards for data formats, catalogs/registries and associated services  Enhanced opportunity for navigation, discovery and comparative data analysis – via global guidelines for metadata vocabularies and repositories linked to statistical information in common formats  Can replicate these models and tools for statistical information systems at national levels Anticipated benefits

30. 29  Transition from projects to SDMX deliverables, on a learn- by-doing basis  metadata standards  technical standards  Pilot for Joint External Debt Hub (involving cost sharing by participating international organisations)  to implement initial version 1.0 standards  to implement re-usable and freely available tools and registries to provide the basis for capacity building, fostering the adoption of SDMX standards  to implement a pilot for creditor and debtor statistics based on these standards and tools Envisaged next steps

31. SDMX Standards Tools Registries Repositories Conformance Tests Monitor Projects / Activities Training / Capacity Building Copyright and Intellectual Property Contacts/Liaison National Accounts SDMX Metadata Standards Metadata Common Vocabulary Balance of Payments External Debt Information Model Architecture International Sponsoring Institutions Formats (EDI, XML) Aligned Subject-Matter Activities (institutions and other fora adopting SDMX standards) www.sdmx.org Monetary and Financial Aligned Implementation Activities (institutions and other fora adopting SDMX standards) Core Subject-Matter Domains use input SDMX Technical Standards examples deliverables Core Statistical Concepts Joint External Debt Hub input OECD.Stat BIS Data Bank Comtrade NAWWE examples

32. Standards Tools Registries Repositories Conformance Tests Monitor Projects / Activities Training / Capacity Building Copyright and Intellectual Property Contacts/Liaison SDMX Metadata Standards Metadata Common Vocabulary External Debt Information Model Architecture Formats (EDI,XML) www.sdmx.org External Debt Joint Hub Core Subject-Matter Domains input SDMX Technical Standards deliverables Core Statistical Concepts input / use use / input input / use Aligned Subject-Matter Activity UN Statistical Commission: Inter-Agency Task Force on Finance Statistics (institutions and other fora adopting SDMX standards) Aligned Implementation Activity BIS-IMF-OECD-World Bank (institutions and other fora adopting SDMX standards) SDMX Standards-setting (Version 1.0) - Joint External Debt Hub Project: Overview deliverables use / input

33. 32 More about SDMX to follow in Working Paper 7 : Inter-agency cooperation for the dissemiantion and exchange of standard metadata www.sdmx.org