1. SDMX DATA STRUCTURE DEFINITION SDMX Training BANK INDONESIA 16-18 SEPTEMBER 2015 YOGYAKARTA, INDONESIA

2. Session Objectives At the end of this morning you will: Know the SDMX model of a data structure definition Understand the techniques to identify the structure of data Identify the concepts in a simple data set Be able to develop a simple data structure definition

3. Session Objectives At the end of this session you will: Know the SDMX model of a data structure definition Understand the techniques to identify the structure of data Identify the concepts in a simple data set Be able to develop a simple data structure definition

4. Data Set

6. Extract from a spreadsheet

7. What’s stopping us processing this data Outside of a spreadsheet processor? Not easy to process text comparison language What is the text e.g. where is the date, country, unit of measure?

8. Data Set http://www.ecb.int/stats/services/escb/html/index.en.html

9. Web site What is on here that is not on the spreadsheet?

10. What’s stopping us processing this data 0utside of a spreadsheet processor? Not easy to process text comparison language What is the text e.g. where is the date, country, unit of measure? Have we lost any information? Metadata Hierarchy

11. What are we missing? The structure of the data What is this? Key

12. Data Structure Key – what is it and does it mean? These are values for what part of the structure?

13. The Key of the Data Dimensions Identify some dimensions Country Frequency Adjustment + others

14. Dimensions NAC Data Structure Definition Key Dimensions BOP Data Structure Definition Key Dimensions Country Frequency Adjustment + others Country Frequency Adjustment + others what’s wrong here?

15. The Key of the Data Dimensions Is “Country” “Frequency”, “Adjustment” relevant to other structures? How do we enable this? Are we missing something? We therefore need Concepts that are independent of use in data structures (and metadata structures)

16. Concepts ESA Data Structure Definition Key Dimensions BOP Data Structure Definition Key Dimensions Country Frequency Adjustment + others Concept uses

17. The Key of the Data Dimensions Is “Country” “Frequency”, “Adjustment” relevant to other structures? We therefore need Concepts that are independent of use in data structures (and metadata structures) What else does a Dimension need?

18. The Key of the Data Dimensions Is “Country” “Frequency”, “Adjustment” relevant to other structures? We therefore need Concepts that are independent of use in data structures (and metadata structures) What else does a Dimension need? Specification of valid content Code Lists or non-coded format (e.g. integer)

19. Data Set Structure: Concepts and Code Lists Code Lists GDP Indicator B1QG00 Gross domestic product at market prices F33200 Long-term securities other than shares TOTEMP Total employment COUNTR Y Adjustment N Neither seasonally nor working day adjusted S Seasonally adjusted, not working day adjusted T Trend CONCEPTS Country GDP Indicator Adjustment Concepts I6 EU 17 BE Belgium DE Germany

20. Representation has code list Code List concepts that identify the observation Data Structure Definition Key Dimensions has format takes semantic from Representation Coded Non- coded Concept

21. What else is required to define a Data Structure?

22. What else is required to define a Data Structure Additional Metadata

23. Attributes has code list Code List Attributes concepts that add metadata has format concepts that identify the observation Data Structure Definition Key Dimensions Concept takes semantic from has format takes semantic from Representation Coded Non- coded Attribute Relationship

24. Anything Else? observations

25. has code list Code List Attributes concepts that add metadata has format concepts that identify the observation Data Structure Definition Key Dimensions Concept Measure(s) takes semantic from has format takes semantic from has format concepts that are observed phenomenon Representation Coded Non- coded Attribute Relationship Measures

26. What do we need in order to be able to process this in a computer system?

27. Data Set Structure Computers need to know the structure of data in terms of: Dimensionality Additional metadata (Attributes) Measures (Observation) Concepts Valid content Code Lists Non coded format (integer, date, text)

28. Concepts play roles in a Data Structure Comprises –Concepts that identify the observation value –Concepts that add additional metadata about the observation value (as a value or the context of the value) –Concept that is the observation value –Any of these may be coded text date/time number etc. Dimension s Attributes Measure Representation

29. ESA.Q.BE.Y.0000.B1QG00.1000.TTTT.L.N.P. 2000Q4 = 1.0 Data Makes Sense 1.0

30. Data Makes Sense – what are we missing?

31. Attributes Attribute Relationship

32. Q. What is required to do this? A. Referencing Mechanism

33. Attribute Relationship Q. What is required to do this? A. Referencing Mechanism

34. Attribute Relationship ESA.Q.BE.Y.0000.B1QG00.1000.TTTT.L.N.P.2000Q4 = 1.0 Do we have a referencing mechanism? Q. What is the referenced “object” A. A specific Dimension Value

35. Attribute Relationship ESA.Q.I6.Y.0000.P60000.1000.TTTT.L.N.P.2000Q1 = 0.7

36. has code list Code List Attributes concepts that add metadata has format concepts that identify a partial key concepts that identify the observation Data Structure Definition Key Group Key Dimensions Concept Measure(s) takes semantic from has format takes semantic from has format concepts that are observed phenomenon Representation Coded Non- coded Attribute Relationship Group Key Dimension(s) Data Set Observation

37. Where Are We? specification of cube sub-set in terms of sub set of valid content valid content in terms of structure (dimensions, attributes, measures) data discovery data providers Dataflow Data Structure Definition

38. Where Are We Data Structure Definition Code List Concept Dimension Attribute Measure references

39. Design a DSD: What do we need to do first? Identify the Concepts

40. Questions?