1. Exchanging Reference Metadata using SDMX
Towards a Global Reference Metadata Reporting Method
I’m presenting this but the Global DSDs have required a great collaboration at all levels in many different organisations – involving senior management, statisticians, technicians, project managers, methodologists, etc.

2. Some Essential Terms
Metadata is information about data. Needed to interpret and identify the data
Structural Metadata is information about the structure of the data. Examples:
The “shape” of a DSD such as the dimensions
The Code Lists IDs and Descriptions
The list of attributes
Reference Metadata is information about data that is designed for humans to understand more about the data, and can be sent separately to the data
Examples: Contact information; The source of the data; Release policy; Quality aspects
Attributes are information required to process and interpret the data. They need to be sent with the data, for example:
The status of an observation: Break in Series; Estimated; Missing
Embargo time: When the data will be available for publication
Confidentiality Status: Is the data internal only or can it be disseminated?
Unfortunately we often say simply “Metadata” and we may be referring to either type

3. The Problem Statement There is very little exchange of ref metadata
Manual processes are involved that are labor-intensive and error-prone
Organisations have very different reference metadata frameworks (list of types of ref metadata)
IMF have a collection of metadata concepts
Eurostat have SIMS
OECD has something that hasn’t got a name (42 types so it could be the meaning of life!)
This makes it difficult for reporters to supply ref metadata
Therefore it doesn’t happen
The lack of harmonisation means that there is a lack of tools

4. The Proposal Make exchange of ref metadata possible using SDMX
Identify the SDMX cross-domain concepts that are related to ref metadata (they’re already in the SDMX Glossary)
Provide a Global Reference Metadata Concept Scheme (MCS)
This is an abstract artefact, therefore you will need…
Also create a Global Metadata Structure Definition (MSD)
This is used to define what goes into a metadata dataset/message but may not serve the purposes of all statistical domains, so…
Describe how to derive custom Metadata Structure Definitions from the Global Metadata Concept Scheme when needed

5. Difference between a Concept Scheme and Data or Metadata Structure Definition
Concept Scheme is the set of concepts (dimensions, attributes) to fully describe the concept or framework:
Labour Statistics Concept Scheme has all concepts to describe Labour Statistics
Balance of Payments Concept Scheme has all concepts to describe BPM
A reference metadata Concept Scheme has all concepts to describe a reference metadata framework (SIMS at Eurostat, DQAF at IMF, Global for SDMX community
The Concept Scheme is abstract, it cannot be used to create messages
Meta/Data Structure Definition is used to create messages
Derived from a Concept Scheme. Example: National Accounts has 5 DSDs that all use concepts from the National Accounts Concept Scheme
It reuses concepts from the Concept Scheme for related DSD/MSDs
Metadata Structure Definition will derive a subset of concepts from the Metadata Concept Scheme

6. Some concepts in the Global Metadata Concept Scheme
Title
Confidentiality
Time coverage
Valuation
Data source
Confidentiality - data treatment
Data presentation - Summary description
Data compilation
Contact
Confidentiality - policy
Data presentation - Detailed description
Adjustment
Contact address
Professionalism
Source data type
Data validation
Contact fax number
Professionalism - code of conduct
Data collection method
Data revision
Contact mail address
Professionalism - impartiality
Time period
Data revision - policy
Contact name
Professionalism - methodology
Time period - collection
Data revision - practice
Contact organisation
Professionalism - statistical commentary
Base period
Data revision - studies
Contact organisation unit
Cost and burden
Base weight
Imputation
Contact person function
Cost and burden - efficiency management
Frequency of data collection
Statistical population
Contact phone number
Cost and burden – resources
Classification system
Quality management
Institutional mandate
Statistical concepts and definitions
Quality management - quality assessment
Institutional mandate – data sharing
Reference area
Statistical unit
Quality management - quality assurance
Institutional mandate - legal acts and other agreements
Sector coverage
Accounting conventions
Quality management - quality documentation

7. The Global MCS and Derivations
Global MCS & Global MSD
Custom MSD (implementable subset of Global MCS +extensions)
Custom MSD (implementable subset of Global MCS concepts)

8. Examples and Metadataflows

9. Mapping to the Global MCS
Statistical
Office
Int.
Org.
Ministry/
Bureau
Central
Bank

10. Implementation Example

11. Implementation User Stories: Statistical agency/dissemination perspective
The agency currently stores its reference metadata in a database and has an extraction method. To output ref metadata to the Global MCS
Keep the existing metadata system components with no changes
Create a mapping between the agency’s existing metadata concepts and the Global MCS concepts
Use a proxy component to do the following:
a. Take input from the existing agency’s metadata component
b. Transform the existing format to SDMX
c. Apply the mapping created in step 2.
d. Output the resulting SDMX that matches the Global/Derived MSD
Further work could improve efficiency:
Change the agency’s metadata format to SDMX to avoid step 3b
Align the agency’s metadata framework to the Global MSC to avoid step 3c

12. David Barraclough david.barraclough@oecd.org
Thank you
David Barraclough
SDMX Official Website:
Technical Working Group:
Statistical Working Group: