1. A Metadata Infrastructure for Bundesbank-Microdata
Status quo, Plan, Implementation
Dr Christopher-Johannes Schild, Deutsche Bundesbank Research Data and Service Centre

2. How did we previously handle metadata? Intermediate solution
Questions
How did we previously handle metadata?
Intermediate solution
What kind of infrastructure do we need?
How do we get there?
CBS Harry Goossens helped with various aspects
Metadata Team of 6
Dr Christopher-Johannes Schild
31. Dezember 2018

3. Central Office of the German Central Bank – Organisation
President
Member of the Governing Council of the ECB
Dr Weidmann
Deputy President
President’s Alternate in the Governing Council of the ECB
Prof Buch
Area III
Area IV
Area V
Area VI
Thiele
Dr Dombret
Dr Nagel
Dr Beermann K R Vo F Rv S H ÖB Z B C IT M P Vb
Communic-ations
Legal
Econ-omics
Financial Stability
Audit
Statistics
Cash
Economic Education, University Technical
Central Bank Cooperation
Payments and Settle-ment Systems
Banking and Financial Super-vision
Controlling Account-ing, Organisation
Inform-ation Techno-logy
Markets
Human Resources
Administration and Premises
K 1
Office for legal issues of banking supervision and financial stability
Office for Cross-Sectional Tasks and Speeches
Office for Intra-al
Tasks
Office for Policy and Procedural Issues, International Auditing Tasks
S 1
H 1
ÖB 1
Z 1
Head of Strategic Office and SSM Secretariat
Rc 1
C 1
IT 1
M 1
Office for Human Resources Policy Issues
Vb 1
Internal Communication
Monetary and Financial Statistics, External
Stock Statistics
Banknote Development and Procurement, Logistics, Security, Gold Storage
Education and Training
Strategy, Policy and Oversight; Payments, Securities Settlement
Risk Control
Corporate Controlling and Strategic Planning
IT Client Liaison Management, Product Management I
Strategic Issues relating to Monetary Policy Operations
Infrastructure
K 2
R 1
Vo 1
F 1
Rv 1
S 2
H 2
ÖB 2
Z 2
Strategic Office SSM-Supervision/ JST-Coordinaton
Coordination of Representative Offices
C 2
IT 2
M 2
P 1
Vb 2
External Communication
National Law
Monetary Policy and Analysis
Macroprudential Analyses
Audits of Cash related Operations,
Balance of Payments Statistics, International Investment Position
Branch Operations, IT Applications and Automation
Economic Education, Information Management, Money Museum
Payment and Account Management Operations
Accounting
IT Operations and Services
Market Analysis and Portfolios
Human Resources Management
Quality, Resource and Change Management; Housing Management
Representative Office New York, USA
K 3
R 2
Vo 2
F 2
Rv 2
S 3
H 3
TZK
Z 3
B 1
C 3
IT 4
M 3
P 2
Vb 3
Language Services
EU and International Law
Public Finances
Macroprudential Surveillance
Audits of Banking Ope- rations (excluding Cash Operations)
General Economic Statistics
Cash Payment and Cash Strategy
TARGET2 & TARGET2
Securities Service Management
Banking Supervision Law and International Banking Supervision
Organisation
Applications Development
Market Operations
Internal Staff Services, Compensation and Pensions
Centralised Construction Management
Centre for Technical Central Bank Cooperation
Representative Office Tokyo, Japan
President’s Office
R 3
Vo 3
F 3
S 4
HDB
Z 4
B 2
Anti-Bureaucracy Officer
IT 5
M 4
Procurement Centre
Financial Law
Macroeconomic Analysis and Projections
International Monetary Affairs
Statistical Information Management, Mathematical Methods
University of Applied Sciences,
Hachenburg
IT Projects Payments and Settlement Systems
Risk Analysis
IT Infrastructure
Market Operations Systems and Project Management
Secretariat for European Affairs
Data Protection Officer
Vo 4
F 4
S 0
Z 5
B 3
IT 6
Balance of Payments, Exchange Rates and Capital Markets Analysis
Macroprudential Policy
Research Data Centre
Securities Settlement and Collateral Management
On-site Inspections, Implementation of International Standards
IT Project and Test Services, ESCB Activities
Compliance
Confidentiality Protection Officer
Deputy President’s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Research Centre
Arbitration Body pursuant to section 14 Prohibitory Injunctions Act
Vo 5
B 4
International and Euro-Area
Accounting and Supervisory Databases
Macroeconomic Analysis
B 5
Coordination of National and Inter- national Off-Site Supervision
August 2015

4. Status quo: Metadata for Macro Data
Publishing Layer
GSBPM Phase 8 Research
JournalArticles
more reports
Internal Analyst / Researcher
Metadata
System
Macro Data
Micro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
ZIS-DWH
Validated macro data with SDMX Metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Primary statistics,
Bundesbank-Tasks
Aggregation
Reports
FDI-Statistics ..
Bank Supervision
Bank Statistics
...
Integration Layer
GSBPM Phase 5 Process
Micro Data
AWMUS, DIREK … … …
... …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata sources:
Administrative data, Survey data, External data

5. Separate Microdata-Lifecycles
Publishing Layer
GSBPM Phase 8 Research
JournalArticles
more reports
Internal Analyst / Researcher
Metadata
System
Macro Data
Micro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
ZIS-DWH
Validated macro data with SDMX Metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Aggregation
“Primary” Publishing Layer
GSBPM Phase 8
Reports
FDI-Statistics
...
Bank Supervision
Bank Statistics
...
Integration Layer
GSBPM Phase 5 Process
Micro Data
AWMUS, DIREK … … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata sources:
Administrative data, Survey data, External data

6. Dispersed, disintegrated Microdata (15 of 177 datasets shown)
President
Member of the Governing Council of the ECB
Dr Weidmann
Deputy President
President’s Alternate in the Governing Council of the ECB
Prof Buch
Area III
Area IV
Area V
Area VI
Thiele
Dr Dombret
Dr Nagel
Dr Beermann K R Vo F Rv S H ÖB Z B C IT M P Vb
Communic-ations
Legal
Econ-omics
Financial Stability
Audit
Statistics
Cash
Economic Education, University Technical
Central Bank Cooperation
Payments and Settle-ment Systems
Banking and Financial Super-vision
Controlling Account-ing, Organisation
Inform-ation Techno-logy
Markets
Human Resources
Administration and Premises
DIREK MiDi
SITS
USTAN
MiMiK
Creditors
MiMik
Debtors
WP-INV
CSDB
PHF
MIR
BISTA
ZENTK
AWMUS
BAKIS
BLS
Corep
EADB
K 1
Office for legal issues of banking supervision and financial stability
Office for Cross-Sectional Tasks and Speeches
Office for Intra-al
Tasks
Office for Policy and Procedural Issues, International Auditing Tasks
S 1
H 1
ÖB 1
Z 1
Head of Strategic Office and SSM Secretariat
Rc 1
C 1
IT 1
M 1
Office for Human Resources Policy Issues
Vb 1
Internal Communication
Monetary and Financial Statistics, External
Stock Statistics
Banknote Development and Procurement, Logistics, Security, Gold Storage
Education and Training
Strategy, Policy and Oversight; Payments, Securities Settlement
Risk Control
Corporate Controlling and Strategic Planning
IT Client Liaison Management, Product Management I
Strategic Issues relating to Monetary Policy Operations
Infrastructure
K 2
R 1
Vo 1
F 1
Rv 1
S 2
H 2
ÖB 2
Z 2
Strategic Office SSM-Supervision/ JST-Coordinaton
Coordination of Representative Offices
C 2
IT 2
M 2
P 1
Vb 2
External Communication
National Law
Monetary Policy and Analysis
Macroprudential Analyses
Audits of Cash related Operations,
Balance of Payments Statistics, International Investment Position
Branch Operations, IT Applications and Automation
Economic Education, Information Management, Money Museum
Payment and Account Management Operations
Accounting
IT Operations and Services
Market Analysis and Portfolios
Human Resources Management
Quality, Resource and Change Management; Housing Management
JALYS
Representative Office New York, USA
TARGET 2
K 3
R 2
Vo 2
F 2
Rv 2
S 3
H 3
TZK
Z 3
B 1
C 3
IT 4
M 3
P 2
Vb 3
Language Services
EU and International Law
Public Finances
Macroprudential Surveillance
Audits of Banking Ope- rations (excluding Cash Operations)
General Economic Statistics
Cash Payment and Cash Strategy
TARGET2 & TARGET2
Securities Service Management
Banking Supervision Law and International Banking Supervision
Organisation
Applications Development
Market Operations
Internal Staff Services, Compensation and Pensions
Centralised Construction Management
Centre for Technical Central Bank Cooperation
Representative Office Tokyo, Japan
President’s Office
R 3
Vo 3
F 3
S 4
HDB
Z 4
B 2
Anti-Bureaucracy Officer
IT 5
M 4
Procurement Centre
Financial Law
Macroeconomic Analysis and Projections
International Monetary Affairs
Statistical Information Management, Mathematical Methods
University of Applied Sciences,
Hachenburg
IT Projects Payments and Settlement Systems
Risk Analysis
IT Infrastructure
Market Operations Systems and Project Management
Secretariat for European Affairs
Data Protection Officer
Vo 4
F 4
S 0
Z 5
B 3
IT 6
Balance of Payments, Exchange Rates and Capital Markets Analysis
Macroprudential Policy
Research Data Centre
Securities Settlement and Collateral Management
On-site Inspections, Implementation of International Standards
IT Project and Test Services, ESCB Activities
Compliance
Confidentiality Protection Officer
Deputy President’s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Research Centre
Arbitration Body pursuant to section 14 Prohibitory Injunctions Act
Vo 5
B 4
International and Euro-Area
Accounting and Supervisory Databases
Macroeconomic Analysis
B 5
Coordination of National and Inter- national Off-Site Supervision
August 2015

7. Increasing Demand for (integrated) Microdata
President
Member of the Governing Council of the ECB
Dr Weidmann
Deputy President
President’s Alternate in the Governing Council of the ECB
Prof Buch
Area III
Area IV
Area V
Area VI
Thiele
Dr Dombret
Dr Nagel
Dr Beermann K R Vo F Rv S H ÖB Z B C IT M P Vb
Communic-ations
Legal
Econ-omics
Financial Stability
Audit
Statistics
Cash
Economic Education, University Technical
Central Bank Cooperation
Payments and Settle-ment Systems
Banking and Financial Super-vision
Controlling Account-ing, Organisation
Inform-ation Techno-logy
Markets
Human Resources
Administration and Premises
DIREK MiDi
SITS
USTAN
MiMiK
Creditors
MiMik
Debtors
WP-INV
CSDB
PHF
MIR
BISTA
ZENTK
AWMUS
BAKIS
BLS
Corep
EADB
K 1
Office for legal issues of banking supervision and financial stability
Office for Cross-Sectional Tasks and Speeches
Office for Intra-al
Tasks
Office for Policy and Procedural Issues, International Auditing Tasks
S 1
H 1
ÖB 1
Z 1
Head of Strategic Office and SSM Secretariat
Rc 1
C 1
IT 1
M 1
Office for Human Resources Policy Issues
Vb 1
Internal Communication
Monetary and Financial Statistics, External
Stock Statistics
Banknote Development and Procurement, Logistics, Security, Gold Storage
Education and Training
Strategy, Policy and Oversight; Payments, Securities Settlement
Risk Control
Corporate Controlling and Strategic Planning
IT Client Liaison Management, Product Management I
Strategic Issues relating to Monetary Policy Operations
Infrastructure
K 2
R 1
Vo 1
F 1
Rv 1
S 2
H 2
ÖB 2
Z 2
Strategic Office SSM-Supervision/ JST-Coordinaton
Coordination of Representative Offices
C 2
IT 2
M 2
P 1
Vb 2
External Communication
National Law
Monetary Policy and Analysis
Macroprudential Analyses
Audits of Cash related Operations,
Balance of Payments Statistics, International Investment Position
Branch Operations, IT Applications and Automation
Economic Education, Information Management, Money Museum
Payment and Account Management Operations
Accounting
IT Operations and Services
Market Analysis and Portfolios
Human Resources Management
Quality, Resource and Change Management; Housing Management
JALYS
Representative Office New York, USA
TARGET 2
K 3
R 2
Vo 2
F 2
Rv 2
S 3
H 3
TZK
Z 3
B 1
C 3
IT 4
M 3
P 2
Vb 3
Language Services
EU and International Law
Public Finances
Macroprudential Surveillance
Audits of Banking Ope- rations (excluding Cash Operations)
General Economic Statistics
Cash Payment and Cash Strategy
TARGET2 & TARGET2
Securities Service Management
Banking Supervision Law and International Banking Supervision
Organisation
Applications Development
Market Operations
Internal Staff Services, Compensation and Pensions
Centralised Construction Management
Centre for Technical Central Bank Cooperation
Representative Office Tokyo, Japan
President’s Office
R 3
Vo 3
F 3
S 4
HDB
Z 4
B 2
Anti-Bureaucracy Officer
IT 5
M 4
Procurement Centre
Financial Law
Macroeconomic Analysis and Projections
International Monetary Affairs
Statistical Information Management, Mathematical Methods
University of Applied Sciences,
Hachenburg
IT Projects Payments and Settlement Systems
Risk Analysis
IT Infrastructure
Market Operations Systems and Project Management
Secretariat for European Affairs
Data Protection Officer
Vo 4
F 4
S 0
Z 5
B 3
IT 6
Balance of Payments, Exchange Rates and Capital Markets Analysis
Macroprudential Policy
Research Data Centre
Securities Settlement and Collateral Management
On-site Inspections, Implementation of International Standards
IT Project and Test Services, ESCB Activities
Compliance
Confidentiality Protection Officer
Deputy President’s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Board Member‘s Office
Research Centre
Arbitration Body pursuant to section 14 Prohibitory Injunctions Act
Vo 5
B 4
International and Euro-Area
Accounting and Supervisory Databases
Macroeconomic Analysis
B 5
Coordination of National and Inter- national Off-Site Supervision
August 2015

8. Metadata for Microdata – the old system
dispersed
hard to find
unstructured
not harmonized
not digitized
not written down
„confidential“
„not available“
rarely shared
Dr Christopher-Johannes Schild
31. Dezember 2018

9. Metadata for Microdata – the old system
Journal
Articles
Publishing Layer
GSBPM Phase 8 Research
Metadata
System
Macro Data
Micro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
ZIS-DWH
Validated macro data with SDMX Metadata
Little data sharing
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Exclusively interpreted microdata
Aggregation
Reports
small
Research
teams
FDI-Statistics
...
Bank Supervision
Bank Statistics
...
Integration Layer
GSBPM Phase 5 Process
DIREK … … … …
exclusive raw
microdata and metadata
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

10. Metadata for Microdata – the old system
Journal
Articles
Publishing Layer
GSBPM Phase 8 Research
Metadata
System
Macro Data
Micro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
ZIS-DWH
Validated macro data with SDMX Metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Exclusively interpreted microdata
Aggregation
Reports
FDI-Statistics
...
Bank Supervision
Bank Statistics
...
Integration Layer
GSBPM Phase 5 Process
DIREK … … … …
exclusive raw
microdata and metadata
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

11. Intermediate Solution (status quo)
External Analyst / Researcher
Intermediate Solution (status quo)
Journal
Articles
Publishing Layer
GSBPM Phase 8 Research
Internal Analyst / Researcher
Metadata
System
Macro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
Micro data
flatfiles
ZIS-DWH
Validated macro data with SDMX Metadata
Micro data
flatfiles
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Aggregation
Reports
FDI-Statistics
...
Bank Supervision
Bank Statistics
...
Research Data and Service Center
Integration Layer
GSBPM Phase 5 Process
DIREK … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

12. Intermediate Solution (status quo)
External Analyst / Researcher
Intermediate Solution (status quo)
Journal
Articles
Publishing Layer
GSBPM Phase 8 Research
Internal Analyst / Researcher
Metadata
System
Macro Data
Data Acces Layer
GSBPM Phase 7 Disseminate
Macro data
cubes
Micro data
flatfiles
ZIS-DWH
Validated macro data with SDMX Metadata
Micro data
flatfiles
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
Aggregation
Reports
FDI-Statistics
...
Bank Supervision
Bank Statistics
...
Research Data and Service Center
Integration Layer
GSBPM Phase 5 Process
DIREK … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

13. The House of Microdata - Solution
Data Reports (PDF)
External Researcher
The House of Microdata - Solution
Publishing Layer
GSBPM Phase 8 Research
Macro Data
Micro Data
Research Data and
Service Center
Metadata
System
Data Acces Layer
GSBPM Phase 7 Disseminate
Micro data
flatfiles
Internal Analyst / Researcher
ZIS-DWH
Validated macro data with SDMX Metadata
Micro data
flatfiles
“House of Microdata”
ZIS-DWH
Validated micro data
+ SDMX metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
“Clean copies”
Company Ratings
Bank Supervision
Bank Statistics
...
metadata model – metadata entering interface
Integration Layer
GSBPM Phase 5 Process
JALYS … …
DIREK … … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

14. The House of Microdata - Solution
Data Reports (PDF)
External Researcher
The House of Microdata - Solution
Publishing Layer
GSBPM Phase 8 Research
Macro Data
Micro Data
Research Data and
Service Center
Metadata
System
Data Acces Layer
GSBPM Phase 7 Disseminate
Micro data
flatfiles
Internal Analyst / Researcher
ZIS-DWH
Validated macro data with SDMX Metadata
Micro data
flatfiles
“House of Microdata”
ZIS-DWH
Validated micro data
+ SDMX metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
“Clean copies”
Company Ratings
Bank Supervision
Bank Statistics
...
metadata model – metadata entering interface
Integration Layer
GSBPM Phase 5 Process
JALYS … …
DIREK … … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

15. The House of Microdata - Solution
Data Reports (PDF)
External Researcher
The House of Microdata - Solution
Publishing Layer
GSBPM Phase 8 Research
Macro Data
Micro Data
Research Data and
Service Center
Metadata
System
Data Acces Layer
GSBPM Phase 7 Disseminate
Micro data
flatfiles
Internal Analyst / Researcher
ZIS-DWH
Validated macro data with SDMX Metadata
Micro data
flatfiles
“House of Microdata”
ZIS-DWH
Validated micro data
+ SDMX metadata
Interpretation and
analysis Layer
GSBPM Phase 6 Analysis
“Clean copies”
Company Ratings
Bank Supervision
Bank Statistics
...
metadata model – metadata entering interface
Integration Layer
GSBPM Phase 5 Process
JALYS … …
DIREK … … … … …
Sources Layer
GSBPM Phase 4 Collect
>100 Microdata Bubbles

16. The House of Microdata - Solution
give departments a more active role in sharing
make data easy to
find
interprete
access
However: it is a secondary database
Challenges:
Technical: make it a primary „metadata-hub“ („single point of fact“)
Organizational: incentives to share

17. „Reasons“ not to share data / metadata
Effort to write down „tacit knowledge“
Effort to structure unstructured metadata
Legal reasons (data protection + licensing)
Coordination / harmonization costs / effort
Loss of exclusive access
Fear of data accountability
Dr Christopher-Johannes Schild
31. Dezember 2018

18. How to make departments share metadata
Point out advantages for data users
Value data experts‘ knowledge
Assist with metadata structuring effort
Coordinate harmonization
Create a „spirit“ of knowledge sharing
See it as a tool for „free data quality assessment“
Provide easy-to-use metadata interface
Dr Christopher-Johannes Schild
31. Dezember 2018

19. “Single Point of Fact” for metadata Easy entering (no SDMX)
Functional Requirements / Specification for a Metadata-for-Microdata Management System
“Single Point of Fact” for metadata
Easy entering (no SDMX)
Direct entering (by the departments‘ data expert)
Easy and powerful search
Dr Christopher-Johannes Schild
31. Dezember 2018

20. Thanks!
Dr Christopher-Johannes Schild
31. Dezember 2018

21. DDI Data Lifecycle Archiving Collection Processing Integration
Interpretation
Distribute Disseminate
Discovery
Research
Analysis
Publishing
Repurposing
Dr Christopher-Johannes Schild
31. Dezember 2018

22. GSBPM - DDI and SDMX (graphic by Arofan Gregory)
Dr Christopher-Johannes Schild
31. Dezember 2018

23. Possible technical solutions
Enhance ZIS-DB (DB2) to also store DDI-style microdata-metadata
Needs new entering interface that translates „human readable“ metadata into ZIS
Needs new search interface that accesses metadata stored in ZIS and renders results „human readable“ B:
Manage DDI-specific metadata (terms, dataset-level, variable-level) in new separate in DDI-database
Keep management of value-level metadata (codelists) in ZIS / SDMX.
Requires regular copying and translating from ZIS / SDMX into DDI- database of value-level metadata (codelists)
Enter codelist-metadata in SDMX as currently the case
Enter DDI-metadata through a new interface
Search in DDI-metadata through a new interface
Dr Christopher-Johannes Schild
31. Dezember 2018