Vincenzo Del Vecchio Banca d’Italia Statistics Collection and Processing Department 2012 ESSnet Workshop – 4 December 2012 The Data Warehouse of Banca d’Italia Guiding Principles and Architecture of an Integrated Statistical Warehouse

2 4 Dec ESSnet Workshop Agenda 1. Guiding principles 2. Architecture of the statistical data warehouse of the Bank of Italy

3 4 Dec ESSnet Workshop Reporting units (Banks & OFI (>4.000), enterpr.& families (>15.000), individuals (> )) Other Institutions (IMF, OECD, ECB, BIS Eurostat, ISTAT, …) Market Providers (Bloomberg, IBCA, Enterprise Register,.…) Internal sources (payment system, accounting system …) Institutional statistics BI users (research, supervision, markets, >2.500 users) Public data (> inquiries/year) Return flows (to > reporting agents) Other Flows (to other Institutions ) 1.1 – Integrated Approach Economic analysis Supervision Central banking Payment system C.C.R. Stat. Domains (> 1 billion observations / year ) DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS

4 4 Dec ESSnet Workshop  Information are shared by many organizational functions and accessible to users who have rights;  Data are collected and processed minimizing redundancies  The integrated use of data from different sources and the reuse of data for many purposes are fostered through:  Organizational Measures (statistical committee, specialized units for warehouse administration, …)  Metodological and Technical Measures (reference information model, common data dictionary …)  Harmonization of Concepts, Code Lists, Data Contents (concepts and data administration) 1.1 – Integrated Approach

5 4 Dec ESSnet Workshop Model of meta-models L4 L3 Model of data definitions (“meta-model”: how to make definitions) Object group: property L1 Data obser. Green cars: 40% Real world L0 (Information Model: Matrix) Cars by colour: percentage L2 Data definitions “dictionary” 1.2 – Information completness Stats Definer Stats Producer Stats User

6 4 Dec Active Definitions (model driven sw) DATA Software Services Administrator Information Model DEFINITIONS User System Automation Time to market Accurate and up-to- date

7 4 Dec 2012 User User oriented model & languages Administrator : Subject Matter Expert Information Model DEFINITIONS Based on Mathematics & Statistics Independent of the IT implementation and the IT people

8 4 Dec ESSnet Workshop Statistical Domain Monetary & financial Balance of Payments … Data type Quantitative / qualitative Periodical / not Multidimensional Time series Registers Questionnaires … Definition Extraction, transformation, transmission Collection and storage CompilationDisseminationUse Unique model and approach (integration of methods and techniques) DATA Software Services Administrator Information Model DEFINITIONS

9 4 Dec ESSnet Workshop  Two different histories:  of the real world (e.g. when something is true or false)  of the information system (e.g. when something is known or unknown)  History of all the I.S. contents:  definitions  data observations 1.6 – Historical representation

10 4 Dec ESSnet Workshop Agenda 1. Guiding principles 2. Architecture of the statistical data warehouse of the Bank of Italy

11 4 Dec ESSnet Workshop A unique information model - the Matrix model – (designed and maintained by the Bank of Italy) – descibing concepts, data structures and algorithms for validation and calculation; A unique data dictionary – a data base structured according to the Matrix model – storing the users ’ definitions; A logically unique warehouse storing the data observations; A common software platform – Infostat - made of reusable services driven by the users ’ definitions stored in the data dictionary The current reference architecture

12 4 Dec ESSnet Workshop SOFTWARE SERVICES Define Send Receive Calculate & Check Release Monitor Inquiry Import/export … W3C, WS-I BPEL GSBPM PROCESSES Receive Check Remarks Release Calculate Microdata Macrodata Indicators Send User application architecture DATA WH. Data Definitions Data Observations Calc. Algorithms MATRIX (SDMX, XBRL, CSV …) Application User Applications DATA DEFINITIONS

13 4 Dec ESSnet Workshop Warehouse Administration Common software services DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS Statistical community / Information system “A” Information segments (surveys, production lines …) Statistical community / Information system “B”

14 4 Dec ESSnet Workshop IMF, OECD, ECB, … D1D1 D2D2 D3D3 D4D4 D5D5 A1A1 A3A3 A2A2 D10D10 D 12 D 13 D 15 D 17 D 16 A 13 A 12 A14A14 Banks & OFI’s reports D 51 D 52 A 53 A 52 A 51 Economic research models D 54 D 53 A 54 C.C.R. D 21 D 22 D 23 D 24 A 22 A 21 D 60 D 61 Statistical bulletin A 60 A 61 Statistical products D 70 A 71 A 70 A 72 D 71 D 72 D 41 A 42 A 41 D 42 Supervision models page – Graph of Data and Algorithms

15 4 Dec ESSnet Workshop Supporting more Warehouses F.I.U. ESCB (RIAD) Economic analysis Supervision Central banking Payment system Shared Data Specific Data Other Italian Instit. … Statistical Communities B.I. Institut. Functions DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS

16 4 Dec ESSnet Workshop INFOSTAT architecture Calculation engine

17 4 Dec ESSnet Workshop THE ’60s & ’70s first IT solutions first “active” software (metadata driven) THE ’80s launch of the integrated approach the Matrix schema and the first integrated solutions THE ’90s integration of many silos applications evolution of the Matrix Model support to GESMES-CB standardization THE 2000s Statistical Dictionary support to SDMX and XBRL standardization THE 2010s INFOSTAT: a service oriented platform 2009: data collection and data quality services : full set of services From 2009 on: migration of old surveys and data bases Brief history of the IT support for Statistics

18 4 Dec ESSnet Workshop Thank you ! Vincenzo