Vincenzo Del Vecchio Banca d’Italia Statistics Collection and Processing Department 2012 ESSnet Workshop – 4 December.

Slides:



Advertisements
Similar presentations
National Institute of Statistics, Geography and Informatics (INEGI) Implementation of SDMX in Mexico.
Advertisements

1 OECD SDMX Expert Group Paris, September 2012 SDMX implementation in ISTAT: from a tactical approach to a strategic perspective Francesco Rizzo.
ESSnet on SDMX phase II Laura Vignola ISTAT Rome, 3-4 December 2012.
IMF DATA DEVELOPMENTS NEW CONCEPTS, NEW TOOLS Gareth McGuinness International Monetary Fund, Statistics Department ESDS International Conference, November.
Prentice Hall, Database Systems Week 1 Introduction By Zekrullah Popal.
Peter Neudorfer Managing the quality of the ECB’s enhanced ‘Register of Institutions and Affiliates Database‘ (RIAD) Meeting of the Group of Experts on.
CNRIS CNRIS 2.0 Challenges for a new generation of Research Information Systems.
Implementation of GSBPM, DDI and SDMX reference metadata at Statistics Denmark UNECE workshop 5-7 May 2015 Mogens Grosen Nielsen
SDMX – AN OECD PERSPECTIVE Paul Schreyer OECD CCSA Special Session, September 2014 Rome.
Chapter 1: The Database Environment
ESSnet on SDMX phase II Dario Camol
Background Data validation, a critical issue for the E.S.S.
The implementation of the SDMX standards by the ECB and the European System of Central Banks Werner Bier (ECB) Gérard Salou (ECB) Sami Airo (Bank.
WP.5 - DDI-SDMX Integration
Ihr Logo Chapter 5 Business Intelligence: Data Warehousing, Data Acquisition, Data Mining, Business Analytics, and Visualization Turban, Aronson, and Liang.
WP.5 - DDI-SDMX Integration E.S.S. cross-cutting project on Information Models and Standards Marco Pellegrino, Denis Grofils Eurostat METIS Work Session6-8.
Overview of SDMX: Statistical Data and Metadata eXchange Technical and Content Standards for Statistical Data Ann McPhail, Division Chief Statistics Department,
M ETADATA OF NATIONAL STATISTICAL OFFICES B ELARUS, R USSIA AND K AZAKHSTAN Miroslava Brchanova, Moscow, October, 2014.
4 April 2007METIS Work Session1 Metadata Standards and Their Support of Data Management Needs Daniel W. Gillman Bureau of Labor Statistics Paul Johanis.
B A C K G R O U N D B R I E F I N G A N D N E X T S T E P S METIS Geneva, February 2004 Statistical Data and Metadata Exchange Initiative.
5 June 2013 SDMX Technical Working Group Luxembourg 1 5 June 2013 SDMX Technical Working Group Luxembourg 1 WP Item 6 The Expressions Language of Banca.
Eurostat Global Implementation of SDMX in National Accounts Committee for the Coordination of Statistical Activities 22 nd Session 4-6 September Ankara,
Metadata Models in Survey Computing Some Results of MetaNet – WG 2 METIS 2004, Geneva W. Grossmann University of Vienna.
Reform and Modernization of Russian Statistics. New Challenges in Data Collection and Compilation International Seminar on Modernizing Official Statistics:
BAIGORRI Antonio – Eurostat, Unit B1: Quality; Classifications Q2010 EUROPEAN CONFERENCE ON QUALITY IN STATISTICS Terminology relating to the Implementation.
Explaining the statistical data warehouse (S-DWH)
Jenny Linnerud, 27/10/2011, Cologne1 ESSnet CORE Common Reference Environment ESSnet workshop in Cologne 27th and 28th of October 2011.
Some background information about official statistics in the European Union (EU) Martin Eiglsperger European Central Bank – DG Statistics* The 2008 World.
1 1 Developing a framework for standardisation High-Level Seminar on Streamlining Statistical production Zlatibor, Serbia 6-7 July 2011 Rune Gløersen IT.
Eurostat SDMX and Global Standardisation Marco Pellegrino Eurostat, Statistical Office of the European Union Bangkok,
1 COLTRAST Collection and Tracing of Statistical Data CoRD Task Force.
Eurostat 4. SDMX: Main objects for data exchange 1 Raynald Palmieri Eurostat Unit B5: “Central data and metadata services” SDMX Basics course, October.
SDMX IT Tools Introduction
Metadata Working Group Jean HELLER EUROSTAT Directorate A: Statistical Information System Unit A-3: Reference data bases.
Open GSBPM compliant data processing system in Statistics Estonia (VAIS) 2011 MSIS Conference Maia Ennok Head of Data Warehouse Service Data Processing.
1 SDMX Global Conference September 2015 SDMX into the future VTL (Validation and Transformation Language) A new technical standard for enhancing.
STRATEGY FOR DEVELOPMENT OF ISIS AND IT STRATEGY IN THE NSI-BULGARIA Main principles, components, requirements.
1 Statistical business registers as a prerequisite for integrated economic statistics. By Olav Ljones Deputy Director General Statistics Norway
13-14 December 2012 SDMX Technical Working Group Paris WP Item 6 Expressions and Calculations.
CCSA session on SDMX implementation Werner Bier and Per Nymand-Andersen European Central Bank CCSA session on SDMX implementation, FAO headquarters, 11.
1 Dissemination on Internet Experience of the Statistical Office of the Slovak Republic Dissemination on Internet Experience of the Statistical Office.
ESS-net DWH ESSnet on microdata linking and data warehousing in statistical production.
3 June 2013 SDMX Technical Working Group Luxembourg 1 WP Item 6 Expressions and Calculations.
1 High Level Seminar for Eastern Europe, Caucasus and Central Asia Countries (EECCA). Quality in Statistics: Metadata Tbilisi, Georgia, June 2012.
Improvement in quality of State Statistics through the development of Statistical Metadata System Speaker : Principal Deputy of the Chairman of the State.
COMMON STATISTICAL PRODUCTION ARCHITECTURE WHERE ARE WE ? June 2106 By Bruno URBAN.
Wiesbaden Group on Business Registers
Data sharing at Deutsche Bundesbank: The House of Microdata
SDMX for SDG Indicators
Prepared by: Galya STATEVA, Chief expert
Metadata Standards for Statistical Classifications
ESSnet on SDMX phase II Laura Vignola
SDMX Introduction and practical exercises
SDMX Visualisation.
Tomaž Špeh, Rudi Seljak Statistical Office of the Republic of Slovenia
2. An overview of SDMX (What is SDMX? Part I)
Integrated Statistical Information System (ISIS) in Croatia By Maja Ledić Blažević, Senior Advisor, Research & Development Dept. and Branka Cimermanović,
SDMX Tools Overview and architecture
Statistical Information Technology
August Götzfried Eurostat unit B 4
Item of the Agenda Latest developments in the CVD project
SDMX and implications for the ESS
Item 7.11 SDMX Progress report
Streamlining statistical production
SOA initiatives at Istat
Implementing the “Vision” within the ESS
SDMX Implementation The National Accounts use case
SDMX Global Conference , Budapest, September 2019
Daniel Suranyi, Krassimir Ivanov
Presentation transcript:

Vincenzo Del Vecchio Banca d’Italia Statistics Collection and Processing Department 2012 ESSnet Workshop – 4 December 2012 The Data Warehouse of Banca d’Italia Guiding Principles and Architecture of an Integrated Statistical Warehouse

2 4 Dec ESSnet Workshop Agenda 1. Guiding principles 2. Architecture of the statistical data warehouse of the Bank of Italy

3 4 Dec ESSnet Workshop Reporting units (Banks & OFI (>4.000), enterpr.& families (>15.000), individuals (> )) Other Institutions (IMF, OECD, ECB, BIS Eurostat, ISTAT, …) Market Providers (Bloomberg, IBCA, Enterprise Register,.…) Internal sources (payment system, accounting system …) Institutional statistics BI users (research, supervision, markets, >2.500 users) Public data (> inquiries/year) Return flows (to > reporting agents) Other Flows (to other Institutions ) 1.1 – Integrated Approach Economic analysis Supervision Central banking Payment system C.C.R. Stat. Domains (> 1 billion observations / year ) DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS

4 4 Dec ESSnet Workshop  Information are shared by many organizational functions and accessible to users who have rights;  Data are collected and processed minimizing redundancies  The integrated use of data from different sources and the reuse of data for many purposes are fostered through:  Organizational Measures (statistical committee, specialized units for warehouse administration, …)  Metodological and Technical Measures (reference information model, common data dictionary …)  Harmonization of Concepts, Code Lists, Data Contents (concepts and data administration) 1.1 – Integrated Approach

5 4 Dec ESSnet Workshop Model of meta-models L4 L3 Model of data definitions (“meta-model”: how to make definitions) Object group: property L1 Data obser. Green cars: 40% Real world L0 (Information Model: Matrix) Cars by colour: percentage L2 Data definitions “dictionary” 1.2 – Information completness Stats Definer Stats Producer Stats User

6 4 Dec Active Definitions (model driven sw) DATA Software Services Administrator Information Model DEFINITIONS User System Automation Time to market Accurate and up-to- date

7 4 Dec 2012 User User oriented model & languages Administrator : Subject Matter Expert Information Model DEFINITIONS Based on Mathematics & Statistics Independent of the IT implementation and the IT people

8 4 Dec ESSnet Workshop Statistical Domain Monetary & financial Balance of Payments … Data type Quantitative / qualitative Periodical / not Multidimensional Time series Registers Questionnaires … Definition Extraction, transformation, transmission Collection and storage CompilationDisseminationUse Unique model and approach (integration of methods and techniques) DATA Software Services Administrator Information Model DEFINITIONS

9 4 Dec ESSnet Workshop  Two different histories:  of the real world (e.g. when something is true or false)  of the information system (e.g. when something is known or unknown)  History of all the I.S. contents:  definitions  data observations 1.6 – Historical representation

10 4 Dec ESSnet Workshop Agenda 1. Guiding principles 2. Architecture of the statistical data warehouse of the Bank of Italy

11 4 Dec ESSnet Workshop A unique information model - the Matrix model – (designed and maintained by the Bank of Italy) – descibing concepts, data structures and algorithms for validation and calculation; A unique data dictionary – a data base structured according to the Matrix model – storing the users ’ definitions; A logically unique warehouse storing the data observations; A common software platform – Infostat - made of reusable services driven by the users ’ definitions stored in the data dictionary The current reference architecture

12 4 Dec ESSnet Workshop SOFTWARE SERVICES Define Send Receive Calculate & Check Release Monitor Inquiry Import/export … W3C, WS-I BPEL GSBPM PROCESSES Receive Check Remarks Release Calculate Microdata Macrodata Indicators Send User application architecture DATA WH. Data Definitions Data Observations Calc. Algorithms MATRIX (SDMX, XBRL, CSV …) Application User Applications DATA DEFINITIONS

13 4 Dec ESSnet Workshop Warehouse Administration Common software services DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS Statistical community / Information system “A” Information segments (surveys, production lines …) Statistical community / Information system “B”

14 4 Dec ESSnet Workshop IMF, OECD, ECB, … D1D1 D2D2 D3D3 D4D4 D5D5 A1A1 A3A3 A2A2 D10D10 D 12 D 13 D 15 D 17 D 16 A 13 A 12 A14A14 Banks & OFI’s reports D 51 D 52 A 53 A 52 A 51 Economic research models D 54 D 53 A 54 C.C.R. D 21 D 22 D 23 D 24 A 22 A 21 D 60 D 61 Statistical bulletin A 60 A 61 Statistical products D 70 A 71 A 70 A 72 D 71 D 72 D 41 A 42 A 41 D 42 Supervision models page – Graph of Data and Algorithms

15 4 Dec ESSnet Workshop Supporting more Warehouses F.I.U. ESCB (RIAD) Economic analysis Supervision Central banking Payment system Shared Data Specific Data Other Italian Instit. … Statistical Communities B.I. Institut. Functions DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS DATA DEFINITIONS

16 4 Dec ESSnet Workshop INFOSTAT architecture Calculation engine

17 4 Dec ESSnet Workshop THE ’60s & ’70s first IT solutions first “active” software (metadata driven) THE ’80s launch of the integrated approach the Matrix schema and the first integrated solutions THE ’90s integration of many silos applications evolution of the Matrix Model support to GESMES-CB standardization THE 2000s Statistical Dictionary support to SDMX and XBRL standardization THE 2010s INFOSTAT: a service oriented platform 2009: data collection and data quality services : full set of services From 2009 on: migration of old surveys and data bases Brief history of the IT support for Statistics

18 4 Dec ESSnet Workshop Thank you ! Vincenzo