Methodology Working Group Luxemburg April 7, 2016 Centre of Excellence on Data Warehousing Harry Goossens

Background & History ESSnet on Datawarehousing (2010 – 2013) Centre of Excellence on DWH (since 2013)

Centre of Excellence on DWH Started as pilot Ensure sustainability of the work, results and acquired expertise of the preceding S-DWH projects Active dissemination of the deliverables: Support and facilitate projects of member states implementing ESSnet results Overview CoE DWH – MWG, April 7, Luxemburg

Centre of Excellence on DWH Main Goals Identifying relevant projects and support requests Support, consultancy and/or expert reports Ad hoc & on request of member states Maintain the knowledge and expertise repository on the CROS portal Best practice cases Experts network Further elaborating ‘living deliverables’ S-DWH Manual Active dissemination of results, knowledge and new developments. Overview CoE DWH – MWG, April 7, Luxemburg

Short Recap: S-DWH Definition A central ‘statistical data store’ for managing all available data of interest, regardless of its source, enabling the NSI to: produce necessary information (= statistics !) (re)use available data to create new data / new outputs execute analysis and perform reporting A warehouse approach to statistics: Provide an architectural model of the statistical data flow, from data collection to statistical output. Overview CoE DWH – MWG, April 7, Luxemburg

Why S-DWH ? The Challenges Rapidly changing info-demand Decrease costs & admin burden Integration & re-use available data sources Shorter life cycle, Quicker delivery Increase efficiency & flexibility Statistical Production Overview CoE DWH – MWG, April 7, Luxemburg

Make optimal use of all available data sources (existing & new) Why S-DWH ? The goal Make optimal use of all available data sources (existing & new) Reusing Statistical Data External data sources Data Collection Overview CoE DWH – MWG, April 7, Luxemburg

The layered architecture of the S-DWH Distinguishes S-DWH Conceptualisation of how to build up a Statistical DWH Provide optimal organisation of all structured data, enabling re-use, creation of new data etc. 4 Layers, covering all statistical activities: Sources Integration Interpretation & Analysis Data Access / Output Plus the vertical Metadata layer Overview CoE DWH – MWG, April 7, Luxemburg

CoE DWH: Status Developed from a pilot into a stable entity 3rd operational year, started januari 2016 Positive Assessment over first 2 years FPA for sustainable frame of operation 4-year horizon Overview CoE DWH – MWG, April 7, Luxemburg

Results 1st line help desk function established on CROS Portal Inventory of needs & relevant S-DWH projects in MS Inventory of tools and solutions developed for a S-DWH which could be used for exchange and sharing knowledge Handbook further elaborated into S-DWH Manual Knowledge and expertise repository on CROS Portal up-to-date BP-case renewed & revised Integrated glossary Specialists network, pool of experts Overview CoE DWH – MWG, April 7, Luxemburg

Results 2 succesfull Workshops 15+ Consultations and support actions Helsinki 2014: 21 NSIs, ECB / 41 participants Dublin 2015: 23 NSIs, Bundesbank / 45 participants 15+ Consultations and support actions Communication & exchange platform on CROS portal https://ec.europa.eu/eurostat/cros/content/centre-excellence-data- warehousing_en Overview CoE DWH – MWG, April 7, Luxemburg

Support actions CSO Ireland: Destatis Germany: CSO Poland Architecture and Metadata Review Corporate Data Vault Destatis Germany: Metadata support CSO Poland General advise on S-DWH topic Active participator in projectteam Australian Bureau of Statistics (ABS): Layered S-DWH Architecture basis corporate architecture Destatis (Germany) Main focus is on support regarding the development of metadata management. It was agreed that after August, a session will be planned for active support CSO (Ireland) CSO is setting up a DWH solution, the Corporate Data Vault. The CoC received a request from CSO for active support, which resulted in a visit in Cork. In that meeting (16 – 17 April) the  concepts of the layered architecture were explained and several questions were discussed and answered. A follow up is planned later. Central Statistical Office of Poland First contact was established at the ESSnet Workshop in Valencia (22 - 24 January). First, CSO defined their support needs more concrete. Back to back with the MSIS meeting in Dublin (14 - 16 April) a first meeting was organised to answer questions and exchange specific information. It was agreed and to plan a support visit for Q3/Q4 2014. INE Portugal The contact with Portugal are renewed, resulting in participation as partner in the  2nd Grant of  the CoC. Also Portugal  will present their practice case at the workshop. Workshop CoC on DWH , Helsinki, 24 – 25 September2014

Support actions Bundesbank Germany: NISRA Nothern Ireland Review of ‘House of Microdata’ concept Support and advice for developtment metadata model NISRA Nothern Ireland Presenting BP-case in Dublin Ad-hoc advice Statistics Serbia BP-case of metadriven IST system Regional Support meeting planned INE Portugal BP-case SIMSTAT, partner in CoE Destatis (Germany) Main focus is on support regarding the development of metadata management. It was agreed that after August, a session will be planned for active support CSO (Ireland) CSO is setting up a DWH solution, the Corporate Data Vault. The CoC received a request from CSO for active support, which resulted in a visit in Cork. In that meeting (16 – 17 April) the  concepts of the layered architecture were explained and several questions were discussed and answered. A follow up is planned later. Central Statistical Office of Poland First contact was established at the ESSnet Workshop in Valencia (22 - 24 January). First, CSO defined their support needs more concrete. Back to back with the MSIS meeting in Dublin (14 - 16 April) a first meeting was organised to answer questions and exchange specific information. It was agreed and to plan a support visit for Q3/Q4 2014. INE Portugal The contact with Portugal are renewed, resulting in participation as partner in the  2nd Grant of  the CoC. Also Portugal  will present their practice case at the workshop. Workshop CoC on DWH , Helsinki, 24 – 25 September2014

Lessons learned Work in progress, stable but slowly Most NSI still in early stage of (re)design, but growing practice experiences to share Availability of resources Different format & more flexibility needed Need for exchanging expertise and experiences still higly actual Overview CoE DWH – MWG, April 7, Luxemburg

Assessment Overview CoE DWH – MWG, April 7, Luxemburg

Assessment Overview CoE DWH – MWG, April 7, Luxemburg

Future Main recommendations assessment Workplan 2016 Importance of network, knowledge exchange Keep actively contacting NSIs and institutions Keep elaborate BP-cases Support & Consultancy higly effective, continue ! Workplan 2016 Regional support meetings Finalise S-DWH Manual (Memobust-style) Newsletter Promoting CoE DWH (ICES, Modernisation WS) Overview CoE DWH – MWG, April 7, Luxemburg

Harry Goossens hct.goossens@cbs.nl Methodology Working Group Luxemburg April 7, 2016 C0E DWH Thank you ! Harry Goossens hct.goossens@cbs.nl