Presentation is loading. Please wait.

Presentation is loading. Please wait.

Vision Infrastructure Project (VIP) Enhanced Dissemination Chain

Similar presentations


Presentation on theme: "Vision Infrastructure Project (VIP) Enhanced Dissemination Chain"— Presentation transcript:

1 Vision Infrastructure Project (VIP) Enhanced Dissemination Chain
2nd SISAI meeting, June 2012 B4 – IT for statistical production Unit B5 – Management of statistical data and metadata Unit B6 – Dissemination

2 VIP 'Enhanced Dissemination Chain'
Vision Infrastructure Projects aim to improve the vertical and horizontal integration of the European Statistical System; this ESS integration is one of the central goals of the ESS vision. Commission Communication COM(2009)404 on the production method of EU statistics: a vision for the next decade As new technologies become available, there is a clear drive to maximise their use and to gear statistical methods toward them. New forms of communicating with users and producers such as web 2.0, as well as other new developments in information technology, are likely to result in profound changes in communication channels and data handling and storage. VIPs deal with improvements which are the precondition for achieving more vertical and horizontal integration of the European Statistical System: - horizontal integration: depart from the approach where statistics are handled domain by domain - vertical integration: joint structures / tools / processes + collaborative networks within the ESS The improvements enable more integrated dissemination of European data and indicators. VIPS fall under the 2009 Commission Communication on the production method of EU statistics. The Communication offers a vision for reforming the production method of European statistics, departing from the classical stovepipe model. Particular attention goes to the usage of the internet and web based technology in a broad sense, as these are nowadays the main vehicle for statistical dissemination. Technology allows achieving dissemination targets such as full access to data, to all users at the same time & under same conditions, free of charge.

3 VIP 'Enhanced Dissemination Chain'
Enhanced Dissemination Chain – 2011 to 2013 Short term Eurostat’s Dissemination Chain: improvement, simplification & better integration Disseminated data: quality improvements EU policy indicators: faster production Involvement of ESS (Member States) in the Dissemination Chain Medium term Further development of Eurobase towards a more integrated data warehouse for Eurostat and the ESS Better integration of Eurostat & national dissemination A 2 years project, October 2011 to October 2013 Short term = within project timeline 1. Simplification and better integration, diminish the risk of disruption of a) the daily data updates and b) the online data access, PLUS increase the performance of dissemination 2. Data quality of the European data which is disseminated, in particular through the use of ex-post checks on all published data (Eurobase functionalities) 3. Producing and disseminating policy-oriented indicators includes the development of (a) a new production line for indicator tables, to facilitate flexible combination of data from several domains and (b) new ways of presenting indicators and dashboards on the Eurostat website, would be used for the EU2020 indicators, the Sustainable Development Indicators, ... 4. Share software components & tools within the ESS Medium Term – definition of medium term 1. Data warehouse Overall context: enterprise architecture; possible strategy. 2. Better integration within the dissemination of Member States. This covers further work to facilitate NSIs showing European data and metadata on their websites.

4 Eurostat Dissemination Chain Simplified view of the data flows
Comext Comext Dissemi- nation Internet tools: Data Explorer, TGM, etc. New Query Generator Eurostat production databases Eurobase This is an extremely simplified representation of Eurostat's dissemination chain. In the current context, the Eurostat dissemination chain is a complex patchwork of software components with scattered responsibilities. Dissemination Chain covers: the reference database Eurobase, the reference database Comext (external trade, industrial production statistics) Comext dissemination and the Eurostat website, including its data access and visualisation tools. Eurobase - main reference database containing data for about 5000 non Comext datasets Comext - reference database for external trade and industrial production data ( datasets) Comext dissemination - database containing data from both Eurobase ( datasets) and Comext (180 datasets) New Query Generator (NQG) - the tool used to customise a dataset’s default view Data Explorer - the main application used to access datasets (data browser) TGM – Table, Graphs & Maps 22/03/2012

5 VIP 'Enhanced Dissemination Chain'
Actions Deliverables 1 Development towards a Eurostat / ESS data warehouse Strategy (context: enterprise architecture) Checking algorithms in Eurobase Simplification and integration of the Eurostat Dissemination Chain More functionality for producing and disseminating Eurostat policy indicators 2 3 Service Impact Manager Possible direct connection between Data Explorer & Eurobase New production line for indicator tables Flexible dashboards for policy indicators What will happen in the 2011 to 2013 period? 4 actions Actions with MS vs. pure Eurostat actions Details in the following 4 More integration of Eurostat data and metadata in NSI dissemination Embeddable visualisation tool SDMX-compliant web services

6 VIP 'Enhanced Dissemination Chain'
Action 1: Development of Eurobase towards an Eurostat / ESS data warehouse Task 1a: Strategy on opening Eurobase towards an ESS warehouse Overall context enterprise architecture Internal discussions, early stage Task 1b: Ex-post data quality controls for data published in Eurobase Input into Eurobase is validated by production units But: errors are possible Negative values, unexpected jumps, outliers, obvious inconsistencies … Task 1a: Series of internal discussion in the overall context of enterprise architecture MS / ESS to be kept informed on this process Task 1b: Work to ensure and improve quality within the reference database Eurobase Existing input checks – but there are limits what can be checked at the input Identify obvious errors – 'real' validation is done in production units, this action just to control if something went wrong Related work to improve quality: presentation of flags, harmonisation of table titles, code lists (outside VIP)

7 VIP 'Enhanced Dissemination Chain'
Action 2: Simplification and better integration of the DISS-CHAIN Task 2a: Service Impact Manager (SIM) Task 2a: The SIM application aims at: - The centralization of all the monitoring information of each Dissemination Chain component (retrieving log files, monitoring statistics, backlog entries or audit tables from databases for all different environments), - The processing of the information collected based on defined rules created from the in-depth knowledge of current support team(s), - Acting as a point of reference regarding follow up of 1) Impact assessment on Eurostat business processes, by means of easy to use dashboards, 2) Eurostat dissemination chain related incident alerts, 3) Open vs. resolved Eurostat dissemination chain related incidents, 4) Eurostat dissemination chain related knowledge base. These are the current targets and proposals, and may be subject to updates or scope changes

8 VIP 'Enhanced Dissemination Chain'
Action 2: Simplification and better integration of the DISS-CHAIN Task 2b: Feasibility of the direct connection between Data Explorer and Eurobase Short-term: as-is analysis, and proposals for improvements of the current DISS-CHAIN Long-term: development of a new system Task 2b: This is to be considered as an analysis of the as-is situation of the dissemination chain, along with proposals for evolution. The implementation of proposals may stretch beyond the scope and timeframe of the VIP-EDC. 8 8

9 VIP 'Enhanced Dissemination Chain'
Action 3: Improve functionality for producing and disseminating policy-oriented indicators Task 3a: New production line for indicator tables Faster and more flexible provision of indicators Task 3b: Flexible dashboards for policy indicators Existing dashboards: PEEIs, HICP Specific solutions 'Reference' dashboard One technical solution ready to be used by different indicator sets Feasibility study on-going Task 3a: Quite frequent requests to make new policy indicators available online very quickly. Current production process relatively slow. Use functionalities available in Eurobase: direct preparation of tables using calculation and other functions. Technical issues and business process and governance issues. Task 3b: Eurostat currently has a number of tools to support the visualisation of statistical data, yet for different sets of functionalities and / or based on different technologies. Simultaneously, other institutions – including other Commission DGs – start also to provide dashboards on their websites. The capacity to deliver dashboards for different domains with reduced time for delivery and software maintenance overhead is becoming a key consideration for Eurostat. Dashboards will: - Be available on Eurostat's portal, - Display data in elaborated ways, possibly relying simultaneously on multiple indicators. Also, dashboards will support advanced display features, e.g. for multiple countries comparison, - Support user interactivity for manipulating and presenting the data, Etc.

10 VIP 'Enhanced Dissemination Chain'
Action 4: Enhance Eurostat dissemination towards better integration with MS's dissemination Task 4a: Simple visualisation tool to be embedded in NSIs' websites Widgets Development together with MS (Task Force) Examples (priority widgets) Task 4a: Widgets are Small web applications To be embedded in any website “To be embedded in any website” = in the beginning, only NSI and Commission services, later this could be a much broader audience, and this could also take the form of an iOS app e.g. Showing a relatively small amount of Eurostat data “Showing a relatively small amount of Eurostat data” = this is about presenting key facts In a simple and understandable way Easy to configure a) Which data b) Which display Focus on 18 indicators, to start with.

11 VIP 'Enhanced Dissemination Chain'
Action 4: Enhance Eurostat dissemination towards better integration with MS's dissemination Task Force meeting 29 & 30 March with 6 NSIs agreed on 2 top priority widgets Example of 2 widgets as they would be shown inside a Eurostat Portal page, in a side column. Layout can be adapted to fit NSI's websites. Now work on architecture of the widgets. 11 11

12 VIP 'Enhanced Dissemination Chain'
Action 4: Enhance Eurostat dissemination towards better integration with MS's dissemination Task 4b: Dissemination web services Additional dissemination product Technology to exchange data directly over the internet (machine to machine) Beta-release available Beta release: - The product is available to anyone wanting to use it, as is and with no warranty that the currently supported service operations will be continued in the future. - A number of improvements are needed. The next version will introduce several in-depth changes. Analysis on-going until 2012Q4, implementation and delivery by 2013Q2. 12 12

13 VIP 'Enhanced Dissemination Chain'
Conclusions – expected project results Eurostat: improvements of the DISS-CHAIN Technical Architecture Functionalities Data Quality Presentation ESS: integration with MS's dissemination Widgets Dissemination web services Project October 2011 – October 2013 13 13


Download ppt "Vision Infrastructure Project (VIP) Enhanced Dissemination Chain"

Similar presentations


Ads by Google