Draft Methodology for impact analysis of ESS.VIP Projects Pieter Everaers, Director Eurostat Directorate A, Cooperation in the ESS, International cooperation, Resources ESS RDG 13 June 2014

A methodology for impact assessment is needed ESSnet on Standardisation :establish the business case of European standardisation initiatives November 2012: new ESS.VIP projects have to be evaluated on the implementation costs and future benefits =>task force May 2014: ESS Vision 2020 agreed by ESSC. Projects will be selected on criteria including cost/benefit The ESSnet on Standardisation was launched with the aim to establish the business case of European standardisation initiatives at the strategic level November 2012: ESSC decided that new ESS.VIP projects have to be evaluated on the implementation costs and future benefits =>creation of the TF "Impact analysis of ESS.VIP projects" May 2014: ESS Vision 2020 agreed by ESSC. Projects will be selected on the basis of criteria. The first one is cost/benefit ESSnet on Standardisation launched with the aim to establish the business case of European standardisation initiatives at the strategic level ESS RDG 13 June 2014

Two initiatives assessed the possible approaches Sponsorship on Standardisation (end 2012- September 2013) proposed: Common assessment approach Link to the life cycle of a new initiative Structured qualitative approach Assessment by all stakeholders in the ESS Scoring mechanism Basis for the European business case Common assessment approach for all types of initiatives in the ESS Methodology contributes to the decision steps in the life cycle of a new initiative Structured qualitative approach with a fixed set of items Assessment by all stakeholders in the ESS from their own perspective Use a scoring mechanism to analyse, summarise and present the results Use scores and comments as a basis for the European business case ESS RDG 13 June 2014

Two initiatives assessed the possible approaches Task Force "Impact analysis of ESS.VIP projects" (April-December 2013) proposed: Draft methodology and pilot project Quantitative and qualitative assessments Information collected at NSI level but impact for the ESS Statistical and financial experts + senior management In the quantitative phase: “benefits” = potential cost savings SWOT based on proposal from ESSnet + flexible items Draft methodology and pilot project (PRIX) Combination of quantitative and qualitative assessments Information collected at NSI level but the purpose is to assess the impact for the European Statistical System Requires contributions and cooperation from statistical and financial experts and the involvement of NSI’s senior management In the quantitative assessment “benefits” = potential cost savings The qualitative assessment is a SWOT largely based on the proposal from ESSnet on standardisation but with flexible items ESS RDG 13 June 2014

Reconcile the 2 methodologies ESSC asked RDG and ITDG-DIME for a single methodology Proposal Combination of qualitative and quantitative analysis 2 phases of the methodology linked to the life cycle of the initiative ESSC asked RDG and ITDG-DIME for a single methodology Proposal Combination of qualitative and quantitative analysis in 2 phases Decisions around a new initiative follow a life cycle. The 2 phases of the methodology apply to specific decision points in this life cycle ESS RDG 13 June 2014

Phase 1 Business case including SWOT and Vision general criteria Phase 2 including detailed quantitative analysis

Phase 1 Business case includes criteria from ESS 2020 Vision SWOT analysis based on 3 fixed items and 3 flexible items Rating of the SWOT items Leads to decision to launch the project or to go to phase 2 ESS RDG 13 June 2014

Phase 1: Vision general criteria A. Does the proposal have a positive business case at both ESS and national levels, in the sense that benefits outweigh costs? B. Does the proposal contribute to the needs and drivers identified in the ESS Vision 2020 such as efficiency, quality and promoting public trust? C. Can the proposal be clearly positioned according to our ambitions for collaboration set out in the ESS Vision 2020? D. Does the scope in that sense look manageable? E. Is the proposal consistent with the full set of other proposals? F. Does the proposal take into account generic European principles like proportionality and subsidiarity? G. Is the proposal in line with the European Statistics Code of Practice? H. Does the proposal observe existing legislation, both at national and EU level? I. Is the proposal in line with the ESS reference architectural decision and design principles? J. Does the proposal sustain a simultaneous production of European and national statistics? ESS RDG 13 June 2014

Phase 1: List of SWOT items 1. STRENGTHS a. Saving of resources b. Improved process and systems quality c. Enhanced flexibility d. Increased harmonisation e. Increased level of detail f. Better availability of metadata 2. WEAKNESSES a. Costs (of development, transition, support and maintenance) b. Lack of agility/flexibility of the new system c. Heterogeneity of the starting point between countries and partners d. Complexity e. Impact of changed process (risk of errors, discontinuity of the data) f. Lack of appropriate skills in the NSIs ESS RDG 13 June 2014

Phase 1: List of SWOT items 3. OPPORTUNITIES a. Reduced burden on respondents b. Enhance standardisation c. Better communication with users and stakeholders d. Opportunity to change to cope with resource constraints e. Shared technical developments f. Possibility to provide tailored feedback to data providers 4. THREATS a. Lack of consistency with national policies (including legal) b. Lack of support from stakeholders/financers c. Lack of suitable skill profiles d. Threat to data security e. Competition from other providers f. Dependency from external actors/providers ESS RDG 13 June 2014

Phase 1: SWOT rating ESS RDG 13 June 2014

Phase 2 Quantitative analysis Focus only on business processes affected by change Estimation of current costs, development costs and future costs of production Cost categories: Staff , IT solutions, Overhead, Other costs In case potential savings cannot be quantified: build on concrete experiences from countries which have modernized their production system, develop scenarios + Adaptation of the SWOT analysis ESS RDG 13 June 2014

To be kept in mind Lessons from the pilot on quantitative approach Analysis of current costs is possible for most countries Data not always available for different steps of the GSBPM Difficult for some countries to estimate their costs =>group countries Future costs: impact analysis of the modernization process at Member State level not systematic An analysis of the data on current costs is possible and very fruitful for most countries Data not always available for different steps of the GSBPM level (e.g. in countries which have already integrated their production processes) Difficult for some countries to estimate their costs: proposal to group countries as a fall back solution Future costs: we could not rely on impact analysis of the modernization process at Member State level since they are still not systematically done ESS RDG 13 June 2014

Way forward Proposal discussed today and in ITDG-DIME steering group 18 June Feed the work of the high level Task Force working on the ESS Vision 2020 Implementation strategy (for DGINS) Test on projects implementing Vision 2020 (2014-2015) Evaluate and adapt the methodology Draft methodology discussed today and in ITDG-DIME steering group 18 June Output will feed the work of the high level Task Force working on the ESS Vision 2020 Implementation strategy (for DGINS Sept 2014) Test on projects implementing ESS Vision 2020 (2014-2015) Evaluate and adapt the methodology ESS RDG 13 June 2014