1. Copyright RSO ® SpA, Milano Framework Contract 30-CE-0121765/00-57 (Supply of services for the further development, reinforcement and promotion of benchlearning) Benchlearning Final Conference Measuring eGovernment Impact Giancarlo Senatore RSO Senior Partner Ghent, Nov 30 th, 2009

2. Copyright RSO ® SpA, Milano 2 THE PROJECT THE PILOTS BENCHLEARNING BENCHLEARNING COMMUNITY INDEX

3. Copyright RSO ® SpA, Milano 3 The project Rooted in eGEPs findings – eGovernment Economics Project, Benchlearning is a mean … to test the comparability of impact indicators, to build measurement awareness and capacities, to share good practices. On a voluntary and flexible basis, 12 public agencies covering 10 European countries have freely committed themselves to join 3 Pilot eGovernment Benchlearning exercises on a 2-yearly time span. Through a systematic data gathering, the Agencies will prove whether eGovernment services and applications are finally delivering the expected outcomes.

4. Copyright RSO ® SpA, Milano 4 Expenditure Study eGOV costs monitoring methodology Expenditure estimate for EU25 Total ICT: 36.5 billion (2004) eGOV only: 11.9 billion (2004) Measurement Framework About 90 indicators Implementation methodology Measurement Framework Economic Model eGovernmentProductivityGDP Growth Scenarios show that future eGovernment research and programmes (2005-2010) could boost EU25 GDP by up to 1.54 percent Back to 2005: the eGovernment Economics Project (eGEP)

5. Copyright RSO ® SpA, Milano 5 Net Costs Set-up Provision Maintenance Efficiency Democracy Cashable financial gains Better organisational and IT architectures Better empowered employees Inter-institutional cooperation Openness and participation Transparency and accountability Political Value Financial & organisational Value Constituency Value Effectiveness Reduced admin. burden Increased user value & satisfaction More inclusive public services The eGEP measurement framework

6. Copyright RSO ® SpA, Milano 6 eGEP EU25 Benchmarking Very simple fully comparable indicators EU must agree with Member States indicators Methodology for new benchmarking National level monitoring of eGOV Less simple indicators, some comparability problems A national level unit can Impose indicators top down Build consensus on most comparable ones Micro-level business case & measurement A public agency can: Select any of eGEP indicators First use them for ex ante business cases Then for steady and continuous measurement Use eGEP implementation tools Sophisticated indicators, no comparability problems How could it be used?

7. Copyright RSO ® SpA, Milano 7 Overall score Type International (EU25) Difficulties score Cooperation Comparability Feasibility Policy system benchmark Cooperation Comparability Feasibility Public policy benchmark Cooperation Comparability Feasibility Organisational benchmark 4=High; 3=Medium-high; 2=Medium; 1=Low; 0= null Cooperation Comparability Feasibility Measurement/ Internal benchmark Member State (holistic) Member State (within vertical and/ or region ) Level Individual public agency (voluntary) 444 444 333 333 213 213 102 102 HIGH MEDIUM LOW MEDIUM-HIGH Lessons from eGEP: impact measurement difficulties

8. Copyright RSO ® SpA, Milano 8 A bottom-up collaborative benchmarking based on a peer- to-peer experimental exchange among fairly comparable public agencies from at least two different EU Member States, designed as a symmetric learning process, that (…) will implement and calculate more sophisticated indicators in a chosen area of impact the ICT enabled services the selected agencies provide and in the process will build transformative capacities. 2008-2009: the Benchlearning challenge

9. Copyright RSO ® SpA, Milano 9 Why to benchlearn? To benchmark only some eGEP indicators: The simplest and more comparable. To boost the public sectors impact evaluation capabilities: Focus on most sophisticated impact indicators; Measurement capacities are built bottom-up. To provide the involved agencies with tangible benefits: Opportunity to look at processes complexity; Identify enabling and hindering factors.

10. Copyright RSO ® SpA, Milano 10 Benchlearning is: Voluntary, bottom up and learning oriented; Flexible, with no need of uniform rigid indicators. Gradually scalable from micro to meso and macro: Groups of similar organisations; Groups of similar verticals / regions; Groups of similar countries. Provides insights and learning on the eGOV value chain: Key drivers and success factors; Main barriers; Organisational processes and input. Benchlearning = bottom-up benchmarking

11. Copyright RSO ® SpA, Milano 11 Analysis of eService set-up and delivery processes Elaboration of new impact indicators Peer-to-peer exchange of experiences Transfer knowledge on measurement tools To understand the success factors and barriers behind processes To collaboratively test its feasibility and comparability To build awareness on good practices To enable PAs measure their own performance To extrapolate the promising areas where EU can become a global leader ACTIVITY AIM What benchlearning is aiming to achieve

12. Copyright RSO ® SpA, Milano 12 Ag.1 Ag.8 Ag.13 vs TEST AND LEARNING Ag.1Ag.2Ag.3 Ag.4Ag.5Ag.6 Ag.7Ag.8Ag.9 Ag.11Ag.12Ag.13 BEST IN CLASS Benchlearning vs. benchmarkig

13. Copyright RSO ® SpA, Milano 13 RANKING CAPACITIES AND LESSONS CAPACITIES AND LESSONS 2 1 3 Ag.13 Ag.8 Ag.1 vs Ag.1 Ag.8 Ag.13 Benchlearning vs. benchmarking: outcomes

14. Copyright RSO ® SpA, Milano 14 First year measurement Set-up: Letter of intent from the participating agencies; Running of a kick-off meeting with all participating agency. As is and mainstreaming: Review of existing measurement systems and data; Analysis of organisational strategy and context; Draft report on indicators and preliminary measurement. First full measurement (or zero measurement): Data gathering instruction to agencies; Remote support to agencies to gather the data; Validation of data and calculation of indicators. How benchlearning works (1/2)

15. Copyright RSO ® SpA, Milano 15 Set-up (same as Y1); As is and mainstreaming (same as Y1); Second full measurement (same as Y1). Continuous exchange of information (www.epractice.eu/community/benchlearning);www.epractice.eu/community/benchlearning Inter-agency workshops. Provision to the agencies of a measurement organisational model (processes and roles); Final recommendations and final report. Second year measurement Exchange activities Sustainability actions How benchlearning works (2/2) By end of Dec 2009

16. Copyright RSO ® SpA, Milano 16 Public sector information indicator Efficiency indicator Simplified version of the eGEP Measurement Framework: eGEP 2.0 EFFICIENCY Standard cost model based indicator Work in progress on number of data field indicator ADMINISTRATIVE BURDEN REDUCTION Plurality of subjective and objective metrics Work in progress on a combined index CITIZEN CENTRICITY Pilot 1Pilot 2Pilot 3 Expected project outcomes: eGEP 2.0

17. Copyright RSO ® SpA, Milano 17 Pilot 1 Efficiency gains Agenzia del Territorio (Italian National Cadastre) Oficina Virtual del Cadastro (Cadastre Virtual Office of Spain) Lantmäteriet (National Land Survey of Sweden) Observers: Regional agencies (Emilia-Romagna, Catalonia) About the Pilot Agencies involved: Pilot measurement of efficiency gains of cadastral eServices: Development of a set of indicators to measure the efficiency gains and savings due to the delivery of cadastral eServices; Definition of the indicators in line with the agencies requests to assess the added value of online cadastral information supply in terms of internal and social impact; Data gathering and analysis (volumes, eService costs, organisation and customer satisfaction programs); Shared experiences among agencies.

18. Copyright RSO ® SpA, Milano 18 Pilot 2 Administrative burden About the Pilot Agencies involved: Piloting of indicators in the Administrative Burden Reduction Field Proof-of-concept service: Business registration Data sources: Standard Cost Model Focus groups and interviews Methodology: Specific focus on users (both business and civil servants) Qualitative and quantitative approach to burden measurement Pilot agencies G2B: Belgium: FPS Economy Slovenia: Ministry of Public Administration Greece: Ministry of Public Administration Project Observers: Fedict, Dutch Ministry of Interior, Greek Information Society Observatory

19. Copyright RSO ® SpA, Milano 19 Pilot 3 User centric impact DirectGov (British eGovernment portal) Service-public.fr (French eGovernment portal) Mojauprava (Croatian eGovernment portal) Additional eGovernment portal data collected from Italy and Hungary About the Pilot Pilot measurement of user centric impact of national government portals: Understand what the EC has already done on measuring e-Government and specific indicators developed for user-centric impact and benchmarking national portals Development of a measurement framework that can be used to benchmark the user-centricity of national government portals. The framework measures 5 key aspects: content richness, service sophistication, user choice and control, quality control and design for usability Shared experiences among agencies. Agencies involved:

20. Copyright RSO ® SpA, Milano 20 Micro level only, single public organisations: 3-4 learning organisations managing homogeneous services Groups assembled from similar countries: Leverage existing collaboration networks. Third party facilitators (EU contractors, governments … ): Intense and in-depth work. Start simple Voluntary participation: Participants self-interested in capacity building/learning. Clear mandate and leadership buy-in: Groups to be assembled not by facilitator. Multi-stakeholders but firm governance: Exchange and consensus But with clear lines of accountability. Generate ownership Benchlearning groups: how to manage them

21. Copyright RSO ® SpA, Milano 21 Having a say in the blog session Suggesting and attending new events Recommending documents for the Community library Recommending documents for the Community library Exchanging ideas on eService impact measurement and evaluation Sharing experience and cases Benchlearning community within ePractice.eu

22. Copyright RSO ® SpA, Milano 22 What results will be presented tomorrow What kind of indicators we selected for the measurement of Efficiency Gains, Administrative Burden Reduction and User Centricity; How we managed the comparability issues due to the structural differences of public services in the different countries; How we took into account the organisational process and the eService costs for the service provision in each country; What have been the results of terms of learning; How we quantified the benefits from eGovernment; ICT-enabling of processes leads to significant savings both for businesses and internally within governments. The results obtained in the pilots illustrate the business cases for better and more convenient eGovernment solutions. …