The Hungarian system of ex post and on-going evaluation focusing on Structural Funds Kinga Kenyeres, Evaluation Division6-7 May, 2010 National Development Agency

The present situation of evaluation – Evaluation Evaluation is a centralized activity in the National Development Agency Separated financial source for evaluation (Implementation OP) Capacity building, cost planning based on The Evaluation Plan, Strong cooperation with Managing Authorities In the focus of the activity: –Project management ToR Legal affairs (public procurement procedures) Project managing of outputs (Inception Report, Semi-final Report, Final Report) Communication about evaluational results –Messages towards policies –Development Policy Academy - Conferences every 3 months. Goal: extending evaluation culture, sharing experiences

The present situation of evaluation – Evaluation Diviso Evaluation is a centralized activity in National Development Agency Separated financial source for evaluation (Implementation OP) Capacity building, cost planning based on The Evaluation Plan, Strong cooperation with Managing Authorities In the focus of the activity: –projectmanagement ToR Legal affairs (public procurement procedures) Project managing of outputs (Inception Report, Semi-final Report, Final Report) Communication about evaluational results –Messages towards policies –Development Policy Academy – Conference every 3 month. Goal: extending evaluation culture, sharing experiences

Mission Evaluation on using of the Cohesion Fund and the Structural Funds Provid e feedback on evaluation results towards Units of execution and planning, Active role in using the EU funds more effectively

Goals 1. The Evaluation activity Provides information for decision-making Provides suitable methods for evaluational topics Perfoms high quality Is cost-effective 2. Transforms the results of the evaluations into messages and forwards them to Stakeholders

Decision-making support Policy planning, level of program coordination –Takes into account the results, impacts of former measures (Ist National Development Plan, ), –Horizontal disciplines –The ex ante evaluation of OP-modification –Support to ActionPlanning Supporting execution –Fine tuning of measures –Practical recommendations in order to improve processes and institutional activities „Service” approach Rising demand for on-going evaluations

Valid Methodology Ex post evaluation –Econometrical methods based on databases and primer data collection – impact assessment –Document analysis, interviews – helps to understand the results of the impact assessment, takes them into a context On-going evaluations –Monitoring database analyses –Process- and organizational analyses Ex ante evaluation –Support to the planning procedure –The goal is impoving the methods of former impact assessment Professional credibility, objectivity

High quality, cost efficiency Evaluation Framework 1.round Selecting evaluators (Autumn 2008 – spring 2009) Public procurement procedure for 4 lots within the framework of a 4-year-contract Technical, financial eligibility on the basis of the strict qualification of experts Comittee for decision including colleagues from Managing Authorities

Evaluation Framework - 4 LOTs 1.Environment (includes: strategic environmental assessment) 2.Infrastructure (transport, energy) 3.Economic and social development (education, health, employment, R+D and innovation) 4.Operative evaluations (includes: system of institution development) High quality, cost efficiency

2. round Procurement of Evaluations through the Evaluation Framework (Spring 2009 – end of 2013) In 4 LOTs: 6 evaluator consortiums compete with each other for one specific evaluation task Call for proposal of evaluators based on well-defined TOR and defined number of expert-days The price is the only criterion for selection. High quality, cost efficiency High Professional Responsibility of Evaluation Unit

Communication of evaluational results 1. Built in the evaluation process Cooperation with Managing Authorities Professional discussions in professional level and in management level –ToR –Evaluation– Project Managing Committee 2. Following the evaluation process Publish evaluational final reports, open opportunity to comment on NDA homepage Conferences/ workshops Inform European Commission and other member states: Executice Summary in English Inform Monitoring Committees The goal is to communicate clear messages

Reaching the goals Evaluation Plan –3-year plan –review every year –Capacity building and cost plan based on Evaluation Plan. –Evaluation Plan is based on the OP execution chapter and the Interactive Executing Handbook of NDA. –Eval Plan has been accepted by the Implementation OP Monitoring Committee –Inputs Legal requirements Information demand of the organization Requirements of other institutional stakeholders OP Monitoring Committe opinion European Commission opinion Currently: requirements of the forthcoming Hungarian EU presidency – shift from ex post evaluations to on-going ones

Rising rate of on-going evaluations, because –Rising interest in facts of implementation and coordination –The findings of ex post evaluations cannot be used easily during implementation Pros: –Increasing overall demand for evaluation –Stakeholders are more and more concerned in having proper quality evaluations Cons: –The very sophisticated methodology of ex post evaluations cannot be used –We have more experience in ex post evaluations Evaluations of the Division The balance between ex post and on going evaluations is a MUST!

Evaluations in the Evaluation Division The Division launched 11 evaluations in 2009 (9 ex posts and 2 on-goings) and 12 in 2010 (8 on-goings and 4 ex posts)

Evaluations in the Evaluation Division

Our on-going evaluations: Evaluation on the financing system of the Intermediate Bodies of the NSRF - finished Evaluation of the action planning process Evaluation of the selection procedures of the NSRF OP mid-term reviews –Strategy-focused mid-term evaluation of the implementation –Provides advice to optimize further implementation Evaluations in the Evaluation Division

Thank you for your attention!