1. Evaluation Expert Committee 2nd meeting PREPARING for MTE of RDPs: A SURVEY OF THE MEMBER STATES Irina RAMNICEANU Helpdesk Evaluation Manager

2. Evaluation Expert Committee 2nd meeting 2 What are the aims of the survey? OUTLINE OF THE PRESENTATION How was the work organized? The MTE in the ongoing evaluation process Coordination and capacity building Towards organising the MTE... 23 June 2009 Difficulties encountered and Helpdesk support

3. Evaluation Expert Committee 2nd meeting 23 June 20093 sheds light on the design of the ongoing evaluation systems – and on how the MTE fits in WHAT ARE THE AIMS OF THE SURVEY? highlights challenges with the MTE provides an EU-wide reference on evaluation activities underpins the MTE Guidelines developed by the Helpdesk 323 June 2009

4. Evaluation Expert Committee 2nd meeting 423 June 2009 HOW WAS THE WORK ORGANISED? Helpdesk developed the questionnaires GEs and MAs filled out the questionnaires Helpdesk drafted the survey report 62 returns so far!

5. Evaluation Expert Committee 2nd meeting 1. “Minimal outsourcing” … 23 June 20095 THE MTE IN THE ONGOING EVALUATION PROCESS ongoing TBD ex antemid-termex post ongoing … in 14 programmes

6. Evaluation Expert Committee 2nd meeting 2. “In-house project” … 23 June 20096 THE MTE IN THE ONGOING EVALUATION PROCESS … in 2 programmes ongoing ex ante mid-term ex post Activity 1 Activity 2

7. Evaluation Expert Committee 2nd meeting 3. “Sequential outsourcing (simple) ” … 23 June 20097 THE MTE IN THE ONGOING EVALUATION PROCESS … in 4 programmes ongoing ongoing TBD ex antemid-termex post

8. Evaluation Expert Committee 2nd meeting 4. “Sequential outsourcing (multiple) ” … 23 June 20098 THE MTE IN THE ONGOING EVALUATION PROCESS … in 4 programmes ongoing 3 ex antemid-termex post ongoing TBD ongoing 2 ongoing 1

9. Evaluation Expert Committee 2nd meeting 5. “Full-outsourcing (1) ” … 23 June 20099 THE MTE IN THE ONGOING EVALUATION PROCESS … in 7 programmes ongoing ongoing TBD ex antemid-termex post

10. Evaluation Expert Committee 2nd meeting 6. “Full-outsourcing (2) ” … 23 June 200910 THE MTE IN THE ONGOING EVALUATION PROCESS … in 26 programmes ongoing ex antemid-termex post

11. Evaluation Expert Committee 2nd meeting 23 June 200911 COORDINATION AND CAPACITY BUILDING 75% of the MAs are enhancing their evaluation capacities They typically use a mix of instruments

12. Evaluation Expert Committee 2nd meeting 23 June 200912 COORDINATION AND CAPACITY BUILDING 60% of the MAs have an evaluation steering group Its role and composition vary significantly across MS.

13. Evaluation Expert Committee 2nd meeting 23 June 200913 TOWARDS ORGANISING THE MTE... Timely appointment of independent evaluators is critical!

14. Evaluation Expert Committee 2nd meeting 23 June 200914 TOWARDS ORGANISING THE MTE... Progress on reviewing evaluation questions, indicators and intervention logic is limited. Responsibility of the MAs?. Arrangements for evaluators’ access to data are well advanced.

15. Evaluation Expert Committee 2nd meeting 23 June 200915 DIFFICULTIES ENCOUNTERED... 80% of the programmes expect difficulties.. Data collection and methodologies A wide range of problems: data gaps retrieve data from/on (non-)beneficiaries net impacts

16. Evaluation Expert Committee 2nd meeting 15 December 200816 23 June 2009 Data collection and methodologies: Thematic working groups (HNV, assessment of impacts) Guidance on specific topics (e.g. GVA) Other: e.g. good practice, questions from the MS... AND HELPDESK SUPPORT

17. Evaluation Expert Committee 2nd meeting 23 June 200917 DIFFICULTIES ENCOUNTERED... 56% of the programmes need(ed) support on the TORs.. Administrative and other matters Other matters: clarifications on indicator fiches explanatory notes on the evaluation questions glossary of terms.

18. Evaluation Expert Committee 2nd meeting 15 December 200818 23 June 2009 Administrative and other matters: MTE Guidelines for the MS Dedicated good practice examples Dedicated FAQ section on the website Glossary of key evaluation terms... AND HELPDESK SUPPORT