1. Evaluation Seminar Czech Republic CSF and OP Managing Authorities David Hegarty NDP/CSF Evaluation Unit Ireland

2. Programme Structure: 6 January, 2005 5 presentations 1. Context and Organisation of Irish CSF Evaluation Process 2. Ongoing Evaluation 3. Mid-Term Evaluation 4. Final (Update) Evaluation 5. Ex-Ante Evaluation and Lisbon Strategy

3. Session 1 Context and Organisation of Irish CSF Evaluation Process

4. Presentation Structure History – Evolution of system through CSF1 and CSF2 The evaluation system now – NDP/CSF programme structure – Organisation and key features – Role of NDP/CSF Evaluation Unit – Role of other stakeholders The evaluation market: who does the evaluations?

5. History History Objective 1 region 1989 to 1999 2 CSF programmes – EU funds = 1.7% of GDP – Focused on economic convergence and tackling unemployment Evaluation context – Little prior tradition of evaluation – Low evalution capacity baseline

6. Evaluation Drivers Evaluation impetus driven by – compliance considerations – EU Commission pressure and support – political priority attached to EU funds Leading to …. – gradual creation of evaluation structures (CSF1) – major expansion in evaluation capacity and output (CSF2)

7. Milestones in Capacity Development Establishment of 3 internal OP Evaluation Units – ESF (1992), Industry (1993), Agriculture (1995) Outputs – Indicators and monitoring systems – Detailed measure level and thematic evaluations Appointment of 6 OP external evaluators – Advisory and “watch-dog” functions – Mid-term evaluation Steering committees with oversight role

8. Milestones in Capacity Development Establishment of CSF Unit in 1996 – function: co-ordination of OP work and good practice promotion – Outputs: reviews of existing practice and guidelines ESRI CSF mid-term evaluation 1997 – Innovative, tranferable methodology – Recommendations implemented – A success story

9. Taking Stock System reviewed in 1998 Lots of activity but … – Little understanding of evaluation purpose – Inconsistent evaluation focus – Quality of work variable – Poor follow-up/utilisation systems – Relatively expensive and inefficient Lessons informed design of current structures

10. The Evaluation System Now Total public investment in NDP/CSF – NDP € 51 billion – CSF € 5.6 billion Programme Structure – Economic and Social Infrastructure (23.8) – Employment and Human Resources (14.2) – Productive Sector (5.8) – BMW region (3.0) – S&E region (4.1) – Peace (.127)

11. Key Features of Evaluation System Key Features of Evaluation System Evaluation system extended to NDP Centralised system, 1 Evaluation Unit – But OP managing authorities fully involved Extensive interim evaluation effort Follow-up procedures in place

12. Agreed Evaluation Approach Formally adopted in CSF 5 key evaluation criteria – Rationale – Relevance – Effectiveness – Efficiency – Impact Provides anchor for evaluation system – Promotes understanding of evaluation purpose – Underpins work programmes and terms of reference

13. Role of NDP/CSF Unit Whats our job: – to help OP managing authorities with monitoring and evaluation What do we do? – Development of performance indicators – Advice on project appraisal techniques – Drafting terms of reference – Commissioning evaluations – Doing evaluations!

14. Organisational Issues Unit funded under Technical Assistance OP – CSF budget of €3.2 million (evaluation measure) – Report through CSF MA to monitoring committee Monitoring committee decides work programme MC composition – CSF MA (Chair) – OP managing authorities – Commission Unit an advisory member of CSF and OP monitoring committees

15. Who are we? Staff complement – 1 Senior Evaluator (Head of Unit) – 4 evaluators – 2 administrative Evaluator staff recruited through open competition 3 year contracts Requirements – Degree in relevant area (economics, public administration) – 3 years experience (evaluation, research, policy analysis)

16. EU Commission Role Member of Technical Asssistance MC – Role in work programme approval Member of Evaluation Steering Committees (more later) OP and CSF monitoring committee membership An active participant in evaluation process

17. Managing Authority and Implementing Ministry Roles OP managing authorities on Technical Assistance MC – Work programme Also on evaluation steering committees Responsible for submisison of evaluations to monitoring committees – decision paper on evaluation recommendations – follow-up on agreed recommendations Implementing departments on steering and monitoring comittees

18. Who does the evaluations? Small evaluation market in Ireland Economic Consultancy firms – Specialise in OP level work ESRI – Macroeconomic modelling capacity needed for CSF level evaluation Limited involvement of foreign firms Some evaluations done internally