1. Evaluation Seminar Czech Republic CSF and OP Managing Authorities Session 3: Mid-Term Evaluation

2. Presentation Structure Organisation –Role of Planning Group and steering committees –Selection of Evaluators The Evaluation Process –Evaluation Issues –Methods Outcomes and Utilisation Case Study –Evaluation of EU horizontal priorities

3. Organisation Started preparation in early 2002 Planning Group established Composition…. Functions –Agree timetable –Approved core TOR (June 2002) –Approve quality criteria –Acted as CSF steering committee

4. Steering Committees Evaluation Steering Committee established for each OP Composition –OP Managing Authority (MA) –Evaluation Unit, CSF MA and Commission –Implementing departments Functions –Adapt core TOR to OP situation –Selection of evaluators (see next slide) –Quality control function

5. Selection of Evaluators Each RFT advertised in EU Journal Selection criteria –TOR coverage and understanding –Methodology –Evaluation experience –Cost –Resources allocated Interviews held in some cases

6. Evaluation Process Key evaluation questions (see Session 1) were –Relevance –Effectiveness Core analytical tasks specified in TOR –review of external developments –progress to date (end 2002) –Efficiency, project selection, indicators –Performance reserve –Horizontal priorities (see later)

7. Methods Analysis of financial and physical performance monitoring data Review of policy papers Consultations –Managing authorities –Implementing ministries and agencies –Other stakeholders More sophisticated approaches used at CSF level –Macro and labour market modelling

8. Outcomes and Utilisation Process worked reasonably well –Planning Group played useful coordination role –All reports completed on time –Met TOR and of acceptable quality Utilisation –Reports considered by MCs and submitted to Commission –Influenced decisions on financial reallocations and performance reserve –But other considerations (N+2) taken into account

9. Case Study: EU Horizontal Priorities 2 horizontal priorities/principles –Equal opportunity (gender) –Environment Apply to all measures Evaluated in OP reports under several headings –Reporting quality –Indicator coverage –Integration to project selection systems Sample of measures for more in-depth analysis CSF evaluation: synthesis based on OP inputs –

10. EU Horizontal Priorities Conclusions –major problems with integration of principles –principles poorly understood –little evidence of implementation influence –absence of indicators and poor quality reporting Recommendations –More focused approach going forward –Concentrate on small number of relevant measures Outcome –Recommendation accepted –Follow-up work carried out by Evaluation Unit –Future effort to concentrate on identified relevant measures