Training Seminar for Evaluation Plan Committee Members, Bucharest, 26/2/2007 Introductory Presentation by David Hegarty David Hegarty Phone: +35312986379 EVALCON Mobile: +353879819905 Evaluation and Economic Consultancy Services Email: david.hegarty@evalcon.eu

Presentation Structure Monitoring and relationship with evaluation What is evaluation and why is it done? Evaluation criteria Evaluation cycle Conclusion

What is monitoring? An ongoing, continuous, systematic process Management function Examines or checks programme coverage and delivery Some monitoring questions How much has been spent? What did we get for it? Who benefited? Are we on track? Uses financial and performance indicator data

Monitoring and Evaluation Monitoring and evaluation are linked processes But monitoring is ongoing Evaluation is discrete Evaluators use monitoring information Indicators Monitoring acts as an early warning system Highlights problems or areas that require evaluation

What is evaluation? “Evaluation” is a common, everyday activity People evaluate films, restaurants etc Firms evaluate investments “Evaluation is an elastic word that stretches to cover judgements of many kinds” (Weiss) Public sphere A tool used in management of public policies, programmes and projects Other tools are monitoring and audit

Defining Evaluation Some definitions: “A judgement of interventions according to the results, impacts and needs they aim to satisfy” (EU Commission) “A critical and detached look at the objectives and how they are being met” (UK Treasury) Involves judgement on basis of criteria More comprehensive than monitoring Applies to policies, programmes and projects

Why evaluate? Evaluation of EU programmes is mandatory 3 main overt purposes in public sector context Planning: to assist in the allocation of resources (ex-ante) Implementation: To improve programme management (interim evaluation) Accountability: To address accountability concerns (ex-post) But evaluation sometimes used for other purposes Justify decisions already made Postpone decisions Public relations Compliance “A rational exercise often undertaken for non-rational reasons” (Weiss)

Purpose of Evaluation Overarching learning purpose “To learn through systematic enquiry how to better design, implement and deliver public programmes and policies” (EU Evalsed Guide)

Evaluation Types Summative evaluation Formative evaluation Accountability focus What has been achieved? What's the value of a programme? Formative evaluation Development or learning focus How can we improve performance and delivery of programme? “When the cook tastes the soup that’s formative evaluation. When the guest tastes it, that’s summative evaluation” Both relevant and useful to public sector

Evaluation Criteria What's the basis for evaluation judgements? 5 main criteria Rationale Relevance Effectiveness Efficiency Impact/Sustainability Effectiveness issue is where monitoring and evaluation meet

Impact/Sustainability Evaluation Criteria Needs Problems Issues Impacts Society Economy Environment Results Intervention Programme Objectives Inputs Outputs Efficiency Rationale & Relevance Effectiveness Evaluation Impact/Sustainability

The EU Evaluation Cycle Before (ex-ante evaluation) Completed in Romania for each OP General aim: improve allocation of resources and programme design During (ongoing evaluation) External developments and their implications? Is the programme meeting its objectives? Can we improve programme management? Required under General Regulation Managing authority responsibility Evaluation plan After (ex-post evaluation) What has been achieved? What difference did it make? EU Commission responsibility

Evaluation Cycle and Focus Stage Ex ante Ongoing Ex Post Evaluation Purpose Planning Implementation Accountability Questions:- Rationale *** * Relevance Effectiveness ** Efficiency Impact

Policy, Programme & Evaluation Cycles Delivery Formulation Review Programme Implementation Design Conclusions Ex-ante Evaluation Interim Ex-post Source: The Guide

Summary/Concluding Points Relationship between monitoring and evaluation Purpose of evaluation …. Need to have an evaluation framework Evaluation questions or criteria Focus varies over policy/programme cycle