1. Training Seminar for Evaluation Plan Committee Members, Bucharest, 26/2/2007
Introductory Presentation by David Hegarty
David Hegarty Phone:
EVALCON Mobile:
Evaluation and Economic Consultancy Services

2. Presentation Structure
Monitoring and relationship with evaluation
What is evaluation and why is it done?
Evaluation criteria
Evaluation cycle
Conclusion

3. What is monitoring? An ongoing, continuous, systematic process
Management function
Examines or checks programme coverage and delivery
Some monitoring questions
How much has been spent?
What did we get for it?
Who benefited?
Are we on track?
Uses financial and performance indicator data

4. Monitoring and Evaluation
Monitoring and evaluation are linked processes
But monitoring is ongoing
Evaluation is discrete
Evaluators use monitoring information
Indicators
Monitoring acts as an early warning system
Highlights problems or areas that require evaluation

5. What is evaluation? “Evaluation” is a common, everyday activity
People evaluate films, restaurants etc
Firms evaluate investments
“Evaluation is an elastic word that stretches to cover judgements of many kinds” (Weiss)
Public sphere
A tool used in management of public policies, programmes and projects
Other tools are monitoring and audit

6. Defining Evaluation Some definitions:
“A judgement of interventions according to the results, impacts and needs they aim to satisfy” (EU Commission)
“A critical and detached look at the objectives and how they are being met” (UK Treasury)
Involves judgement on basis of criteria
More comprehensive than monitoring
Applies to policies, programmes and projects

7. Why evaluate? Evaluation of EU programmes is mandatory
3 main overt purposes in public sector context
Planning: to assist in the allocation of resources (ex-ante)
Implementation: To improve programme management (interim evaluation)
Accountability: To address accountability concerns (ex-post)
But evaluation sometimes used for other purposes
Justify decisions already made
Postpone decisions
Public relations
Compliance
“A rational exercise often undertaken for non-rational reasons” (Weiss)

8. Purpose of Evaluation Overarching learning purpose
“To learn through systematic enquiry how to better design, implement and deliver public programmes and policies” (EU Evalsed Guide)

9. Evaluation Types Summative evaluation Formative evaluation
Accountability focus
What has been achieved?
What's the value of a programme?
Formative evaluation
Development or learning focus
How can we improve performance and delivery of programme?
“When the cook tastes the soup that’s formative evaluation. When the guest tastes it, that’s summative evaluation”
Both relevant and useful to public sector

10. Evaluation Criteria What's the basis for evaluation judgements?
5 main criteria
Rationale
Relevance
Effectiveness
Efficiency
Impact/Sustainability
Effectiveness issue is where monitoring and evaluation meet

11. Impact/Sustainability
Evaluation Criteria
Needs
Problems
Issues
Impacts
Society
Economy
Environment
Results
Intervention
Programme
Objectives
Inputs
Outputs
Efficiency
Rationale
&
Relevance
Effectiveness
Evaluation
Impact/Sustainability

12. The EU Evaluation Cycle
Before (ex-ante evaluation)
Completed in Romania for each OP
General aim: improve allocation of resources and programme design
During (ongoing evaluation)
External developments and their implications?
Is the programme meeting its objectives?
Can we improve programme management?
Required under General Regulation
Managing authority responsibility
Evaluation plan
After (ex-post evaluation)
What has been achieved?
What difference did it make?
EU Commission responsibility

13. Evaluation Cycle and Focus
Stage
Ex ante
Ongoing
Ex Post
Evaluation
Purpose
Planning
Implementation
Accountability
Questions:-
Rationale
*** *
Relevance
Effectiveness **
Efficiency
Impact

14. Policy, Programme & Evaluation Cycles
Delivery
Formulation
Review
Programme
Implementation
Design
Conclusions
Ex-ante
Evaluation
Interim
Ex-post
Source: The Guide

15. Summary/Concluding Points
Relationship between monitoring and evaluation
Purpose of evaluation ….
Need to have an evaluation framework
Evaluation questions or criteria
Focus varies over policy/programme cycle