1. Evaluation of SF in Romania
Creating A
Procedural Manual for
Structural Fund Evaluations
Ann Hyde
CSF MA - Twinning Project
Bucharest, Romania May 2006

2. What is the purpose of a guidance manual for evaluation?
Guidance for each evaluation unit.
Source of information.
Guarantee a uniformity of approach.
Step by step guidance about the whole process of evaluation.
Training and capacity building tool.

3. When will an evaluation manual be used?
When deciding
When an evaluation should take place?
What should be evaluated?
Who should be involved in the evaluation?
How the evaluation should be carried out.

4. Main Sections Introduction – Managing evaluations
concepts and definitions
Managing evaluations
Systems, structures
Preparing for evaluations
Procedures, kick off
Contracting evaluations
Procurement and managing the contract
Disseminating the results of evaluations
Debriefing, publishing and distribution

5. Concepts and Definitions
What is evaluation – definition of aims
What evaluation is not.
Types of evaluations relevant for SF?
What can be evaluated?
The key evaluation issues.
The uses of evaluation

6. Managing SF evaluation
Defining the operational structure for evaluation – proposed model
Setting the Evaluation standards.
Identifying (agreeing) the flow of information across the organisational structures.
Establishing an evaluation steering committee.

7. Preparing for an evaluation
Identifying the aim and purpose of the evaluation?
Defining the scope of the evaluation
Defining the key evaluation questions.
Formulating the analytical requirements
Taking stock of available information?
Defining the uses and users of the evaluation

8. Preparing for an evaluation
Describing the programmes to be evaluated
Describing the methodologies to be followed in data collection and analysis
Identify stakeholders
Prepare an indicative evaluation work plan and budget.
Elaborating a project concept for the evaluation.

9. Contracting and Procurement
Preparing the terms of reference
Deciding on type of tender
Preparing advert for tender
Issuing tender/ call for proposals
Evaluating tender
Drafting contract
Signing contract
Scheduling kick off meeting

10. Evaluation Steering Group
Define purpose of the evaluation steering group;
Prepare evaluation steering group timetable.
Arrange steering group meetings.
Define Quality Assurance and compliance checks

11. Kick Off Meeting Define purpose of kick off meeting.
Prepare evaluation stakeholders for the evaluation kick off
Prepare contractor for the kick off
Ensure ToR, evaluation questions and timetable are up to date.
Arrange kick off meeting.

12. Managing the evaluation period
Define flow of information between contractor – evaluation unit – steering group.
Define reporting procedures.
Set up system for commenting on draft evaluation report (s)
Establish QA for report.

13. Evaluation complete Prepare stakeholders for debrief;
Prepare contractor for debrief;
Prepare for publication of evaluation report;
Define parameters for information dissemination;
Arrange full distribution.

14. Evaluation Follow Up Prepare evaluation follow up procedures;
Prepare action plan for implementation of recommendations;
Prepare timetable for follow up;
Establish QA procedures for evaluation procedures.

15. The first evaluation
In the beginning God created the heaven and the earth. And God saw everything that He made. “Behold,” God said, “it is very good.” And the evening and the morning were the sixth day.
And on the seventh day God rested from all His work. His archangel came then unto Him asking, “God, how do you know that what you have created is ‘very good’? What are your criteria? On what data do you base your judgement? Aren’t you a little close to the situation to make a fair and unbiased evaluation?”
God thought about these questions all that day and His rest was greatly disturbed.
On the eight day God said, “Lucifer, go to hell.”