EVALUATIONS in the EU External Aid

What is Evaluation? Evaluation consists in judging the results of an action in order to check their conformity with set objectives. It has two core purposes:  Learning and Accountability. The European Commission uses evaluations both at project/programme level as well as on a more strategic level (sector, country, horizontal).

Core purposes Learning – to improve the effectiveness of EU interventions. By learning from successes and failures, evaluation generates knowledge about what has worked under which conditions, enabling those who develop strategies, design projects and implement actions to improve its effectiveness. Accountability – to programme stakeholders and taxpayers. Evaluation contributes to providing accountability for the use of resources and the results delivered by comparing the performance of the action to its original objectives.

Evaluations Projects and programmes: These evaluations are conducted at the intervention level. They provide an in-depth understanding of project/programme performance and lessons learned for the future. Strategic evaluations assess the results of geographic, sector / thematic policies over a significant period of time. They contribute to accountability by assessing the quality of financial interventions as a whole and provide recommendations and lessons for policy formulation and programming. They are managed by the central Evaluation Unit.

Projects and programmes evaluations Has a programme reached its stated objectives? Where there any unforeseen positive or negative effects? What could be learned from the experience and how can implementing parties do an even better job next time? How can an on-going actions be improved to maximize its results? Evaluations on programme level are used to improve on-going operations and to learn from experience. Good practice but also negative experience can help to design and manage better actions in the future.

Types of evaluations There are three types of evaluations: mid-term, final and ex-post. Mid-term evaluations (performed mid-way during implementation of an Action) are focused on progress to date and -by explaining why progress is happening or is not happening as planned- provide recommendations on how to improve the Action in order to achieve the expected objectives, taking into account problems and opportunities.

Evaluators Independent external experts are hired to conduct the evaluation. Reference Group composed on interested stakeholders and, if applicable, donors active in related programme accompany the evaluation and ensure that the recommendations are taken up.

Evaluators Evaluations are conducted by a team of independent professional evaluators which use classical 5 evaluation criteria in their work: Relevance Efficiency Effectiveness Impact Sustainability Additional criteria could be added

Reference Group Evaluation contracts are managed by the PM with steering of the evaluation's narrative by the Evaluation Reference Group. Reference Group functions include: definition of evaluation questions, discussions and validation of findings, conclusions, lessons and recommendations

Phases of the evaluations Inception phase: initial data collection and definition of methods of analysis; background analysis; reconstruction of intervention logic, kick-off Desk phase: in-depth documents analysis, methodological design of the Field phase Field phase: meeting at country level, data collection and analysis Synthesis phase: final analysis of findings, formulation of conclusions and recommendations Evaluation phases could be merged, descriptions - adapted

Contracting of evaluators FWC SIEA 2018: Framework Contract Services for the Implementation of External Aid 2018 FWC BENEF 2013: expired 31 March 2018

Six thematic lots Sustainable management of Natural Resources and resilience Infrastructure, sustainable growth and jobs Human Rights, democracy and peace Human development and safety net Budget Support Innovative financing for Development

Contract award procedures Request for services to be sent to 4 eligible FWC contractors Deadline for submission of offers is minimum 14 days Evaluation of offers: 14 days Assessment of offers is done on the basis of a best value for money, 80/20 weighting technical quality/price Specific contract signature: 1 week Reporting at the end of implementation