Evaluation What is evaluation? Theoretical, methodological, and practical concerns Focus on development cooperation

What is evaluation? ”…a systematic and objective assessment of an ongoing or completed project, program or policy, its design, implementation or results”

Varieties of use Accountability Learning Instrumental Conceptual Legitimisation Tactical Ritual

Different types of evaluations Process evaluation – focuses on the planning and implementation of an activity as well as outputs and other intermediary results. Impact evaluation – focuses on the effects, outcomes and impacts – brought about with the help of outputs

What is being evaluated? Results chain: Input – activity – output – outcome – impact Inputs lead to activities, which cause outputs, which cause outcomes, which cause the impact.

Inputs are the resources that are needed to carry out the planned activities. For example people, money, goods, materials, infrastructure, and technology. Activities are the things that are being done. For example digging wells, conducting training session, distributing seeds and tools, forming savings groups, and setting up clinics.

Outputs are the immediate results of activities Outputs are the immediate results of activities. For example, the output of digging wells would be the number of functioning wells in a community. Or the output of a training session would be the number of trained individuals. Outcomes are medium-term results caused by outputs. Examples of outcomes might be households with access to clean drinking water or the percentage of trainees who start a business or find a job. The impact is the long-term, broad societal change that the outcomes lead to. For example, reduced poverty or decreased malnutrition might be the impact of a project.

Along which criteria? Relevance Effectiveness Impact Sustainability Efficiency

Relevance ”Doing the right things” Does the intervention match the needs and priorities of its target group, and is it technically adequate?

Effectiveness ”Doing things right” The extent to which the objectives of an intervention have been achieved as a result of the implementation of planned activities. Has the intervention achieved its objectives or will it do so in the future?

Impact The entire range of effects, not only the foreseen. What are the intended and unintended, positive and negative, effects of the intervention on people, institutions and the physical environment?

Sustainability The likelihood that the benefits from an intervention will be maintained at an appropriate level for a reasonably long period of time after the withdrawal of donor support. Will the benefits produced by the intervention be maintained after the cessation of external support?

Efficiency Cost-effectiveness. Optimally efficient if the value of the results is greater than the value of any alternative use of these reources.

Evaluation Design Baseline information is important Indicators for input, activities, output, outcomes, or impact are needed Case studies, surveys, experiments etc. Attribution: To what extent have the identified changes been caused by the intervention rather than by factors outside the intervention?

The practice of evaluation Terms of Reference Inception report Evaluation report Management response

Terms of reference Evaluation purpose: Intervention background: determine evaluation topic and identify the results chain Intervention background: state the context and rationale for the intervention Stakeholder involvement: who is going to use the evaluation?

Evaluation questions and criteria (depend on the type of evaluation) Are we implementing the programme as planned? Are interventions working or making a difference? Recommendations and lessons What will we learn? Methodology How are we evaluating? Design, data sources, data analysis, quantitative, qualitative etc.

Work plan and schedule Reporting Evaluation team Evaluation activities and schedule Budget Reporting Dissemination of results Evaluation team Evaluation team members and their competences

The Inception phase The evaluators’ response to the ToR Interpretation of the evaluation questions Methodology for data collection and analysis Evaluation work plan

Evaluation report Executive summary Introduction The evaluated intervention Findings Evaluative conclusions Lessons learned Recommendations Annexes

Management response An overall assessment from the evaluated organization’s point of view of the relevance, accuracy and usefulness of the evaluation Response to the recommendations Action plan

Exercise With the point of departure in Sida’s Evaluation Manual “Looking Back, Moving Forward”, design an evaluation of one of the development policies described in the document “Government Communication 2004/05:4 – Sweden’s Global Development Policy”. The proposed evaluation design should include purpose, evaluation criteria (i.e. effectiveness, impact, relevance, sustainability, or efficiency), specific evaluation questions, design, methodology, and material. The assignment should be carried out in the groups described below. Each participant should contribute to the assignment and each group should be prepared to orally present their evaluation in front of the class on October 29th (15 minutes per group maximum).