Professor Prosper Ngowi; pngowi2002@yahoo.com +255 754 653 740       The What, When, Why and How Questions in Evaluating Interventions: Selected Cases Professor Prosper Ngowi; pngowi2002@yahoo.com +255 754 653 740 1

Outline Evaluation of various interventions in forms of projects and programmes Selected key evaluation issues What is normally being evaluated When should evaluations be done Why evaluations are done Who should do evaluation How can evaluations be done Source of information: Theoretical and practical hands-on involvement in over 20 evaluation assignments

Introduction & context Evaluation as part of project cycle management Selected evaluation assignments/cases About 20 evaluation and related assignments Actual evaluation of projects Mid-term and end of project evaluations Public, private as well as civil society organizations (CSOs) sectors projects

Selected Evaluation Projects End-of-project evaluation of Bukombe Community Economic Strengthening Initiative (Heifer 2017) Evaluation of the Impacts of BTC primary health project in Karagwe – May – June 2009 Evaluation of the Impacts of the Regional Facilitating Agencies (RFAs) for TACAIDS in Mwanza and Kagera Evaluation of the Impacts of DANIDA agricultural projects in Mbeya and Iringa – February 2004 Evaluation of the Impacts of MKURABITA Programme for the governments of Norway and Tanzania, Sept 2005

Cases … Evaluation of the Impacts of UNDP Volunteers Programme on HIV/AIDS in Kagera, Kigoma, Rukwa and Lindi: 2007 Evaluation of the Impacts of Medicos Del Mundo Primary Health Project in Karatu and Ngorongoro – 2008 Evaluation of KOICA Projects in Tanzania – September – October 2008 Final Evaluation of Beekeeping Project in Rufiji, Kigoma and Kibondo for BTC (May – July 2010) Final Evaluation of Kilombero Valley Ramsar Site Integrated Management Plan for BTC (December 2010 to January 2011) Evaluation of the MCA-T – January to February 2013 Evaluation of Water Sector Development Program (WSDP), 20113 n

Cases … Appraisal of DANIDA’s private sector development programme (February – March 2013) Review of ILO Youth Employment Facility (YEF) in Tanzania, Kenya and Uganda (September 2013) Evaluation of Binti Simama Imara Project for Social Action Trust Fund (SATF), Nov to Dec 2014 Evaluation of CSOs Capacity Building/Support Programme for WWF; Dec 2014 to January 2015. Evaluation of CARE International Pastoralist Programme: March – May 2016 Evaluation of YEE) Project for Plan International (May 2017)

What was evaluated: The OECD evaluation criteria Relevance: The extent to which intervention is suited to the priorities and policies of the target group, recipient and donor Key relevance questions: To what extent are the objectives of intervention still valid? Are the activities and outputs of the intervention consistent with the overall goal and the attainment of its objectives? Are the activities and outputs of the intervention consistent with the intended impacts and effects?

What is evaluated … Effectiveness A measure of the extent to which an intervention attains its objectives Key effectiveness questions: To what extent were the objectives achieved / are likely to be achieved? What were the major factors influencing the achievement or non-achievement of the objectives?

What is evaluated … Efficiency Measures the outputs in relation to inputs Assesses whether intervention uses the least costly resources possible in order to achieve the desired results Compares alternative approaches to achieving the same output; Is about adopting the most efficient process Key efficiency questions: Were activities cost-efficient?; Objectives achieved on time? Was the intervention implemented in the most efficient way compared to alternatives?

What is evaluated … Impact: Is about: Positive and negative changes produced by an intervention, directly or indirectly, intended or unintended Impacts and effects resulting from the intervention (causality, contribution, attribution) Key impact questions: What has happened as a result of the intervention? What real difference has the activity made to the beneficiaries? How many people have been affected?

What is evaluated … Sustainability Is concerned with measuring whether benefits of an activity are likely to continue after project exit Environmental and financial sustainable Key sustainability questions: To what extent did the benefits of intervention continue after project exit What were the major factors which influenced the achievement or non-achievement of sustainability of the intervention?

Why evaluate? Tracking record: Improving remaining part of project (mid- term) Informing similar projects and projects in similar environment Learning

When to evaluate? Mid-term At project’s half life cycle End/final/terminal Just before project closure

Who can evaluate? Internal evaluators Can be biased, partial, subjective External evaluators Unbiased, impartial, new look/perspectives Both internal and external evaluators: Joint/participatory Last as best

How to evaluate: Techniques Participatory Secondary data: Project document, baseline, various reports Primary data Field visit; Interviews; FGDs; Observations Photographic: Sounds, voices and pictures from the field: Heifer DVD example

The End Thank