Screen 1 of 27 Targeting Monitoring and Evaluation of Targeting LEARNING OBJECTIVES Understand the basic measures to monitor and evaluate targeting activities. Use qualitative information to integrate measures of targeting accuracy.

Screen 2 of 27 Targeting Monitoring and Evaluation of Targeting INTRODUCTION Good feedback information can be very important in fine-tuning or correcting the targeting during an ongoing operation. Which types of information and analysis are useful in monitoring and evaluating how well targeting works in practice? Food security information systems can make an important contribution.

Screen 3 of 27 Targeting Monitoring and Evaluation of Targeting INTRODUCTION KEY QUESTIONS: Has the targeting system identified the right people? How many of the target group have been assisted? Has the targeting system succeeded in concentrating resources on the intended beneficiaries? Have any of the resources intended for the target group been diverted elsewhere?

Screen 4 of 27 Targeting Monitoring and Evaluation of Targeting CAUSES OF TARGETING “ERROR” In the real world, no targeting system is perfect. The imperfections are called targeting “errors”. Some potential causes of targeting error: Technical problems Inaccurate needs or vulnerability assessment Gaps in data or information Poor design Flawed implementation Governance problems Malpractice, especially diversion or misuse of resources Weak administrative capacity Poor accountability Poor analysis of power structures and interest groups Clash of values Cultural issues

Screen 5 of 27 Targeting Monitoring and Evaluation of Targeting THREE APPROACHES TO MEASURING ACCURACY 1. How many people were correctly or incorrectly targeted? 2. How much aid reached the right people? 3. Who are the beneficiaries (and the non-beneficiaries)? Key questions:

Screen 6 of 27 Targeting Monitoring and Evaluation of Targeting COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? The accuracy is often measured in terms of how many people are correctly or incorrectly receiving benefits. The diagram shows how the target group and the beneficiaries may not exactly overlap Target population Beneficiaries (population receiving assistance) TOTAL POPULATION CORRECT TARGETING EXCLUSION ERROR INCLUSION ERROR 1. How many people?

Screen 7 of 27 Targeting Monitoring and Evaluation of Targeting Correct targeting is usually measured in terms of coverage. COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Coverage is the percentage of the target group who receive benefits. Coverage and exclusion error are complementary. TARGET GROUP Exclusion Error Coverage Example – Coverage and exclusion error

Screen 8 of 27 Targeting Monitoring and Evaluation of Targeting COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Perfect targeting Community of 100 people. 20 meet the targeting criteria.  Perfect targeting BeneficiariesNon-beneficiaries 

Screen 9 of 27 Targeting Monitoring and Evaluation of Targeting Example – Perfect targeting totalBeneficiariesNon-beneficiaries total Target group 200 Exclusion error = 0/20 = 0 Non- target group 800 Inclusion error = 0/20 = 0 % Coverage = 20/20 = 100% COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE?

Screen 10 of 27 Targeting Monitoring and Evaluation of Targeting COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with adequate resources) Imperfect targeting (with adequate resources) BeneficiariesNon-beneficiaries  TotalBeneficiariesNon-beneficiaries Total Target group 205 Exclusion error = 5/20 = 25% Non-target group 805 Inclusion error = 5/20 = 25% Coverage = 15/20 = 75%

Screen 11 of 27 Targeting Monitoring and Evaluation of Targeting COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with under-resources) Imperfect targeting B (with under-resourcing) BeneficiariesNon-beneficiaries  TotalBeneficiariesNon-beneficiaries Total Target group Exclusion error (or under- coverage) = __% Non-target group 70 1 Inclusion error =__% Coverage =__%

Screen 12 of 27 Targeting Monitoring and Evaluation of Targeting COVERAGE, EXCLUSION AND INCLUSION – HOW MANY PEOPLE? Example – Imperfect targeting (with under-resources) Imperfect targeting B (with under-resourcing) BeneficiariesNon-beneficiaries  TotalBeneficiariesNon-beneficiaries Total Target group / 30 Exclusion error (or under- coverage) = 37 % Non-target group /20 Inclusion error = 5 % /30 Coverage = 63 %

Screen 13 of 27 Targeting Monitoring and Evaluation of Targeting 2. How much aid? Another approach to measuring the success of targeting is by assessing the quantity or proportion of aid actually received by the target group. EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID?

Screen 14 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Beneficiaries  Beneficiaries (20) Target group (15)Non-target group (5) (25% inclusion error)  Target groupNon-target group  Example – Imperfect targeting (with adequate resources)

Screen 15 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Targeting Effectiveness = the percentage of transfer resources actually delivered to the intended target group. Target groupNon-target group  targeting effectiveness = 75% EffectivenessLeakage Effectiveness and leakage are complementary.

Screen 16 of 27 Targeting Monitoring and Evaluation of Targeting IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? The 20 sacks have been distributed as follows: 18 to the correctly targeted households; and only 2 to the non-target group. Example – Imperfect targeting (with adequate resources) - 2 Target groupNon-target group  Effectiveness = ?Leakage = ?

Screen 17 of 27 Targeting Monitoring and Evaluation of Targeting IDENTIFYING TARGET GROUPS – METHODS OF TARGETINGEFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? The 20 sacks have been distributed as follows: 18 to the correctly targeted households; and only 2 to the non-target group. Example – Imperfect targeting (with adequate resources) - 2 Target groupNon-target group  Effectiveness = 18/20 = 90% Leakage = 2/20 (or ) = 10% …of course, the opposite can also happen...

Screen 18 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Example – Imperfect targeting (with adequate resources) - 3 Target groupNon-target group  ?? Missing resources Effectiveness = ?Leakage = ? Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.

Screen 19 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Example – Imperfect targeting (with adequate resources) - 3 Target groupNon-target group  ?? Missing resources Effectiveness = 12/20 = 60% Leakage = = 40% Target group -> 12 sacks in total. Non-target group beneficiaries -> 4 sacks. Four sacks are unaccounted for.

Screen 20 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? TOTAL POPULATION TARGET GROUP Targeting Performance = % of resources received by the target group / % of target group in population. Targeting effectiveness can be compared with the results of an untargeted distribution. The accuracy of the targeting can be measured in terms of how much more the target group receive from the targeted programme.

Screen 21 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? It is worse than no targeting. PERFORMANCE < 1 PERFORMANCE = 1 PERFORMANCE > 1 The targeting has had no effect. Targeting has improved the outcome for the target group.

Screen 22 of 27 Targeting Monitoring and Evaluation of Targeting EFFECTIVENESS, LEAKAGE AND DILUTION – HOW MUCH AID? Dilution = the amount of aid received by each targeted beneficiary, as a percentage of the planned amount. Dilution is the opposite of concentration (the aim of targeting). It is a common targeting problem in which aid is shared among a larger number of people than the programme was designed for.

Screen 23 of 27 Targeting Monitoring and Evaluation of Targeting BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT? 3. Who are the beneficiaries? The third group of targeting indicators falls under the question of who is included and who is excluded by the targeting system. It is very useful to compare the characteristics of the beneficiary and non-beneficiary populations. If there is a clear difference between them it means that the targeting has been effective.

Screen 24 of 27 Targeting Monitoring and Evaluation of Targeting BENEFICIARY & NON-BENEFICIARY PROFILES – WHO’S IN, WHO’S OUT? It is also useful to look at the profiles of people in the exclusion and inclusion errors. Exclusion Inclusion Do the excluded target group members have any factor in common that might suggest systematic marginalization? Who are the people wrongly included among the beneficiaries? Why and how the error is happening?

Screen 25 of 27 Targeting Monitoring and Evaluation of Targeting THE BOTTOM LINE Standard of accuracy and targeting priorities are different in different contexts. Targeting is more accurate in countries with higher incomes, good administrative capacity, and high levels of inequality. In emergency operations, the priority is often to maximize coverage, while in other situations, the priority may be to eliminate inclusion error and leakage.

Screen 26 of 27 Targeting Monitoring and Evaluation of Targeting THE BOTTOM LINE Contextual information and qualitative assessment of how the targeting system is working help to interpret the error measurements in a meaningful way. Why are problems or errors happening? What can be changed in order to reduce them? What do beneficiaries and their community think about the targeting? How are targeting decisions actually made, and by whom? How transparent and accountable is the process? Does the targeting match the programme’s objectives? Could the targeting design have been better?

Screen 27 of 27 Targeting Monitoring and Evaluation of Targeting SUMMARY No targeting system is perfect. Usually some degree of inaccuracy must be accepted to make the programme workable and affordable. A judgement has to be made about which kinds of error, and what degree of error, are acceptable given the programme’s objectives and context. The most useful indicators for measuring the accuracy of targeting fall under 3 key questions: 1. How many people were correctly or incorrectly identified as beneficiaries? Coverage = % of target group receiving benefits Exclusion = % of target group not receiving benefits Inclusion = % of beneficiaries who are not target group members 2. How much of the resources for distribution reached the right people? Effectiveness = % of transfer resources received by the target group Leakage = % of transfer resources not received by the target group Performance = % of transfer resources received by the target group, divided by the % they would have received from an untargeted distribution 3. Who is included, who is excluded?