Targeting the Ultra Poor: An Impact Assessment IIT Kanpur Abhay Agarwal Research Consultant Centre for Micro Finance – IFMR Research December 3 rd, 2012

Outline of Presentation Overview of the study –Objectives, timeline Experimental design Major Findings Conclusions

Key Steps in conducting an RCT  Carefully designing the evaluation study  Randomly assigning treatment and control groups  Collect baseline data  Verify that the assignment looks random  Monitor intervention so that integrity of experiment is not compromised  Collecting follow-up data at regular intervals  Estimating program impacts by comparing mean outcomes between treatment and control groups  Assess whether these are statistically significant or statistically insignificant

Example of a Randomized Controlled Trial Impact Evaluation: Targeting the Hard Core Poor (THP) Project partner: Bandhan Principal Investigators: Dr. Abhijit Banerjee, Dr. Esther Duflo, Dr. Jeremy Shapiro (MIT/JPAL), Dr. Raghabendra Chattopadhyay (IIM-C)

Overview Program Objective (Bandhan) To provide free income generating assets, training, and other assistance to help ultra poor households secure a regular source of income, and to graduate them to potential microfinance clients. Study Objective (CMF) CMF’s study looks to measure the impact of this program in 3 blocks of Murshidabad district in West Bengal using Randomized Controlled Trials (RCTs)

Study Background:Graduation model Graduation model based on “Challenging the Frontiers of Poverty Reduction-Targeting the Ultra Poor” (CFPR-TUP) program pioneered by BRAC Model being replicated and evaluated (orchestrated by CGAP and the Ford Foundation in partnership with local organizations) in 9 locations –Ethiopia, Haiti, Honduras, Pakistan, Peru, Yemen and India in three places (with Bandhan, SKS, and Trickle Up)

Question to answer: Is Bandhan’s THP program resulting in an improvement of socio-economic conditions for these ultra poor households? Principal Investigators: –Dr. Abhijit Banerje (MIT), Dr. Esther Duflo (MIT), Dr. Raghabendra Chattopadhyay (IIM-C), Dr. Jeremy Shapiro (Yale)

Program Design: THP Aim: “graduate” the poorest of the poor to microfinance Treatment households get to choose an income generating asset, receive an allowance for ‘X’ number of weeks depending on the asset Every week, Bandhan THP staff visit the households to discuss progress in these households In approximately 18 months after receiving the asset and successfully retaining it, beneficiaries receive graduation training After successful completion of training, beneficiaries join Bandhan’s microcredit operations (individual loans, joint liability)

Experimental Design: Asset Transfer Asset typeInitialEndline-I Cow9075 Cow + goat1419 Goat Non-farm3233 Sheep11 Pig44 Total293273

Evaluation timeline (2007 – 2011) Listing and selection of households Baseline Midline Endline- I Endline- II

What do we go about evaluating? Bandhan’s perspective Each HH should have at least two sources of income Each member of the HH should be able to have at least two meals per day Must be able to access preliminary treatment in case of minor ailment Will have access to safe drinking water Must be growing two types of vegetables/fruits All children eligible to attend schools must be in school Must be staying in a safe residence

Main focus of endline studies (CMF) Endline I (pre-graduation) Asset base of households Dependency on other households Consumption patterns Food security Time use of adults and children Overall health and sanitation awareness Income stability and vulnerability to shocks Endline II (post loan) Asset base of households Creation of household enterprises/business Consumption patterns Food security Time use of adults and children Overall health and sanitation awareness Income stability and vulnerability to shocks Loans & Savings Social acceptance of graduates

Practical constraints during experimental design Low take-up rate of product (57%) –Why would people want to refuse a free potential income source? –Funders required take-up by 300 beneficiaries Attrition: several communally disturbed hamlets in various blocks of the district People associating CMF field staff with Bandhan staff Assets returned, beneficiaries termed ineligible for loans Control households and non-research households unhappy with their situation

Major findings: Consumption increase in food consumption for treatment group -mean difference of Rs. 64 per person per month (significant at 1% level), representing 15 % of control group mean

Major findings: Time Use Adults in treatment households work more hours per day, on average, than adults in control households

Major Findings: Assets & Income Program increased household asset base of livestock and durable goods –On average, program households have 1.2 more goats, 0.3 more cows, and 0.5 more fruit trees than non-participants –Even without including livestock (which were transferred through the program), this effect persists On average, selected households had irregular income from sales of livestock of Rs. 460 more than control households However, the monthly flow of income (minus the cost of fodder), is unchanged

Additional findings: Food Security and Health indicators  food security  decreased food insecurity for treatment (less likely to skip or reduce meals, especially among adults)  health  increase in health knowledge (hand washing, etc.) among treatment  decreased emotional stress and increased life satisfaction among treatment  little discernable impact on physical health (slow moving)

Additional findings: Non monetary transfers and financial indicators  transfers / crowd out  treatment gives approximately 1 more meal per month (10% of mean) to other households  receive 50% less food gifts than control (Rs. 13 vs Rs. 30 per month)  financial variables  no effect on credit (increased interest in borrowing)  increased formal savings (through Bandhan), not necessarily increased total savings

Additional Findings: Financial Behaviour and Confidence Treatment households appear to borrow less from formal and informal sources, but the finding is not statistically significant. Households score higher on an index of financial autonomy (can women take decisions on assets/operating savings accounts/household spending decisions) Households express higher confidence in their ability to take loans and join savings groups Possibly affected by survey connection with Bandhan

Findings: Food Security Treatment households report lower food insecurity than control households –Adults less likely to skip meals –Households are 7% more likely to report that they eat enough every day Treatment households show higher self-perception of their household situation

Conclusion positive effects 18 months after asset transfer –on consumption –other measures of well being (food security, emotional health) non-agricultural enterprises appear important in income generation –evidence of heterogeneous effects follow up, examining long run effects and graduation to microfinance (ongoing)

In the pipeline Academic Research Paper Endline II fieldwork completed recently. Will assess long term impacts of the program, particularly the graduation of beneficiaries to microfinance, loan repayment patterns, and business growth Comparisons to be made with other ultra-poor programs as well as with other evaluations of the graduation model Great value in performing cost-benefit analysis of the program, particularly for scaling up – Maharashtra govt

Publications and Presentations Research Paper World Bank (CGAP) memos Policy Memos Factsheet College of Agricultural Banking (RBI), Pune Regional Roundtables Cost-benefit Analysis

Livestock holdings: irregular & regular income Housholds assigned to treatment received Rs. 460 more in irregular income for livestock For monthly flow income, treatment and control are not statistically distinguishable

Questions? Comments Contact:

Comparing cows and goats

Disaggregrated food consumption Where is this evidenced increase in food consumption expenditure potentially coming from?

Other impacts: Financial confidence TUP Beneficiaries express more financial confidence and score higher on an index of financial autonomy (can women take decisions on assets/operating savings accounts/household spending decisions)

Other impacts: Food security Food security has improved – a strong measure of welfare Treatment households show higher self-perception of their household situation