November 6, 2003Social Policy Monitoring Network1 Evaluation of the Family Allowance Program (PRAF) * in Honduras: Health and Nutrition Impacts Rafael Flores 1, Saul Morris 2, Pedro Olinto 3, Juan Medina 3 and Oscar Neidecker 3 1 Emory University, 2 London School of Hygiene and Tropical Medicine and 3 International Food Policy Research Institute * PRAF/BID Fase II

November 6, 2003Social Policy Monitoring Network2 Presentation Project description Evaluation design and sample evolution Demand and supply side interventions Impacts on service utilization and coverage of preventive health interventions Impacts on nutrition

November 6, 2003Social Policy Monitoring Network3 Programa de Asignación Familiar PRAF Honduras (Family allowance program) IADB ($45.2 million) GH ($ 5.1 million)

November 6, 2003Social Policy Monitoring Network4 The PRAF/BID Fase II project Goal –Increase the stock of human capital in poor families of Honduras Purposes –Improve the health and nutritional status of children less than three years of age –Increase the performance in math and language of children enrolled in primary school grades 1- 4

November 6, 2003Social Policy Monitoring Network5 The PRAF/BID Fase II project Outputs –Increase the demand for preventive health care among pregnant women, new mothers and children < 3 –Decrease the desertion of school children enrolled 1- 4 –Increase the enrollment of children 6-12 years of age

November 6, 2003Social Policy Monitoring Network6 Questions Do cash transfer programs increase human capital formation? Would government money be better spent on improving service quality? Or are the two approaches necessary complements?

November 6, 2003Social Policy Monitoring Network7 Outcome and impact indicators IndicatorChange % of women attending ante-natal care  five times during their pregnancy + 33% % of children under age 3 taken to health control and weighed in the last month + 12% % of children age months of age with complete vaccination + 10% Height for Age Z-score for children < 5 years old % of children age months of age with anemia - 10%

November 6, 2003Social Policy Monitoring Network8 Demand-side Intervention, Health & Nutrition Thrice-yearly transfer to families with pregnant women and/or children < 3 years of age (max 2) Conditional upon 5 ante-natal care visits during pregnancy; perinatal check-up within 10 d of delivery ; monthly well-child check-ups. Value covers: –Direct costs, $0.00 –Opportunity cost = 12 full days of women’s labor per year, valued at wage rate for most common agricultural activities –$46.30 per year (  US$ 4 per beneficiary per month)

November 6, 2003Social Policy Monitoring Network9 Supply-side Interventions, Health & Nutrition Cash injection for Health Centers, determined in part by size of target population and number of health professionals employed (around $6,000 per year) Implementation of a process of Total Quality Management at each Health Center (expansion of highly successful pilot project in non-project municipalities) Implementation of community-based growth monitoring and counselling intervention (AIN-C)

November 6, 2003Social Policy Monitoring Network10 Demand-side Intervention, Education Thrice-yearly transfer to children aged 6-12 in grades 1- 4 (max 3) Conditional upon not missing more than 20 days of school per year and not repeating a grade more than once Value covers: –Direct costs, $35.22 –Opportunity cost = average contribution to family income of children aged 6-12 in rural area, $22.87 –Opportunity cost represents approx. 9 days of a child’s wages in coffee harvest

November 6, 2003Social Policy Monitoring Network11 Supply-side Interventions, Education Cash injection for elementary schools, determined in part by size of target population, grades and number of teachers employed (around $6,000 per year) University level training of teachers (expansion of highly successful training program in non-project municipalities) Empowerment of the school PTAs

November 6, 2003Social Policy Monitoring Network12 Evaluation design: cluster-randomized trial Demand (D) Supply (S) WithWithout (W/o) Voucher (V) Group 2 (G2)Group 1 (G1) No Voucher (NV) Group 3 (G3)Group 4 (G4)

November 6, 2003Social Policy Monitoring Network13 Factorial layout with repeated measures Factorial layout with repeated measures Time  Base Line (B) Aug-Dec 2000 Mid term (M) May-Oct 2002 Final 04 WithW/oWithW/oWithW/o V G2 (20: 1543 ) G1(20: 1543) G2 (1492) G1 (1541) G2G1 NV G3(10: 742) G4(20: 1458) G3 (674) G4 (1367) G3G4 D S 

November 6, 2003Social Policy Monitoring Network14 Randomization Carried out by children, in presence of stakeholders + state institution in charge of administrative probity –Transparent and fair Stratified by 1997 nutritional status to ensure balance between groups (5 strata of 14) Within each stratum, random allocation by drawing balls

November 6, 2003Social Policy Monitoring Network15 Randomization

November 6, 2003Social Policy Monitoring Network16 Analysis Strategy Y ij:k:lm = μ + D l + S m + T j + DS lm + DT lj + ST mj + DST lmj + G k:lm + M i:k:lm + TG jk:lm + ε ij:k:lm (Murray 1998) If p < 0.10 for DST lmj G k:lm correlation among households within a municipality M i:k:lm correlation among the repeated observations taken on the same household TG jk:lm correlation among households within a time x municipality

November 6, 2003Social Policy Monitoring Network17 Comparisons What is the impact of the household cash transfers? (G 1M - G 1B ) > (G 4M - G 4B ) What is the impact of the incentives to service providers? (G 3M - G 3B ) > (G 4M - G 4B ) What is the combined impact of the two interventions? (G 2M - G 2B ) > (G 4M - G 4B )

November 6, 2003Social Policy Monitoring Network18 Sample evolution by group CategoryDemandD+SSupplyControl Original (%) Derived (%) Deaths (%) Lost (%) Persons

November 6, 2003Social Policy Monitoring Network19 Sample evolution by age, women Category Original (%) Derived (%) Deaths (%) Moved (%) Persons

November 6, 2003Social Policy Monitoring Network20 Sample evolution by age, men Category Original (%) Derived (%) Deaths (%) Moved (%) Persons

November 6, 2003Social Policy Monitoring Network21 Coverage and leakage of Vouchers 2001 Coverage / Leakage DemandD+SSupplyControl NHB Coverage (%) Leakage (%) EB Coverage (%) Leakage (%)

November 6, 2003Social Policy Monitoring Network22 Demand-side Intervention Coverage (78-79%) and leakage (4-5%) of vouchers were adequate Frequency distribution of vouchers was inadequate (two times per year and always behind schedule) The demand-side interventions have been implemented at a level that impact can be measured

November 6, 2003Social Policy Monitoring Network23 Children less than 2 years old living in communitties with and without AIN-C AIN-CDemandD+SSupplyControl Without (%) With (%) Children

November 6, 2003Social Policy Monitoring Network24 Supply-side Interventions Process of Total Quality Management started but did not finish. Inadequate training in Quality Assurance Transfers of equipment and supplies (not drugs) worth approximately US$ 1041 per Health Center, 17% of what was planned Community-based nutrition intervention (“AIN-C”) covering 11-22% of the under-2 population In summary the supply-side interventions have been implemented at a minimum level

November 6, 2003Social Policy Monitoring Network25 Impact on service utilization

November 6, 2003Social Policy Monitoring Network26 Antenatal care (5 or more visits) TimeDemandD+SSupplyControl % [n=313] 54.6% [293] 47.3% [110] 47.4% [232] % [351] 38.1% [328] 35.1% [154] 48.9% [309] Change18.0%16.5%12.2%-1.5% DD19.5% *** 18.0% *** *** p< 0.001

November 6, 2003Social Policy Monitoring Network27 Child taken to health centre at least once in last 30 days *** ***

November 6, 2003Social Policy Monitoring Network28 Post-partum check up TimeDemandD+SSupplyControl % [n=311] 23.8% [282] 22.9% [109] 28.6% [231] % [ % [321] 16.1% [155] 21.0% [300] Change2.8%1.1%6.8%7.6% DD- 4.8% ** - 6.5% ** - 0.8%-

November 6, 2003Social Policy Monitoring Network29 Impact on coverage of preventive health interventions IndicatorDemandD+SSupply Tetanus toxoid % (mother’s report, mr) DTP1/pentavalent % (health card, hc) 6.9*9.5**0.2 Measless % (hc) Weighed last 30 d % (mr)21.7***17.4**7.8 Weighed last 30 d % (hc)15.7**13.1**8.3 **  p  0.01

November 6, 2003Social Policy Monitoring Network30 Conclusions Providing cash transfers to poor Honduran families on the condition that they keep up-to- date with preventive health care had a large impact on the utilization of antenatal care and well baby check-ups, but not of perinatal check- ups Childhood immunization series started more opportunely and growth monitoring was markedly increased while coverage of measles vaccine and tetanus toxoid immunization in pregnant women was not effected

November 6, 2003Social Policy Monitoring Network31 Height for Age Z scores by consumption deciles

November 6, 2003Social Policy Monitoring Network32 Stunting (%) by treatment group

November 6, 2003Social Policy Monitoring Network33 Height -for-age Z score (means) by treatment group and time YearDemandD+SSupplyControl Change Double Difference

November 6, 2003Social Policy Monitoring Network34 Which countries have 50% stunting Afghanistan Ethiopia Nepal North Korea Yemen Zambia

November 6, 2003Social Policy Monitoring Network35 Weight-for-height Z score (means) by treatment group

November 6, 2003Social Policy Monitoring Network36 Anemia (%) in children months old by treatment group

November 6, 2003Social Policy Monitoring Network37 Conclusions No evidence of nutritional impact after two years of implementation Anemia results contrary to the expected The project was not able to work as a social safety net when the coffee crisis hit and current situational status of children deteriorated Levels of stunting and iron deficiency continue to be high