Water Quality Case Study: Kenya Rural Water Project SPRING CLEANING NOTES Water Quality Case Study: Kenya Rural Water Project Michael Kremer, Harvard University and NBER Edward Miguel, U.C. Berkeley and NBER Jessica Leino, World Bank Sendhil Mullainathan, Harvard University and NBER Clair Null, Emory University Alix Zwane, Bill and Melinda Gates Foundation Presented by Vivian Hoffmann, University of Maryland DIME-AADAPT SAR December 19, 2009 1 1

Motivation 2 million children die of diarrheal diseases annually, contaminated water is important pathway of transmission Domestic water is a common demand in CDD initiatives (especially by women) Impact evaluations can help identify effective technologies for improving water quality Same methodologies can also help identify which technologies are valued by beneficiaries Essential to understand willingness-to-pay for scale-up and long-run sustainability

Setting Rural Western Kenya Demographics Water Hygiene & Sanitation SPRING CLEANING NOTES Setting Rural Western Kenya Demographics Mothers have 6 years of education 4 children under age 12; 1 or 2 children under age 3 per compound Water Nearest source is 8 minute walk from compound Most households get water from unprotected springs < 20% of HH’s meet US drinking water standard < 30% boiled yesterday’s drinking water Hygiene & Sanitation > 80% have a pit latrine > 90% have a soap Let me just give you a sense for the context in which we’re working. We show balance on all these characteristics (and more) in Table 1 of the paper. 3 3 3 3

October 2008 - CAS Trickle Down 4 SPRING CLEANING NOTES Instead of improving water quality in the home. Water quality at the source can be improved. The usual source that one thinks of in rural africa might be a well, but many people also actually use naturally occurring springs. ** This picture shows a young boy collecting water at a naturally occurring spring. -- As you can see, some wood has been placed around the eye of this spring, but the water pools at the collection point where it can easily be contaminated with surface water run-off. In an agricultural area with incomplete sanitation coverage, this makes it easy for fecal matter (from either humans or livestock) to contaminate the collected water. -- You can also imagine in this picture how contamination in transport and storage might occur. Children sometimes collect water and can easily touch it in open containers. If this kid here has fecal matter on his hands and makes contact with the spring water (which is likely), he could easily contaminate it. Similar things can happen within the home. When water is scooped out of the top of storage containers with a dipper, it is hard to avoid touching the water. October 2008 - CAS Trickle Down 4

October 2008 - CAS Trickle Down 5 SPRING CLEANING NOTES ** Contrast this with a beautiful protected spring ;) -- Spring protection seals off the source of a spring (the “eye”) and encases it in concrete so that water flows out from a pipe rather than seeping from the ground. -- Since the water does not pool on the surface, it can be collected prior to contamination from run-off. -- Protection does not prevent all contamination at the source (i.e., groundwater contamination) -- Contamination in transport, storage still possible -- Spring protection costs roughly $1000 to construct ($800-$1100 depending on the conditions, soil, slope, etc.), and a well-maintained spring can easily last for 10 years and usually serves 20-40 households in our area, but many more households in more densely populated peri-urban areas. -- Maintenance is also pretty simple. You need to protect the catchement area upstream, and keep storm water / drainage ditches clear. The labor cost is around $55 per year. October 2008 - CAS Trickle Down 5

The Project (Phase I) Multiple interventions to improve water quality SPRING CLEANING NOTES The Project (Phase I) Multiple interventions to improve water quality Researchers worked with a local NGO 652 springs were identified for protection to improve water quality and convenience Due to capacity constraints, able to protect only one quarter of total springs each year Randomized order of phase-in Compared with household water treatment intervention (chlorine) Let me just give you a sense for the context in which we’re working. We show balance on all these characteristics (and more) in Table 1 of the paper. 6 6 6 6

Household Water Treatment Point-of-use treatment with dilute chlorine reduces diarrheal disease by around 40%, yet relatively few households use chlorine even in areas exposed to several years of vigorous social marketing In Kenyan study area: 70-90% of households familiar with local brand of chlorine About as many volunteer that “dirty” water is a cause of diarrhea Only 5-10% of households regularly use chlorine to treat their water

150 mL bottle treats a household’s water supply for roughly one month SPRING CLEANING NOTES Dilute chlorine (similar to chlorination in centralized water supplies in rich countries) Safe: developed by US CDC & PAHO, distributed and marketed by Population Services International in over 20 countries Chlorine smell and taste is strong at first (prevents overdosing / kids drinking straight from bottle), fades after a few hours One capful disinfects 20L of water, with residual protection against recontamination 150 mL bottle treats a household’s water supply for roughly one month Costs 20 KSh (US$0.30), a quarter of the daily agricultural wage Let me describe the product for you. 8 8 8 8

Spring Protection vs. Chlorine Distribution Study Design (Phase I) Spring Protection vs. Chlorine Distribution Identify Springs Protect in years 1-2 Protect in years 2-4 randomize Baseline HH survey, water quality tests, GPS data Interventions Follow-up HH survey & water quality tests 6 mo chlorine + coupons control 6 mo chlorine + persuasion 6 mo chlorine 6 mo chlorine + coupons 6 mo chlorine + persuasion 6 mo chlorine control

Data Distance to main source of drinking water (GPS) Water quality (fecal coliform) at the spring Water quality in the home Residual of chlorine in drinking water for study households and social contacts Coupon redemption from shopkeepers Child diarrhea reported by mother

SPRING CLEANING NOTES ** This is a Google Earth screenshot of one of our sample areas. You can see household locations in orange, sample springs as yellow thumb tacks, and alternative sources as black dots. -- Consider HH 849013. They would naturally use the alternative source 85102 or even 84901 if spring #849 weren’t very good. But after it’s protected they could switch over to it. We can capture these sorts of switches in our data, and using GPS location as well as self-reported walking distances, can measure how much farther households are willing to walk to use a higher quality water source. We discuss in detail this switching, and how we can use it to estimate WTP for spring protection in the Spring Cleaning paper. We can back out a revealed preference valuation for improved source water quality by measuring how much extra time people are willing to spend collecting water from springs, post protection, relative to other sources, and multiplying this by a value of time. To our knowledge, this is the first such use of this approach to measuring the value of improved drinking water quality in a developing country– and it generates results that are far below stated preference valuations. We will come back to comparing revealed preference valuation for source water quality interventions and the point of use treatment in a minute.

Summary of Spring Protection Results Protecting springs led to 66% less source contamination, moderate gains at home; child diarrhea fell by a 25% Households value spring protection at 12.7 work days ($4.52-$9.02) per year Policy implications: comparing this to the cost of spring protection, it appears socially optimal to only protect springs with large numbers of household users. (does not include health benefits) ** Summarize our main findings, and answering the four questions posed at the start of the talk. (2) There is still plenty of debate over the causes of diarrheal disease (water quality, quantity, hygiene, etc.). These results are consistent with waterborne pathogens (those that would be reduced by an improvement in water quality) being a key determinant of diarrheal disease (i.e., maybe the key disease vector is dirty hands for example). (4) The first policy pay-off from our WTP estimates is this finding about when it might be socially optimal to protect springs. -- Perhaps this is not too surprising, since people do not value spring protection very much. It isn’t a market failure when no one really wants the product.

Household Water Quality Impacts See table 2 for more details When a household got a supply of chlorine they actually used it. At the unannounced follow-up visit, 79% of hh that received WaterGuard reported that their current supply of drinking water was treated and more than half (58%) had detectable levels of chlorine in their drinking water. I’m telling you both of these numbers because the first is likely an overstatement, but the second is likely a lower bound. Because chlorine decays, especially in ceramic containers, which is what many of the hh in our sample use, the chlorine test has some false negatives. Factoring in baseline take-up rates and time trends, we estimate the effect of the intervention to be a 69 percentage point increase in self-reported chlorination and a 53 percentage point increase in validated chlorination (Table 2, columns 1 and 5). These are huge effects relative to baseline self-reported and validated chlorination rates of 6% and 2%, respectively. Remember– this is a product available in shops that they mostly knew about, and could have adopted themselves but did not do so. Unlike most Epi studies– there were no implicit reminders triggered by visits from field staff. We see little evidence that take-up is related to various household characteristics– this is presented in table 2. Most importantly, the households that might benefit the most, those with young children vulnerable to diarrheal mortality are not more likely to use it.

Child Health Impacts See table 4 for more details***** As you would expect, the drastic reductions in E.coli and the big progress on the % of HH’s whose water met US EPA standards translated into major health benefits. Prevalence of 20% among kids under 3 in control households. Treatment associated with 7-8 percentage point reduction in diarrhea on average (Table 4 columns 3-4, significant at 95%) WaterGuard & spring protection appear to be substitutes (Table 4 column 5) No differential treatment effects for boys versus girls or on the basis of other household characteristics (latrines, hygiene knowledge, mother’s education, etc.)

Price and WaterGuard Use SPRING CLEANING NOTES Price and WaterGuard Use Briefly discuss difference between self-report and positive chlorine tests here; danger of courtesy bias 15 15

Can price serve as a screening mechanism? Some argue that price will help to screen out those who aren’t likely to value or use the product Ashraf, Berry, and Shapiro (2008) in peri-urban Zambia Present study found no evidence that households who stand to benefit most from cleaner water (i.e. those with young children) have higher willingness to pay Draws into question appropriateness of retail model

Study Design (Phase II) Alternative Approaches to Promoting HH Chlorine Use Sample of Springs randomize Flat fee promoter + coupon for one free bottle Social Marketing Control Chlorine dispenser + incentive pay promoter Incentive pay promoter + coupon for one free bottle

Point-of-collection Chlorine Dispenser SPRING CLEANING NOTES Point-of-collection Chlorine Dispenser Cost-reduction is b/c of packaging reductions (actual chlorine itself is cheap; custom-made little plastic bottles aren’t) Promoters were given an “emergency” stock of WG to refill if necessary, but IPA visited regularly to refill. 18 18 18

Point-of-collection Chlorine Dispenser Drastically cuts the cost of supplying chlorine Approximately 25% of current individually-packaged retail cost Salience Convenience Walk home provides agitation and some of wait time Dose is more precise; doesn’t get on hands Habit formation Links water treatment to existing habits associated with water collection Harnesses social network effects Makes decision public 19

Take-up by Treatment Arm As in previous study, persuasive messages don’t seem to have accomplished much (community script had small effect, but only in short run) Promoters did a great job; seem to make demand much less sensitive to price (verified take-up barely falls in medium-run) Surprisingly little difference based on pay scheme – going forward will be looking into effectiveness of unpaid promoter Dispensers actually going way up between short- & medium-run: highest take-up rates by far 20

SPRING CLEANING NOTES Persuasion and Peers More intensive marketing messages had no effect on take-up Rich relationship data on all pair-wise combinations of sampled households in each spring community Intervention drastically increased frequency of conversations about WaterGuard Mixed evidence of take-up effects depending on outcome On basis of self-reported chlorine, comparison household twice as likely to use chlorine if all of her close friends were members of treatment group (sig. at 90% confidence) On basis of positive chlorine tests, no effect of social networks Members of the same tribe and community leaders are especially influential on the basis of either measure ALSO MENTION THAT WE DON’T KNOW IF NETWORK EFFECTS WOULD BE MECHANISTIC THROUGH SHARING OR SOMEHOW LEARNING (CONVINCING OTHERS TO GO OUT AND BUY) – THIS DIDN’T FIT ON SLIDE -- Let HH’s volunteer descriptions of their relationships with one another -- Common familial relationship types reflect survey protocol (mother of youngest child) and cultural tradition of moving to husband’s village: 20% are mother/daughter-in-law and 25% are wife of brother-in-law -- Relatively ethnically homogenous (given recent political developments, important statistic) Respondents in treatment group three times as likely to report conversation about WaterGuard with another household Respondents twice as likely to report conversation about WaterGuard if the other household was in the treatment group -- Around 20% of relationships are “lopsided” (one side says doesn’t know, the other says talk) -- Average number of total close contacts (not just treatment) is 4 – consistent w/ randomization. Those who got WG are randomly determined, critical for the analysis that follows. 21 21 21 21

Take-up of Free Chlorine Results very consistent between first and second phases of research. Doesn’t seem to matter much whether the free supply is delivered to your home or available at the source. Encouraging b/c home delivery totally unsustainable. Dispensers might be feasible. Take-up is very high even among “non-baseline” HH’s – those who weren’t surveyed and thus didn’t get the expensive personal message from the NGO staff. Very encouraging. 22

Conclusions: Household Water Treatment Very high take-up rates of chlorine for point-of-use water treatment when it is provided for free But demand is very sensitive to price Persuasive messages don’t seem to make much difference beyond short-run; mixed evidence on peer effects, but community leaders do seem important Local chlorine promoters were very influential, even when price discounts ran out Changing the way the product is delivered could be the solution: Point-of-collection chlorine dispensers hold the most promise for a sustainable strategy for increasing take-up at scale Drastically reduces cost (mainly through packaging) Harnesses peer effects by making use decision public Helps to build habits of consistent use (more convenient)

Lessons for Impact Evaluation Possible to learn a lot by trying multiple strategies varied technology, price, message Iterative design allows development of effective new approaches e.g. free delivery works (but is expensive)  chlorine dispenser makes free delivery much cheaper next step: how to manage and finance dispensers?

SPRING CLEANING NOTES Thank you! 25 25 25 25