Hugh Waddington What works in WASH? Evidence from systematic reviews Hugh Waddington Geneva Evaluation Week 7 May 2015 International Initiative for Impact Evaluation

Hugh Waddington 3ie is managing 3 linked work streams Impact evaluations Funding available for 2 studies RFQ (2-stage awards) Review of proposals from June 7 th Presentation of baseline findings in Kathmandu (Nov-Dec 2015) Draft report in December 2016 Mid-term review WSSCC medium-term strategy IFMR contracted Draft report by January 2016 Systematic reviews 2 questions Call closes May 12 Contracting in June Draft reports in 2016

Hugh Waddington Call for SR proposals: deadline 12 May 2 questions developed with WSSCC and 3ie-WSSCC programme advisory group What is the effectiveness of interventions aiming to promote sanitation and hygiene behaviour change in communities? To what extent has the sanitation and hygiene sub- sector taken into account the life-cycle approach in the design, implementation, maintenance and use of programmes during the MDG period?

Hugh Waddington “Literature reviews are like sausages... I don’t eat sausages as I don’t know what goes into them” Dean Karlan

Hugh Waddington 3ie Evidence gap map gapmaps.3ieimpact.org

Hugh Waddington A sector-wide logic model

Hugh Waddington Key evidence gaps 138 Impact and 26 SRs of WASH evidence in low and middle income countries. Mostly measure diarrhoeal health outcomes (not very rigorously) Very few studies examining demand-side approaches (eg CLTS, san marketing) Few estimate impacts for sanitation and hygiene programmes as part of scale up No rigorous prospective studies of sanitation programmes in SSA Few on governance

Hugh Waddington 3ie systematic reviews

Hugh Waddington 9 Methodology Extensive search of published and unpublished sources (updated in ) –PubMed, Embase, LILACs, Web of Science –JOLIS, IDEAS, British Library for Development Studies (BLDS), Cochrane Library, scholar.google –Personal communication with leading researchers –Hand-search, back-referencing and citation tracking Inclusion criteria: –IEs measuring impact of intervention on diarrhoea morbidity using experimental (RCTs) and quasi-experimental methods –reported specific water, sanitation, and/or hygiene intervention(s); –were conducted in low- or middle-income countries; –use an infant or child as the unit of observation; and –estimate impact on diarrhoea morbidity, measured under non- outbreak conditions.

Hugh Waddington WASH systematic reviews search process 74,181 records identified through database search Screening Included Eligibility Identification 1,024 records identified through other sources 49,472 records after duplicates removed 1,869 records screened 47,603 records irrelevant 225 full-text articles assessed 120 full-text articles excluded 137 studies included in quantitative synthesis of effects 44 studies included in qualitative synthesis ‘Factual’ evaluation studies (eg qualitiative, ethnographic etc) ‘Counterfactual’ impact evaluation studies

Hugh Waddington 11 Diarrhoea impacts: evidence from 71 studies update ongoing)

Hugh Waddington Sustainability 1: less impact over longer periods

Hugh Waddington 13 Sustainability 2: compliance falls over time Ceramic filter provision in Cambodia; 3 years later only 31% households were still using the filters (Brown et al, 2007) Pasteurisation in Kenya; 4 years later only 30% continued to pasteurise their water (Iijima et al, 2001) Programme promoting POU water disinfectant in Guatemala 1 year later; repeated use among only 5% of households from original trials (Luby et al, 2008). Water filters in Bolivia; compliance 67%; assessment made 4 months after trial ended (Clasen et al, 2006)

Author name Sustainability 3: lack of WTP (Kremer et al. 2012) In Kenya, access to free chlorine increased uptake to over 60 percent, whereas coupons for even a 50 percent discount had a minimal effect

Hugh Waddington Sustainability of hygiene impacts Source: Waddington et al ie Systematic Review

Hugh Waddington Scalability of hygiene promotion Vietnam (Chase & Do, 2012): –Handwashing BCC (not soap) scale-up through 1) mass media & 2) inter-personal communication –No health or productivity effects Peru (Galiani & Gertler): –Hygiene promotion through 1) mass media & 2) community level (health facilities, schools) –No impacts health or nutrition (compliance measured) Bangladesh (Huda & Luby 2012): –SHEWA-B local community hygiene promoters –Low compliance rates & no impacts on diarrhoea or ARIs

Hugh Waddington Supply doesn’t create its own demand Reduction in child disease rates not observed by carers or seen as substantial enough benefit to warrant costs (money/time) – what role for health education? Adoption of innovations (social change) is a slow process (early adopters vs. laggards) – what role for ‘triggering’ (CLTS)? Perceived benefits important (e.g. user satisfaction, time-use, safety) – role for appropriate technology/maintenance?

Hugh Waddington 18 Thank you Systematic review evidence library: