Innovating the measurement of humanitarian impact Valid International Innovating the measurement of humanitarian impact The case of coverage assessment of community-based management of acute malnutrition (CMAM) programmes

Humanitarian Impact of CMAM: Meeting Need Efficacy – Effectiveness - Coverage Valid International 22/02/2019

Efficacy of CMAM protocol Efficacy is about how well the CMAM protocol works in ideal or controlled settings This is measured by cure rate of CMAM and estimated through a clinical trial The CMAM protocol, cure rate is close to 100% in uncomplicated cases in ideal or controlled settings Little room for significant improvement in efficacy Uncomplicated cases – cases at or just below MUAC admission criteria or mild oedema Valid International 22/02/2019

Effectiveness of the CMAM protocol Effectiveness is the cure rate of the CMAM protocol in a normal patient cohort under programme conditions In programme conditions, normal varies hence effectiveness varies Varying severity of cases – less severe cases, better effectiveness Varying compliance to treatment – better compliance, better effectiveness Varying attendance to health facility – less defaults, better effectiveness Effectiveness can be improved! Valid International 22/02/2019

Coverage Coverage is the capacity to meet need Crudely, coverage = no. in the programme / no. who should be in the programme Good coverage depends on Thorough case-finding and early treatment-seeking Good retention from admission to cure Coverage and effectiveness are linked They depend on the same things Good coverage  Good effectiveness Valid International 22/02/2019

Meeting Need Meeting need requires both high effectiveness and high coverage: Met need = Effectiveness x Coverage Since effectiveness and coverage are linked, then Maximising coverage maximises effectiveness and met need Valid International 22/02/2019

Measuring Coverage Method Date Description Modified EPI method Before 2002 Uses nutrition survey data. Problems with heterogeneity and sampling bias. No information on barriers to coverage. CSAS 2002 Spatial sample. Coverage estimated locally (mapped) and globally. Some information on barriers to coverage. Bit too expensive for routine M&E. SLEAC 2008 Rapid method. Classifies coverage at SDU level. Some information on barriers to coverage. Can estimate and map coverage over wide areas (e.g. national coverage surveys). Designed for low cost M&E at clinic or district level. SQUEAC Semi-quantitative method. In-depth analysis of barriers and boosters to coverage. Mapping of coverage using small area surveys. Estimation of coverage using Bayesian techniques. Designed for routine program monitoring tool (intelligent use of routine monitoring data / other data may be collected on a “little and often” basis). S3M 2010 Wide-area version of CSAS using improved spatial sampling and more efficient use of data. Some information on barriers to coverage. Pilot currently ongoing. Valid International 22/02/2019

SLEAC / CSAS Mapping Coverage Valid International 22/02/2019

SLEAC / CSAS Barriers Valid International 22/02/2019

SQUEAC Concept Map of Barriers and Boosters Valid International 22/02/2019

Policy Implications Experience over the past decade has shown that investigation of coverage and factors affecting coverage using these methods informs programme reforms and improves coverage and effectiveness. Programmes that audit coverage using these methods commonly achieve coverage within Sphere minimum standards (i.e. > 50%). The greater accessibility of the new methods allows coverage audits to be done regularly. This in turn potentially ensures more programmes to achieve minimum standards. The current challenge is ensuring that these levels of performance are achieved in national CMAM programmes. Valid International 22/02/2019