1. Programme Data and Coverage Surveys Challenges to improve programming
Pleasure to be here with old colleagues and experts in the field who have contributed enormously to the management of SAM
I present a relatively simple view of the challenges to improve programming with current available data.
UNICEF 2013

2. Nutrition Programming - Coverage is critical
Coverage is critical for nutrition programming. Here the densely worded diagram presents the Lancet recommended interventions through the life cycle
These Activities / Nutrition Pillars
The bulleted community based Management of Acute Malnutrition highlighted in pink here represents a just a part of the overall programming.
We recognize the weaknesses in the implementation at the same time, we cannot dedicate 100% of efforts to treatment.
UNICEF has to try to maintain an appropriate balance between Prevention and Treatment.
With that in mind, we go into estimates of annual case loads.

3. Annual estimated caseloads of severe acute malnutrition across the Sahel
In 2010, Nutrition Cluster in countries described their own methods
variations of
Annual caseload = Pop 6-59 m * Prevalence SAM *Conversion Factor (X) + Safety Margin (X%)
From 2012, a standard calculation used in all countries
following calculation defined by Mark Myatt
Annual caseload = Pop 6-59 m * Prevalence SAM *Conversion Factor (2.6)
In 2010, The Nutrition Cluster or equivalent in countries described their own methods in West and Central Africa.
These were Annual caseload = Pop 6-59 * Prevalence SAM *Conversion Factor (X) + Safety Margin (X%)
X country example– set annual case load estimates to feasible number and achieve target
Y country example– estimate case load as rigorously as possible and struggle and not reach target
From 2012, Regional Director of UNICEF WCAR insisted on a standard calculation for all countries. The Garenne calculation clearly described by Mark Myatt was used
and was well accepted . Except for in the Lancet crowd. Different story.

4. Severe Acute Malnutrition
What information is needed for case load estimation of severe acute malnutrition ?
Current Cases of
Severe Acute Malnutrition
New cases
Exits
Accurate incidence data from effective large scale programmes
Accurate population and prevalence estimates
Duration of case of severe acute malnutrition as defined by WHZ and MUAC
Velocity of increase or decrease of new cases following seasonal / temporal variation
Everyone knows what is needed to estimate the caseload, BUT I would like to emphasize the arc in read that represents the
Velocity of increase or decrease of new cases following seasonal / temporal variation.
We can model seasonal variation based on available data, but we cannot predict the future, so we cannot predict the annual variation that affects the number of potential SAM cases.

5. Mapping of geographic coverage of northern Nigerian states
100% of targeted severe acute malnutrition caseload achieved in only ~30 % geographic area of northern states
The map of Nigeria presents the geographic coverage of the management of SAM program.
The pink areas are the LGAs with the program in operation. These were chosen based on the prevalence and the population affected across the LGAs.
In 2012, The program reported to have met 100% of the targeted caseload of the northern states with only about 30% of geographic coverage.
We recognize the incomplete delivery of services across the northern states and problems that we have in these data and are attempting to address this with significant new support.

6. Comparison of coverage with the severe acute malnutrition caseload in Maradi, Niger in 2011
Estimated number of children 6-59 months of age with
severe acute malnutrition in Niger, May 2011
Prevalence of SAM- WHZ 1.6% in May 2011
102,500 SAM cases treated in Maradi in 2011
Coverage estimates of 24% in Maradi from 5 region coverage survey in 2011
Assuming no over-reporting the annual caseload corrected by coverage would be – 425,000 cases
Population 6-59m of Maradi ~578,000
We follow with a more complex example taking into account coverage survey results.
The map of Niger presents the estimated number of children with SAM based on population and the May 2011 National Nutrition Survey. The regions with the highest number of children affected are Tillabery and Maradi. But the population concentration in this small region has proven to be fertile ground.
Maradi Niger has one of the strongest management of acute malnutrition programmes in the world. There are several NGOs (Befen, Forsani, MSF, ACF, Alima) that support health districts with high quality care and training for other regions. In 2011, 1/3 of the reported new admissions in Niger were delivered in Maradi. IF you were to expect to find high coverage anywhere…
The prevalence of SAM based on WHZ in May 2011 was 1.6%. There were a reported 102,500 cases treated by the regional program. With the coverage estimate of 24.1% and assuming no over reporting error, the annual caseload corrected by the coverage estimate would be 425,000 SAM cases. If each case was one child, this would corresponds to 68% of the Maradi 6-59m population
These calculations are improbable when reviewing the SAM prevalence of 1.6%
(3.1 million total population of Maradi).

7. Why are there such discrepancies?
Inputs to annual caseload estimates
Prevalence of severe acute malnutrition
Population estimates
Prevalence to incidence conversion factor
Coverage estimates
Why are there such discrepancies?
In the Inputs to annual caseload estimates
- the prevalence of severe acute malnutrition can be greatly overestimated when there is poor data quality. But this doesn’t explain the problem.
- Population estimates, there are errors, some countries have better data than others
- Prevalence to incidence conversion factor – forces a standard model on a non-standardized world. For example the conditions in Senegal are clearly different than those in Chad.
Coverage Estimates, when you talk with Saul Guerrero about the data collection at the community level, it is very convincing.
But surveys are tricky beasts.

8. LQAS Sampling Methods
With coverage estimates, there are no Niger results using other sampling methods to verify those estimates made with S3M methods
National level surveys collecting IYCF indicators with LQAS samples
Liberia IYCF results
Nigeria IYCF results
For validation of coverage estimates of the Niger Survey, we must have other direct estimates to compare. Unfortunately there are no other data on coverage using direct methods.
There are estimates of IYCF indicators. We can compare indicators collected with LQAS methods to other surveys with standard sampling.

9. Measures of Exclusive Breastfeeding with LQAS in Liberia
I would like to review the measures of exclusive breastfeeding in the LQAS survey of 2011 in Liberia. This was done by Measure Evaluation, who I used to work for, so it better be good.

10. Measures of Exclusive Breastfeeding with LQAS - Liberia
The national level results range from 22 to 34 %. The exclusive breastfeeding county level results of the LQAS sampling survey produce estimates from 77 to 93%. These are 2-3 times higher than results from large national surveys.
These counties represent 1/3 of the national population.

11. Measures of Exclusive Breastfeeding with LQAS - Nigeria

12. Measures of Exclusive Breastfeeding with LQAS - Nigeria
The measures of exclusive breastfeeding in the Nigeria LQAS baseline household survey produce estimates that are 3 – 9 times higher than those of the regional estimates from the MICS 2007.
There are issues here of comparing Tangerines to Clementines, seasonality and other points that confounding the comparison, but this does not explain the orders of magnitude of difference in these estimates.
The lesson learned here is that Sampling Methods that that require employment of large number of teams complicates training, testing and supervision of data collection. Poor training, standardization and supervision leads to poor quality data and poor quality estimates.

13. Presentation of data quality indicators into coverage survey reports
Analysis of number of identified cases by data collection points (min, max, mean, median)
Distribution of cases with MUAC < 115mm, Bilateral Oedema, reported appetite
Quality of MUAC measure (accuracy and precision of anthropometrist measures, digit preference, flagged data, use of colored vs non colored MUAC strips)
Age estimation and sex of child
socio-demographic variables of child and or household – comparison to survey data results in households with children with GAM.
Population size of sampling points
GPS validation of survey sampling points
Verification of child in programme with RUTF in HH, treatment programme follow-up cards
Capture / Recapture data analysis
What we would like to see is the presentation of data quality indicators into coverage survey reports
Guessing that a lot of this work has been done, I just have not seen it.
These recommendations come from Nutrition specialists working in country offices
After standardization exercises MUAC measures often have a technical error of measurement of 5mm. With the exact measure of MUAC, could calculate a range of confidence with +/- 5 mm of the reported measure.

14. Management of severe acute malnutrition programme data
I would like to race through programme data
Here in the graphic, we present the trends in weekly admissions. Niger is currently treating about 330,000 new admissions a year
These data are critical for monitoring trends. If biases remain constant then we can interpret variation as real changes. Here where Fete is marked corresponds to the end of Ramadan, and you can see that the number of new admissions dropped considerably.
Problems with programme data
- New Admissionhas been confused with number of cases in programme
- If data are aggregated at too high a level, the information is not helpful for decision making at programme level
- Only new admissions is not sufficient.
New Admissions, Verification with stocks use

15. Stocks and programme exits
Rapid increase of scale of programme often leads to quality issues. Without programme data, these issues are not addressed.
Programme data support:
Integration of management of SAM into regular programme delivery
Ensure lives saved by programme (avoid stock-outs, ensure malaria treatment)
Incorporate preventive interventions (WASH/Nutrition minimum package)
We need stocks and programme data to help validate programme operation and quality of reported information.
With the rapid increase of scale of programing, there are often serious quality issues. Without programme data, these issues are not addressed.
Correct reporting helps to integrate programme into regular health care delivery. The medical leadership in countries often do not buy into community based programming as they have not taken on their appropriate role. They need to be included in the in-patient care and total programme supervision to ensure ownership.
Program data are needed also to ensure that lives are saved by the interventions, to avoid stock outs, treatment with antibiotics and antimalarials.
Also to ensure that preventive interventions such as the WASH Nutrition minimum package is available and in use at implementation sites.

16. Programme data needs Real time data on: New Admissions Stocks
Information Flow
Programme data needs
Ministry of Health
Health Management Information System
Department of Nutrition H
Real time data on:
New Admissions
Stocks
Programme Exits
Without these data, there is no identification or response to critical events that cripple programme delivery.
Monthly reports sent by
or on demand
Regional Health Supervisors H H
IN terms of program data needs, we should have real time data on at least
New Admissions
Stocks
Programme Exits
Without these data, there is no identification or response to critical events that cripple programme delivery.
This is the work of systems building, but we are getting closer to have functional and appropriate tools. For the money invested already in these programs, this is a minimal point to achieve.
District Health Chiefs
IFP
OTP
SFP

17. To address these data challenges
Analysis framework for improved understanding of annual caseloads and programme data compared to coverage estimates
Recommendations for what types of programme evaluations should be conducted when. Timely production of results for critical programme management decisions prior to the hunger season.
For Coverage Surveys of large scale programmes (national or regional)
Standardized robust and cost appropriate sampling methods
Data collection in one month
Standardized reporting models including data quality measures

18. Conclusions Prevention and treatment are two sides of the same coin
Coverage is critical but without quality programme data, coverage estimates are less relevant.
Timely accurate regular coverage estimates should be used to modify and improve programme implementation
Prevention and treatment are two sides of the same coin
The management of severe acute malnutrition program has given health clinics in many countries a reason to exist. We have to continue to build on the energy that is generated by the visual proof of effective programming.
Coverage is critical but without quality programme data, coverage estimates are less relevant.
Timely accurate regular coverage estimates should be used to modify and improve programme implementation