 Evaluation of Surveillance Systems St Lukes-Roosevelt

Problems with our field  Programs often do more harm than good  Programs don’t collect data, so no benefit shown  The data we do collect is often not useful for improving program quality or guiding policy

How do we show benefit, impact, change?  Surveillance  Ongoing  Surveys  One point in time

Definition  Public health surveillance is the ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and mortality to improve health *CDC, Atlanta GA

Key concept

Why evaluate a surveillance system?  Ensure that problems of public health importance are being monitored efficiently and effectively  Recommendations about the system should focus on improving quality, efficiency, and usefulness

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity: combine a practical structure with ease of use  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility: ability to adapt to changing information needs or operating conditions with minimal time, effort, cost  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality: completeness and validity  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability: willingness of persons or organizations to participate  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability

Contraceptive prevalence rates in Afghanistan, WHO

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity: ability to detect cases OR ability to detect outbreaks  Predictive value positive  Representativeness  Timeliness  Stability

Incidence* of Shigella Dysentery Central Bosnia, RegionPrewarMay-July 1993 Sarajevo City (+1250%) Zenica City (+1690%) Tuzla Region (-10%) *Cases per 100,000 per month

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive: proportion of persons identified as cases who truly are cases  Representativeness  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness: system accurately describes events over time and space (time, person, place)  Timeliness  Stability

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness: speed between steps; appropriateness in delays  Stability

Epidemic curve, outbreak of mumps, Montreal

Epidemic curve, cholera

What should be evaluated?  System attributes: determine priorities  Simplicity  Flexibility  Data quality  Acceptability  Sensitivity  Predictive value positive  Representativeness  Timeliness  Stability: reliability and availability; resources

Steps in evaluating a surveillance system  Stakeholder engagement  Describe the system: importance, purpose, resources  Focus the evaluation design  **Gather evidence regarding performance  Justify and state conclusions, make recommendations

Malaria Surveillance  Purpose (CDC): (a) identify local transmission; (b) guide prevention recommendations for travelers  Additional benefits (JE)  Identify emerging species; treatment failures; local outbreaks  Historically  Tracking elimination  Case definition  Malaria cases confirmed by blood film, rapid diagnostic tests, PCR

Malaria Surveillance  The system  **National Malaria Surveillance System  National Notifiable Disease Surveillance System (1878 cholera, smallpox, plague, yellow fever at overseas consules)  Direct CDC consultation