Surveillance Evaluation Assist Prof Dr. Kwankate Kanistanon, DVM, MS, PhD

What is health surveillance “The tracking and forecasting of any health event or health determinant through the continuous collection of high quality data, the integration, analysis, and interpretation of data into surveillance products and the dissemination of surveillance products to those who need to know a specific public health purpose or policy objective”

Why evaluate surveillance To improve and support a health surveillance

Evaluation helps answering the questions 1. What are the success and deficiency of the surveillance system? 2. Is the surveillance system meeting its piblic health objective? 3. How does surveillance support and benefit stakeholders? 4. What measures could improve performance and productivity of the surveillance system?

What to evaluate? 1. The quality of the information that their systems produce 2. The effectiveness in supporting the objectives of the programs that they serve 3. The effectiveness in supporting informed decision-making 4. The efficiency of their systems

Effectiveness How well surveillance can produce purposed outcome Is the information use? Response to the objectives of surveillance

Efficiency Cost, time, and labor Resources consumed and the value of outcomes

6 Steps for surveillance evaluation 1. Establishing the context of the surveillance 2. Developing evaluation questions 3. Designing the process for data collection and management 4. Summarizing and presenting the findings 5. Reviewing an evaluation report 6. Following up on the use of findings

Step 1: Establishing context Why evaluate? You must be able to identify the reasons for evaluation Who are stakeholders for evaluation? You must identify the persons or organization that will benefit from the evaluation What is this surveillance system? You must find information of surveillance before you evaluate it

What is this surveillance? Purpose Population under surveillance What data is collected and summarized Who uses the surveillance What action is expected from data analysis and interpretation Activities of surveillance

Example of surveillance activities Identify the disease, case, population Develop data collection tools, active or passive surveillance Data collection Data audit and analysis and Interpretation Report writing Dissemination of results

Step 2: Evaluation questions Questions are designed to ask for 2.1. Acceptability 2.2. Simplicity 2.3. Flexibility 2.4. Data quality 2.5. Positive predictive value 2.6. Sensitivity

Step 2: Evaluation questions 2.7. Representativeness 2.8. Timeliness 2.9. Stability Compliance

2.1 Acceptability = Willingness of persons and organizations to participate in the surveillance Are the data collected within the normal course of operations? Is there a mutual understanding of jurisdictional mandates, addressing security, and privacy

2.2 Simplicity The ease of operation Appropriate technologies and labor

2.3 Flexibility Accommodate changes in operating conditions Can the system respond to new conditions, such as changes in case definition, labor, equipment Can the system accept, process and forward another system’s information

2.4 Data quality Can data describe the health event according to the objective of the surveillance? Is the quality of data acceptable? Missing data Correct conversion Historical and current data having different meaning Same data have more than one definition

2.5 Positive predictive value = The percentage of cases reported to the system that actually have the health problem How much the diagnosis depend on skill of observers? Is there a process that the diagnosis is confirmed?

2.6 Sensitivity = Percentage of cases with disease detected by the surveillance Ability to detect outbreak in a timeframe correct number of diseased animals

2.7 Representativeness The surveillance can explain the health problem by person, place, and time Can describe population characteristics Can describe diseased animals characteristics

2.8 Timeliness Appropriateness of interval between disease occurrence and Report to health agency Identifying the incidence by health agency Implementation of control measures

2.9 Stability Stability is measured by amount of time required to manage and disseminate the information to decision makers

2.10 Compliance Comply with law or policy Ethics Confidentiality Security Privacy

Step 3: Data collection/Management Collect data for evaluation Observation Interview the surveillance staff Reviewing the surveillance report

Step 4: Findings Summarizing information gathered from the evaluation

Step 5: Reviewing the report Presentation and dissemination of results How and whom should the report sent to? What evaluation report should say? Clarify the weak and strong point of the surveillance Any suggestion for surveillance?

Step 6: Follow-up Activities after the evaluation Allocation of resources Creation of a policy Technical assistance to the surveillance

How to describe the evaluation results 1. Scoring system 1 = Must be improved 2 = Poor 3 = Fair 4 = Good 5 = Excellent 2. Descriptive results: บรรยาย 3. Combined system: Scoring and text at some points

Goal of evaluation To improve and support a health surveillance