Training Workshop – Module 3 Coverage Evaluation for Preventive Chemotherapy Description of the method Training Workshop – Module 3

Phases of a Coverage Evaluation Module 1: Planning (central level) Survey Area, Survey Population, Sample Size, Selection of Subunits Module 2: Implementation (local level) Creating segments, enumerating households, interviewing respondents Module 3: Interpretation and Action (central level) Interpretation of results, developing action plan

Coverage evaluation surveys: module 3: interpretation & action phase

Analysis and Interpretation of Results Once coverage survey data collection has been complete in 30 selected segments it is time to compile, analyze and interpret the results. The two most common objectives of coverage surveys are: To estimate the PC coverage and determine if it meets or exceeds the target coverage threshold To validate the reported coverage Point estimates of coverage can be calculated by hand or within the CSB. These coverage figures can be calculated by hand as well as within the CSB program. We’ll show both methods

Analysis and Interpretation of Results 1. Estimating PC Coverage Because the survey methodology produces an equal probability sample, no weighting of the results is required. The estimated coverage can be calculated as follows: Programme Reach = [Number of “yes” responses to offered the drug] [Total number of people surveyed]

Analysis and Interpretation of Results Programme Reach Interpretation: Indicates the proportion of the population that has the opportunity to participate in MDA There is no official target threshold for this indicator Low coverage could be an indicator of supply chain issues, drug distributor challenges, or inadequate social mobilization Can be compared with surveyed coverage to isolate the rate of individual compliance with MDA

Analysis and Interpretation of Results 1. Estimating PC Coverage Because the survey methodology produces an equal probability sample, no weighting of the results is required. The estimated coverage can be calculated as follows: Survey Coverage = [Number of “yes” responses to having swallowed the drug] [Total number of people surveyed]

Analysis and Interpretation of Results Estimating PC Coverage Compare the survey coverage with the target coverage threshold Disease Survey Population Target coverage threshold Lymphatic filariasis Everybody living in the survey area >65% Onchocerciasis Schistosomiasis May vary, based on national treatment priorities and could include: School age children(5-14 years) High risk adults 75% (applies only to SAC) Soil-transmitted helminthiasis (STH) Preschool age children (1-4 years) School age children (5-14 years) Women of child-bearing age Everybody living in the survey area at the time of MDA (for LF) Trachoma 80%

Analysis and Interpretation of Results Estimating PC Coverage Interpretation: Survey Coverage < Target Threshold → MDA not effective and needs improvement X% X% = Survey Coverage

Analysis and Interpretation of Results Estimating PC Coverage Interpretation: Survey Coverage > Target Threshold → MDA effective, programme functioning well X% X% = Survey Coverage

Analysis and Interpretation of Results Estimating PC Coverage Interpretation: Survey Coverage near Target Threshold → calculate confidence interval to determine if it is above target threshold X% Note: the confidence intervals around survey coverage cannot be calculated by hand because they need to incorporate the cluster sampling. X% = Survey Coverage

Analysis and Interpretation of Results A results entry form is available in the CSB for data entry. This form will automatically calculate the programme reach coverage and survey coverage overall and disaggregated by gender.

Analysis and Interpretation of Results Complete Exercise 3.1 Open saved CSB file

Analysis and Interpretation of Results

Analysis and Interpretation of Results 2. Validation of Reported Coverage The survey coverage can be compared to the reported coverage. If the two figures are similar then reported coverage can be considered “validated”. If the two figures are different then there may be a problem with the reported coverage. Reported Coverage Y% Survey Coverage X%

Analysis and Interpretation of Results 2. Validation of Reported Coverage But “similar” vs. “different” can be subjective. A more objective method is to calculate the two-sided 95% CI around the survey coverage and see if the reported coverage falls within this confidence interval. Survey Coverage ( ) Reported Coverage Y% 95% confidence interval Validated!

Analysis and Interpretation of Results 2. Validation of Reported Coverage But “similar” vs. “different” can be subjective. A more objective method is to calculate the two-sided 95% CI around the surveyed coverage and see if the reported coverage falls within this confidence interval. Consider a DQA Survey Coverage Not Validated ( ) Reported Coverage Y% 95% confidence interval

Analysis and Interpretation of Results 2. Validation of Reported Coverage An online tool exists to help you calculate the 95% CI and design effect: https://analysis.linkssystem.org/ Save your data in the Results Entry Form as a csv file (be sure not to add or remove any rows or columns) Go to the website listed above Upload your saved Results Entry Form (csv file)

Analysis and Interpretation of Results Complete Exercise 3.2

Developing an Action Plan The Action Plan helps programmes to convert the results of a coverage survey into concrete steps. The Action Plan should be developed collaboratively with the national NTD program.

Developing an Action Plan Table 3 in the WHO “Coverage Evaluation Surveys for Preventive Chemotherapy: Guidelines for Implementation” offers potential causes to investigate and corrective actions to take for each of the most common coverage survey findings.

(and in Context of Nigeria) Develop an Action Plan Complete Exercise 3.3 (and in Context of Nigeria)