Malaria-specific Slides Ghana Case Study Malaria-specific Slides

Data for Decision Making

Class Activity: Is it Monitoring or Is it Evaluation? 1 The Director of Health wants to know if interventions being implemented in Region A are increasing ITN use in pregnant women and children under five in that region Minister of Health requires information on quantities of RDTs used in health facilities in 2009 A country director is interested in finding out if the population knows about and is using the voucher scheme for ITNs 1. Evaluation – If use is your outcome, monitoring of use is your output 2. Monitoring – This is not related to program impact 3. Monitoring

Class Activity: Is it Monitoring or Is it Evaluation? 2 The Director of MCH wants info on # of pregnant women receiving two or three doses of SP (IPT1 and IPT2) Current under-five mortality rate needs to be determined NOTE: DATA ARE KEY TO MONITORING AND EVALUATION 1. Monitoring if use is your output 2. Evaluation

Component 1a: Decision Maker Decision maker is a person responsible for acting at any level: Lower levels: Community Leader Middle level: DDHS Top level: Program Manager, D-G Global level: RBM Executive Director/ WHO DIR-GEN Various levels. Not just top level. We all need data. Examples of decision making at lower levels: A family decides whether or not to take their child with a fever to the clinic based on their impressions of the quality of care during past experiences. An M&E officer decides what type of data collection forms will be most effective in a specific context. The Global Fund decides whether or not to allocate funding to a specific grant.

Stakeholder Analysis (Tool on pen drive, useful for M&E plan to identify stakeholders and their needs) Name of Stakeholder Organiza-tion (and specific individual if required) Stakeholder Description Potential Role in Activity and Use of Results Level of Knowledge of Topic Level of Commit-ment (positive and negative) Constraints to Participating in Activity When to Involve NMCP–Program Manager Oversees malaria control policy/ strategy development and coordinates implementation Primary audience – Access to sites. Service guideline revision High, extensive Strongly supports scale-up of malaria control strategies Busy schedule. Needs at least four weeks lead time Through-out This tool can be found on your CD. It can be utilized in your M&E plan to identify stakeholders and their needs.

Examples of Decisions (could be Deductive, Inductive, or Logical) Policymaking: e.g., ITN Policy Strategic Planning: RDTs/ IRS Targeted Program Management: e.g., Zoning with staff to enhance monitoring Resource Allocation: e.g., GBF Budget Drugs and Commodities, Human Resources, Infrastructure and Equipment

“Making Data Speak” Results: Stakeholders took informed decision to change from chloroquine to ACTs Implementation Framework drawn with timelines, persons responsible, resources needed Task Teams formed to address various aspects Development of Anti-Malaria Drug Policy/Procurement of ACTs Incorporation into Country Drug Policy and Standard Treatment guidelines

“Making Data Speak” (cont.) Results: Updating of training manuals, guidelines Communication and behavior change communication Launching/adoption of new policy Training/capacity strengthening Monitoring: drug quality, pharmacovigilance, use, prescriber habits

Anti-malaria drug policy change is an on-going process Development of Policy Updating of Policy Implementation of Policy** Re-evaluation of Policy Monitoring of Policy

Data Analysis

Average number of confirmed malaria cases per month Mean Sum of the values, divided by the number of cases – also called average Average number of confirmed malaria cases per month Month Cases 2008 Jan 30 Feb 45 Mar 38 April 41 May 37 Jun 40 Jul 70 Aug 270 Sep 280 Oct 200 Nov 100 Dec 29 Total number of cases Number of observations The most commonly investigated characteristic of a collection of data (or dataset) is its center, or the point around which the observations tend to cluster. The mean is the most frequently used measure to look at the central values of a dataset. The mean takes into consideration the magnitude of every value, which makes it sensitive to extreme values. If there are data in the dataset with extreme values – extremely low or high compared to most other values in the dataset – the mean may not be the most accurate method to use in assessing the point around which the observations tend to cluster. Use the mean when the data are normally distributed (symmetric). Mean number of cases Very sensitive to variation

Median number of confirmed malaria cases Represents the middle of the ordered sample data For odd sample size, the median is the middle value For even, the median is the midpoint/mean of the two middle values Median number of confirmed malaria cases Month Cases 2008 2009 Dec 29 24 Jan 30 May 37 32 Mar 38 35 Jun 40 39 April 41 Feb 45 42 Jul 70 65 Nov 100 80 Oct 200 150 Aug 270 Sep 280 - Median for 2008 Median for 2009 Not sensitive to variation

Mode number of confirmed malaria cases Value that occurs most frequently It is the least useful (and least used) of the three measures of central tendency Mode number of confirmed malaria cases Month Cases 2008 2009 Dec 29 24 Jan 30 May 37 32 Mar 38 35 Jun 40 39 April 41 Feb 45 42 Jul 70 65 Nov 100 80 Oct 200 150 Aug 270 Sep 280 - Mode for 2008 Mode for 2009

Annual Parasite Incidence (API) Number of microscopically confirmed malaria cases detected during one year per unit population Confirmed malaria cases during 1 year API X 1,000 Population under surveillance For example: To calculate the API, you can divide the number of confirmed malaria cases by the total number of people under surveillance.

Has the Program Met its Goal? Our target is to have 80% of children under age 5 sleep under an ITN every night. Have we met our goal? How can you tell? Facilitator note: Wait for a participant response before giving the answer. No, the goal has not been met. Country 3 is doing the best but has reached only a little more than half of the goal for ITN.

Interpreting Data Does the indicator meet the target? What is the programmatic relevance of the finding? What are the potential reasons for the finding? What other data should be reviewed to understand the finding (triangulation)? How does it compare (trends, group differences)? Conduct further analysis. When interpreting data, we may ask these questions: What is the relevance of the unmet target for the program? Is it because we are not meeting our coverage or efficiency goals? Is our quality of care poor? What could be causing this? How are we doing in comparison with other clinics? Districts? (Analyze secular trends, poor data quality, program success, etc.)

Practical Question: Data Source: Are ANC clinics in country X reaching their coverage targets for IPTp? Data Source: Routine health information Now we are going to consider how we could answer the following question: Are ANC clinics reaching their coverage targets for testing and counseling?

Data Source General ANC Registers Which of these variables are relevant for answering your question? Have you defined the use of each relevant variable? Code Variables 1. New ANC clients 2. Group pre-test counseled 3. Individual pre-test counseled 4. Accepted HIV test 5A. HIV test result – Positive 5B. HIV test result – Negative 5C. HIV test result – Indeterminate 6 A. Post-test counseled – Positive 6 B. Post-test counseled – Negative 8A. ARV therapy received – Current NVP 9. IPTp-1 10. IPTp-2 Answers: 1) New ANC clients, IPTp-1 2) New ANC clients = Denominator, IPTp-1 and IPTp-2 = Numerator Now we are going to consider how we could answer the following question: Are ANC clinics reaching their coverage targets for testing and counseling?

IPTp Coverage – Facility Performance Number of ANC clients receiving IPTp Code Variables Facility 1 Facility 2 Facility 3 Facility 4 Facility 5 9. IPTp-1 536 1435 39 969 862 10. IPTp-2 372 542 38 452 780 Question: Among the five facilities, which one performed better? So, we’re going to focus on these elements… Answer: Cannot tell because we don’t know the denominators

IPTp Coverage – Facility Performance Number of ANC clients receiving IPTp Code Variables Facility 1 Facility 2 Facility 3 Facility 4 Facility 5 1 New ANC Clients 744 2708 105 1077 908 9. IPTp-1 536 1435 39 969 862 10. IPTp-2 372 542 38 452 780 Question: Now that you have the denominators, which facility performed better? Indicator Facility 1 Facility 2 Facility 3 Facility 4 Facility 5 % of new ANC clients who receive IPTp-1 in the past year 72% 53% 37% 90% 95% % of new ANC clients who receive IPTp-2 in the past year 50% 20% 36% 42% 86% So, we’re going to focus on these elements… Response: Facility 5

Are facilities reaching coverage targets? Let’s assume that the national coverage target for pregnant women receiving IPTp is 80%. Are the facilities reaching the coverage target? What else can we interpret from this information? Facility 1 needs to do a better job following up and should increase IPTp coverage a bit. Facility 2 does a better job with IPTp-1 coverage than IPTp-2 but needs to increase coverage of both. Facility 3 does a good job administering IPTp-2 to patients who receive the first round but needs to increase initial coverage and maintain follow-up. Facility 4 does a good job with IPTp-1 coverage, but this falls off with IPTp-2. Is this loss due to follow-up, or is the facility not administering IPTp-2 when patients return? Facility 5 can be seen as a model, and we could investigate their best practices for use in other programs. This information does not tell you why coverage is at these levels. You would have to investigate further, but you can see which facilities you need to work with.

Data dissemination and presentation

Number of malaria cases (n) Relative frequency (%) Tables Percentage contribution of reported malaria cases, by year (2000–2007), Kenya Year Number of malaria cases (n) Relative frequency (%) 2000 4 216 531 8 2001 3 262 931 6 2002 3 319 339 7 2003 5 338 008 10 2004 7 545 541 15 2005 9 181 224 18 2006 8 926 058 17 2007 9 610 691 19 Total 51 400 323 100.0 In this table, we already have the total number of observations (or n) in the second column but we should add a title and the source of the data. To analyze this table, we should look at the relative frequencies. What do they tell us? There is an increasing trend in the number of reported malaria cases and in the relative frequency of cases. Does this mean that there is an increase in malaria cases? What would this say about our programs? It is important to take into account what we already know when interpreting these data. We know that since 2000 there has been an increased effort toward malaria control. During this time period, the quality of treatment has improved, as has the quality of routine information systems. When taking this knowledge into account, how would we interpret these data? From 2000 to 2007, the number of reported malaria cases increased. This may not reflect an actual increase in cases, but rather an increase in care seeking and reporting. Due to improved case outcomes seen after the introduction of ACTs in Kenya in 2004, individuals with fever began to seek care at formal medical facilities at higher rates. Furthermore, the routine information system improved during this period of time and thus reported more complete information. Source: WHO, World Malaria Report 2009

Bar chart In this bar chart, we’re comparing the categories of data (any net or ITN). What should be added to this chart to provide the reader with more information? On the next slide, we see how the graph has been improved and is now self-explanatory.

Bar chart Source: Quarterly Country Summaries, 2008 Label the y-axis (vertical axis) and add data labels (numbers). Label the x-axis (horizontal axis), the source of the data. Label the title. Put in a target (optional). Label data values (optional). By adding a title, we know the population to which the graph is referring. By labeling the y-axis, we know that the values are percentages rather than absolute numbers. Adding the source of the data (x-axis) lets us know the data’s derivation and where to find the data. To interpret this chart, we should look at several things, such as the target, the utilization coverage, the trend over time, and the mean number of enrollees. The target is to test 50% of new enrollees at each site in each quarter. What is the utilization coverage (% of the target population utilizing services)? What is the trend over time? Note, we can’t calculate the mean because we don’t know the number of enrollees for each site and percentages cannot be averaged. Source: Quarterly Country Summaries, 2008

Stacked bar chart A stacked bar chart is often used to compare multiple values when the values on the chart represent durations or portions of an incomplete whole, such as the percentage of children taking each type of medication for fever when not all children received medication at all. What should be added to this chart to provide the reader with more information?

Stacked bar chart % Children <5 with Fever who Took Specific Anti-Malarial, 2007–2008 Add a title and data labels. You could also add the source of the data but it isn’t necessary if all of your tables and graphs are derived from the same source/dataset.

Histogram

Line graph Number of Clinicians* Working in Each Clinic During Years 1-4, Country Y Add a label to the y-axis, a title, and a footnote. In some settings, ‘clinicians’ may mean only doctors; to be clear, the footnote lets the reader know we’re referring to both doctors and nurses in this case. *Includes doctors and nurses.

Caution: Line graph Number of Clinicians* Working in Each Clinic During Years 1-4, Country Y *Includes doctors and nurses.

Pie chart A pie chart displays the contribution of each value to a total. In this type of chart, the values always add up to 100. What should be added to this chart to provide the reader with more information? What should be changed about this chart to make it more readable?

Pie chart A pie chart displays the contribution of each value to a total. In this case, we used the chart to show the contribution of each quarter to the entire year. For example, the first quarter contributed the largest percentage of enrolled patients. To improve the understanding of the pie chart, we’ve added a more descriptive title and value labels. On the previous chart, we couldn’t tell if the values are numbers or percentages. Adding the sample size lets us know the total number of observations. It is also important to have charts that are attractive, easy to look at, and easy to read. The chart on the previous page was so colorful that it was distracting, the colors were so bright that it was hard to look at the chart, let alone read it. While these colors are not the most interesting, they let the reader focus on the chart and not how ugly it is. The last chart was an exaggeration, but make sure you do not make the same mistake on a smaller level. Caution about using pie charts with a scientific audience. N=257

How should you present… Prevalence of malaria in Ghana over a 30-year period? Data comparing prevalence of malaria in 10 different countries? Data on reasons why individuals are not using ITNs (out of all individuals surveyed who own an ITN and are not using it)? Distribution of patients tested for malaria by parasite density? Line graph Bar chart Pie chart Histogram

MEASURE Evaluation is a MEASURE project funded by the U.S. Agency for International Development and implemented by the Carolina Population Center at the University of North Carolina at Chapel Hill in partnership with Futures Group International, ICF Macro, John Snow, Inc., Management Sciences for Health, and Tulane University. Views expressed in this presentation do not necessarily reflect the views of USAID or the U.S. Government. MEASURE Evaluation is the USAID Global Health Bureau's primary vehicle for supporting improvements in monitoring and evaluation in population, health and nutrition worldwide.