1. Basic Monitoring and Evaluation (M&E) for Control of Malaria in Pregnancy
Module 8
Version 2

2. Learning objectives
By the end of this module, learners will be able to:
Define “community-based information system” (CBIS) and state why such a system is important
Describe the importance of recordkeeping
Define basic M&E concepts and state the differences between monitoring and evaluation*
State why M&E is important in programming
*Health care workers only

3. Learning objectives, cont.
Differentiate between data and information
Describe the qualities of good data
Identify sources of data (facility and community)
Demonstrate how to fill out the community health worker (CHW) register and the referral form

4. What is a CBIS?
A process of collecting information for services that are provided at the community level (i.e., outside of health facilities [HFs])
Requires planning, collecting, managing, reporting, and using information
The process is carried out with some level of community participation

5. Information communities can report to the CBIS
Births and deaths (vital events registration)
Notification of cases of infectious diseases and outbreaks (disease surveillance), for example:
Ebola
Lassa fever
Plague
Identification of pregnant women
Identification of orphans

6. Information the CBIS can collect for community-based malaria programs
Records of:
Home management of malaria for children and adults
Rapid diagnostic tests (RDTs)
Services concerned with malaria in pregnancy, for example:
Antenatal care (ANC)
Community-directed intermittent preventive treatment in pregnancy (c-IPTp)
Home-based prevention of malaria, such as:
Indoor residual spraying
Use of mosquito repellents
Use of insecticide-treated bed nets or long-lasting insecticidal nets (ITNs/LLINs) 5

7. Information the CBIS can collect for community-based malaria programs, cont.
Referrals for:
Case management of complicated malaria and other illnesses
ANC services
Adverse reactions to drugs 6

8. Remember, “If it is not in writing, it does not exist!”
What is recordkeeping?
Recordkeeping is a process of documenting services provided over a period of time using specific formats designed for the purpose
Remember, “If it is not in writing, it does not exist!”

9. Reasons why recordkeeping is important
The need to know how many people are provided specific services over a period of time, for example:
How many pregnant women received doses of intermittent preventive treatment in pregnancy (IPTp)?
How many pregnant women attended ANC?
How many pregnant women were referred for complicated malaria?
The need to know how the program is performing in HFs, for example:
What proportion of HFs provide ITNs/LLINs at ANC?
What proportion of HFs provide IPTp with sulfadoxine-pyrimethamine?

10. Reasons why recordkeeping is important, cont.
For planning purposes, for example:
The quantities of different drugs and other supplies that are required to meet clients’ needs over a specific period of time
Whether the right people are receiving the services
To encourage service providers to work more effectively and be accountable to the community and HFs.

11. What is reporting?
A process of providing a summary of activities carried out over a period of time, for example number of times different services are provided in a given time period.

12. Why is it important to report services provided?
Summarizes accomplishments
Shows progress at a glance
Informs planning
Gives supervisors an easy overview of each CHW’s activities over a period of time
Provides feedback to funders, fund recipients, and program beneficiaries

13. M&E

14. What is M&E?
Monitoring is a process of routine tracking of program activities on a regular basis.
Evaluation is a process of measuring the extent to which program outcomes and output are achieved.

15. What does the chart on the right tell you about the program?
What is monitoring?
Often called process evaluation
A continuous, systematic process of collecting, analyzing, and using information to track the efficiency with which a program is achieving its goals and objectives
Process evaluation measures how well program activities are being performed. This information is sometimes collected on a routine basis, such as through staff reports, but may also be collected periodically in a larger-scale effort (World Health Organization 2004).
What does the chart on the right tell you about the program?

16. What is monitoring? (cont.)
Provides regular feedback that measures change over time in any of the program components such as:
Costs
Personnel
Implementation
An unexpected change observed in monitoring data may trigger the need for a more formal evaluation of activities

17. What are the things to monitor?
Progress of project activities:
Services
Training
Outreach
Meetings
Information, education, and communication activities
Use of resources:
Budget/actual expenditure
Commodities
Equipment
Supplies
Staff
Results:
Achievement of project objectives
Coverage
Outputs
Impact
Effect on project beneficiaries
Institutionalization and sustainability:
Government acceptance of ownership
Government financial inputs
Scale-up plans
Staffing
Changes in policies and procedures
Organizational systems

18. What are the things to monitor at community level?
Stock levels of all the malaria commodities and medicines:
Artemisinin-based combination therapies (ACTs)
Sulfadoxine-pyrimethamine (SP)
RDTs
ITNs/LLINs
Types of ACTs provided

19. What are the things to monitor at community level? (cont.)
Storage facility and condition of commodities, for example:
ACTs SP
RDTs
Training in all aspects of malaria control
Stock levels and use of data capturing tools

20. What is evaluation?
A systematic process, limited in time, of collecting, analyzing, and using information to assess your program’s effectiveness, relevance, and impact in achieving its goals
Often involves measuring population-level changes in knowledge, attitudes, behaviors, skills, community norms, utilization of health services, and health status
Provides regular feedback that helps a program analyze the consequences, outcomes, and results of its actions
Source: World Health Organization 2004

21. What are the things to evaluate?
Effectiveness—Is the project achieving satisfactory progress toward stated objectives?
Efficiency—Are the effects being achieved at an acceptable cost, compared with alternative approaches?
Relevance—Are project objectives still relevant as time passes?
Impact—What difference has the project made?

22. What are the things to evaluate at community level?
Availability and accessibility of malaria commodities and medicines
Use of ITNs/LLINs by the community
Appropriate and effective use of ACTs
Appropriate storage conditions (temperatures) of ACTs and RDTs
Reduction in fever/malaria cases seen in routine reports
Number of referrals
Number of structures adequately treated with indoor residual spraying

23. Brainstorming session!
What are the differences between monitoring and evaluation?

24. Good-quality data

25. What are data?
Facts and statistics collected together for reference or analysis
A set of values of qualitative or quantitative variables
Pieces of data are individual pieces of information
Data are collected, measured, analyzed, and reported
Data can be visualized using graphs or images
Data can be collected at both community and HF levels
Adapted from Padilla 2016

26. Collecting and visualizing HF data
The data use poster (also known as a monitoring wall chart) is used to:
Chart monthly HF data
Visualize performance trends
Facilitate discussion among providers and during supervision on key indicators and project areas, for example:
IPTp third dose (IPTp3)
At least one ANC contact

27. What makes good data? Good data are those that are:
Reliable
Valid
Complete
Timely
Precise
Truthful (have integrity)
Should be easy and cost-effective to collect and sufficiently up-to-date

28. What can reduce data quality?
Unclear data definitions
Poor data collection instrument or tool
Untrained data collectors
People selected for data collector training were poorly chosen
Poorly trained data officers
Unreliable data entry system
Insufficient data checks
Insecure data storage system
Frequent turnover of data officers
Illegible handwriting
Typing or transcription errors

29. What are examples of data sources in your community?
There are routine sources (continuous data collection) and nonroutine sources
The HF:
Routine service statistics
Patient’s/client’s record booklets
Charts
Registers (ANC, delivery, inpatient, outpatient)
The community:
Survey findings
Census
CHW reports
Immunization reports
Vital statistics in registration centers

30. Data are helpful for... The individual client The HFs Communities
The district, subnational, and national health systems

31. Data are necessary to... Adequately monitor health conditions
Provide continuity of care:
Give the correct number of doses of IPTp at the correct time in pregnancy
Communicate effectively among health care providers or among health care sites (if a patient is referred)
Provide information for health planning and budgeting at HF, district, subnational, and national levels

32. Who is responsible for data?
The client first provides accurate information
The volunteers and service providers record this information on the client’s card and in the register
The HF in-charge
M&E officers at district, subnational, national malaria elimination or control program, and ministry of health levels

33. Demonstration: How to fill the data collection tools for c-IPTp
Tools for capturing community-based data:
CHW register (records daily cases, including c-IPTp)
CHW monthly tally sheet
Referral form
Exercises on completing community-based data capturing tools

34. Sample data collection tools CHW register
Name of household member
Sex
M/F
Age
(yrs)
Year (fill in)
Pregnant now? (Y/N)
Age of pregnancy
IPTp1
date given
IPTp2
IPTp3
ITN/LLIN date given
RDT done? (result)
ACT1 date given
ACT2 date given
ACT3 date given
ACT4 date given

35. CHW monthly tally sheet
Service
Tally
Total
ITN/LLIN to pregnant woman
ITN/LLIN to under-5
IPTp1
IPTp2
IPTp3
RDT: Positive
Negative
Indeterminate
ACT: 2 3 4
Referral of pneumonia
Referral of diarrhea

36. Summary
The planning, collection, management, reporting, and use of information are carried out with community participation
Data are needed for the community, HF, district, subnational areas, national malaria elimination or control program, and ministry of health to:
Identify and prioritize health problems
Allocate resources adequately

37. Summary, cont. Referral is crucial for:
Additional ANC services not provided at the community or HF levels, for example:
Hematinics
Tetanus toxoid immunization
Case management of complicated malaria and other illnesses (all cases with fever illness should be referred to HF)

38. Summary, cont.
To collect high-quality, reliable data, we need effective supportive supervision
Establish a reporting, feedback, and data use mechanism, for example:
Monthly graphing of service statistics
CHWs monthly meetings
Malaria focal persons monthly meetings
HF staff weekly meetings

39. References
Padilla E Substation Automation Systems: Design and Implementation. West Sussex, UK: John Wiley & Sons.
World Health Organization (WHO) Compendium of Indicators for Monitoring and Evaluating National Tuberculosis Programs. Geneva, Switzerland: WHO. Accessed October 2, 2018.

40. Thank you!
Any questions or comments?