Monitoring and Evaluation Module 12 – March 2010

Project Partners Funded by the Health Resources and Services Administration (HRSA)

Module Overview  Monitoring & Evaluation (M&E) framework and components  Records, registers and reports  Recording and evaluating response to TB treatment regimens  Supervision International Standards 13 and 21

Learning Objectives At the end of this presentation, participants will be able to:  Describe what is meant by “Monitoring and Evaluation”  Discuss the importance of collecting data and ensuring the accuracy of the data  Explain ways in which the data are used to evaluate treatment  Describe how M&E activities can benefit both TB and HIV/AIDS programs

Monitoring & Evaluation System  A key element of the Stop TB Strategy  Allows programs to: Monitor progress and treatment outcomes of individual patients Evaluate the overall performance of the TB program at various levels (local, district, national) Identify areas of program improvement and weakness Ensure accountability

Monitoring  What is it? Routine tracking of services and program performance

Monitoring (2)  How is it done? Through information collection, data input, analyzing the data, and reporting what is found in that analysis outcome report  Why should we do it? To better assess how well a policy or program is achieving its intended target

Evaluation  What is it? Episodic assessment of results that can be attributed to program activities Types of evaluation related to M & E: ➜ Process evaluation: assesses the progress in program implementation and coverage ➜ Outcome and impact evaluation: measures the effect of the program activity on the target population

What are the Targets?  Stop TB Partnership/WHO 70% TB case detection and 85% cure rate by 2005  Millennium Development Goals (MDG): Halt, and begin to reverse, the incidence of major diseases such as HIV/AIDS and TB by 2015  Decrease TB prevalence and death rates to 50% of the 2000 estimates  United Nations General Assembly Special Session (UNGASS) – global targets

Where do Indicators Fit In?  Indicator: a specific, observable, and measurable characteristic or change that shows the progress a program is making toward achieving a specific outcome  Indicators may be expressed in terms of: Number Rate Proportion Percentage

Limitations of Indicators Indicators DO NOT:  Measure everything  Tell us why a problem may exist or how to fix it  Determine if problems identified are amenable to intervention  Tell us which interventions are most cost effective

What are some possible uses of data collected by the National HIV/AIDS and TB Programs?

Using and Disseminating Data M & E can improve and enhance NAP and NTP work by:  Identifying areas of strengths and weaknesses  Helping plot progress toward program goals  Allowing a program to see trends and to identify high risk groups in order to better target TB control efforts  Providing justification for needed resources  Identifying training and supervision needs  Increasing public awareness about TB  Advocating for policy changes and allocation of funds

Monitoring & Evaluation Framework

Types of M & E Activities  Staff training  Supervision  Completion of reporting forms  Discussions during staff meetings  Ensuring medicine and laboratory stock supplies  Quality control activities  Analysing data and preparing reports

Standard 13: Recording & Reporting A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients

Standard 21: Recording & Reporting All providers must report both new and re- treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies

Why is accurate reporting and record keeping important?

Data Quality Assurance  Ensures that the information collected adequately represents the program’s activities  Accurate data – measuring what it is intended to measure  Reliable data – collected and measured the same way by all program personnel over time

Reporting Forms and Registers  Request for Sputum Examination  Tuberculosis Treatment Card  Tuberculosis Identity Card  Basic Management Unit TB Register  TB Laboratory Register  Quarterly Report on TB Case Registration  Quarterly Report on Sputum Conversion  Register of TB Suspects  Quarterly Report on Treatment Outcomes  Yearly Report on Program Management in Basic Management Unit

Tuberculosis Treatment Card

Tuberculosis Treatment Card (2) (RH) (RHE)(Other)

Preparing a TB Treatment Card Activity

Tuberculosis Identity Card  This card contains an extract of information on the treatment card  It is given to the patient at the start of treatment  It is used to record daily DOT and must be used during the intensive phase of treatment  It also serves as a reference document for TB status after treatment  It should be presented to the doctor whenever the patient falls ill in the future

Tuberculosis Identity Card (2)

Basic Management Unit TB Register  This revised register is the cornerstone of an NTPs monitoring & evaluation system  It records essential information for notification & treatment outcome by district  It should always be kept up to date with data on sputum smear examinations and treatment outcome  Where electronic data collection systems are available, the information from the register should be entered into the database at least once every month

Treatment Outcomes  Cured  Treatment completed  Treatment failure  Died  Default  Transfer out

Treatment Outcomes CurePatient whose sputum smear or culture was positive at beginning of treatment but who was smear- or culture-negative in the last month of treatment and on at least one previous occasion Treatment Completed Patient who has completed treatment but who does not meet the criteria to be classified as a cure or a failure Treatment Failure Patient who is sputum smear-positive at five months or later during treatment – or – Patient found to harbor a MDR strain at any point of time during treatment, whether smear- negative or -positive

Treatment Outcomes (2) DiedPatient who dies for any reason during the course of treatment DefaultPatient whose treatment was interrupted for 1 month or more Transfer Out Patient who has been transferred to another recording and reporting unit and whose treatment outcome is not known. Treatment Success A sum of cured and completed treatment (smear-positive or culture-positive patients only)

Supervision

How is supervision used in your TB and HIV/AIDS Prevention and Control Programs?

Role of Supervision in M & E  Supervision is a process of guiding, helping, training, and enabling staff to improve their performance in order to provide high quality health care services

Purpose of Supervision  Provide leadership and direction to staff  Ensure effective program implementation  Monitor operations and evaluate achievement of goals  Ensure adherence to laws and policies  Avoid confusion or duplication of efforts

Purpose of Supervision (2)  Monitor that all necessary tasks are properly performed  Ensure that resources are properly used and are available to staff, including training and supplies to carry out their duties  Ensure accountability

Barriers to Effective Supervision  Lack of commitment  Lack of proper planning and time management  Lack of tools for Monitoring & Evaluation  Insufficient staff  Problems with transportation  Lack of confidence or preparation

Roles and Responsibilities  National Level – NTP and NAP Planning, implementing, monitoring, and evaluating Program at all levels  Regional/District/Parish Levels Coordinating, supervising, planning, implementing, monitoring and evaluating all aspects of TB and HIV/AIDS Programs in the region

How Can M&E Information be used in TB and HIV Programs?  Identify gaps in performance  Monitor treatment outcomes  Measure the impact of an intervention or policy change  Identify populations for enhanced control and prevention efforts  Identify local problems as they arise  Ensure high quality TB and HIV prevention and control strategies are consistently provided

Summary: ISTC Standards Covered Standard 13: A written record of all medications given, bacteriologic response, and adverse reactions should be maintained for all patients Standard 21: All providers must report both new and re-treatment tuberculosis cases and their treatment outcomes to local public health authorities, in conformance with applicable legal requirements and policies

Summary  Several approaches are used to monitor and evaluate TB and HIV/AIDS programs including supervision, training and the keeping of records and registers  Accurate and timely reporting and record keeping is important. It allows true assessment of Program achievements