1. Unit 10. Monitoring and evaluation
Photo from:
WHO. Revised TB recording and reporting forms, Version WHO/HTM/TB/
TB Infection Control Training for Managers at the National and Subnational Levels

2. Objectives After this unit, the participant will be able to:
Define monitoring and how it is essential to supervision
List two indicators to use in monitoring infection control at the national level, and two for use at the facility level
[Review slide]

3. Outline Definitions: monitoring vs. evaluation
Types of indicators: input, output, outcome and impact
Choice of indicators
Examples of indicators at global, national and facility levels
Setting targets
[Review slide]

4. Develop the plan (to include goals and objectives)
Lifecycle of IC plan
Develop the plan (to include goals and objectives)
Implement
This is the same diagram we discussed in the Managerial unit.
Like other good plans, the IC plan has goals and objectives.
As a plan is being implemented, it will need to be fine-tuned as the situation changes.
Monitoring helps the manager decide what revisions are needed in the implementation, so objectives are met.
Monitor,
evaluate
Revise

5. Monitoring is….
The routine tracking and reporting of priority information about a programme or project, its inputs and intended outputs, outcomes, and impact
[Review slide]
We’ll define inputs, outputs, outcomes and impact in a few minutes.
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit-2009 version.

6. Monitoring
Uses data obtained through record keeping, regular reporting, surveillance systems, observation or surveys
Helps managers determine which areas require greater effort
Is essential for supervision
Is also used to measure trends over time, so methods need to be consistent to ensure appropriate comparison
[Review slide]
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

7. Evaluation is…
The rigorous, scientifically-based collection of information about the program activities, characteristics, and outcomes that determine the merit or worth of the programme
[Review slide]
Citation:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

8. Evaluation studies
Provide credible information for use in improving programmes, identifying lessons learned, and information decisions about future resource allocation
[Review slide]
[Ask participants:]
Can you name some ways that monitoring and evaluation are different?
[Possible responses]:
The main purpose of monitoring is to help managers routinely monitor progress and make needed corrections during programme implementation.
By contrast, evaluations are done to determine the worth of a program.
While monitoring is regular and ongoing, Evaluation studies are done less frequently.
They also use different methods. Monitoring uses routinely collected information.
Evaluation requires a special study (although it should leverage any data or surveys are are regularly undertaken).
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

9. M&E framework long term, cumulative effect of the programme over time
Impact
long term, cumulative effect of the programme over time
Outcome
short and medium term effects of the outputs
Output
products or deliverables of the programme’s interventions
We’ll work our way from the bottom to the top of this framework.
For a programme to achieve its goals and objectives, inputs (financial, human and material resources) must result in outputs (products or deliverables)
If the outputs are achieved, the programme will have a positive outcome, which is a short and medium term effect.
These outcomes should have a long term impact.
In this monitoring and evaluation (M&E) framework, the blue boxes are linked (yellow arrows) in a chain of results.
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
Diagram adapted from: Dominique Mathiot
Input
financial, human and material resources

10. Example of a results chain
Impact
Fewer health care workers with TB
Outcome
Coughing patients are rapidly identified, separated, and served
Output
HIV clinics have written triage procedures for coughing patients
Our example is a country which has identified HIV clinics as high risk settings for TB transmission.
Let’s work our way from top to bottom of the diagram; we’ll follow the chain of results first using the white boxes.
One of the overall goals of the country’s IC plan is to decrease the spread of TB in order to protect health care workers and other patients.
To have this impact, the country wants to be sure that coughing patients are rapidly identified, separated, and served.
This will require written procedures be in place for coughing patients, which in turn requires there be an IC focal person identified and trained in each facility.
We can check our logic by working our way back up the results chain by following the yellow arrows from bottom to top.
We want to be sure that inputs, lead to outputs, which lead to outcomes and finally the desired impact.
Starting at the bottom are the resources needed. One input is human resources: IC focal point staff identified in each HIV clinic. They participate in an IC training, including a workshop to finalize a national template for a clinic triage policy.
The output is written triage procedures in each clinic. The trained IC focal point staff take the national template, adapt it to their settings, and put in place a written triage policy specific to their clinic.
The short term outcome should be that coughing patients are identified, separated, and rapidly served.
Over the long run, the impact will be less TB spread in the clinics. Fewer health care workers will have TB.
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
Diagram adapted from: Dominique Mathiot
Input
Infection Control focal point staff identified and trained

11. Indicators Provide useful tools for supervision at all levels
Are we doing the right things?
Are we doing them right?
Help managers and supervisors determine:
What’s going well and should be reinforced and replicated elsewhere?
What needs improvement and how can the gaps be addressed?
[Review slide]
Citation
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

12. Choose indicators that:
Are able to measure performance
Provide strategic information needed to make good decisions for managing and improving programme performance
Are reliable and measurable on a regular basis
Are clear, with well defined numerators and denominators using standard definitions
[Review slide]
The next few slides present examples of indicators.
You can chose those that are the most useful for supervision of the IC programme at each level.
Citation
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01

13. Standardized IC indicators for collaborative TB/HIV activities
Proportion of health-care facilities providing services for people living with HIV that have infection control practices that include TB control
Proportion of health-care workers, employed in facilities providing care for people living with HIV, who developed TB during the reporting period
WHO recently developed standardized, internationally accepted indicators for monitoring TB/HIV collaborative activities. Here are the two that address IC.
While specific to facilities providing care for people living with HIV, they can be applied to any facility with a risk of TB transmission.
The first is the proportion of health facilities that have IC practices.
The second is the proportion of health-care workers who developed TB.
The citation in the Instructor’s notes [see below] provides definitions for these 2 indicators, suggests how to collect the information, and assesses the strengths and limits of these indicators.
Citation
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/

14. Global IC indicators and targets
By 2012, 50% of countries should have
Developed a national TB infection control plan
Set up national surveillance of TB disease among health workers
Assessed major health-care facilities and congregate settings for TB IC
Reported on the implementation of TB infection control
By 2013, all countries
As we discussed in the introduction to the class, here are global IC indicators in the 2009 WHO Policy on Infection Control.
The targets are for 50% of countries to achieve them by 2012, and all countries by 2013.
[Review slide]
Citation
WHO. WHO policy on TB infection control in health-care facilities, congregate settings, and households WHO/HTM/TB/

15. Examples of national IC indicators
How many health care workers had TB in the past year?
Have health care workers been trained in the past year?
Of the country’s tertiary (referral) hospitals, how many have:
A person in charge of TB infection control?
A TB IC assessment done in the past year?
Training on TB IC conducted in the past year?
Here are examples of infection control indicators for the national level.
These are part of WHO’s TB data collection form countries completed in 2009.
[Review slide]

16. Examples of facility level IC indicators
Facility level plan in place
Person responsible for TB IC
Health facility assessment done
Training conducted
Triage and separation of TB patients
# TB cases or suspects missed at intake
Adequate ventilation and/or UVGI
Use of respirators by HCWs in high risk settings
[Review slide]
“Missed at intake” means that TB symptoms or history were detected only after the patient enters a private room with a clinician, or after numerous visits while symptomatic, instead of upon first entry into the facility.
Citation
WHO. TB infection-control in the era of expanding HIV care and treatment. Addendum to WHO Guidelines for the prevention of TB in health are facilities in resource-limited settings

17. Facility-level indicators
Impact
Number of health care workers with TB
Outcome
Number of TB cases or suspects missed at intake to the facility
Output
Facility-specific, written triage procedure exists
Let’s go back to our example.
Here are some indicators that could be used to during a supervisory visit to a district clinic to monitor the implementation of this part of the country’s IC plan.
[Review slide]
[Ask participants]
Which of these indicators may also be useful for a national manager to assess the country-wide implementation of this part of the national IC plan?
[Possible responses]
Number of facilities with IC focal persons
Number of facilities with written triage procedures to identify and separate coughing patients
Number of health care workers with TB
Citations:
WHO. A guide to monitoring and evaluation for collaborative TB/HIV activities WHO/HTM/TB/ , WHO/HTM/HIV/09.01
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.
Diagram adapted from: Dominique Mathiot
Input
Infection control focal person is identified and trained

18. Setting targets Gap analysis: baseline compared to need not yet met
Constraints to meeting the need
Feasibility for scaling up
Setting ambitious yet realistic targets is an important element of the planning process.
Good targets are based on a comprehensive and up to date analysis of the situation.
A gap analysis compares the baseline to the need.
Setting targets also requires consideration of constraints to meeting the need, and feasibility for scale up.
Citation
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

19. Example of setting a target
By 2015, __ of the country’s HIV clinics will have a written policy for identification and separation of coughing patients
Gap analysis: baseline is 0, need is 100%
Constraints: no template policy, limited training capacity, few IC focal persons in the HIV clinics
Feasibility: resources available to address the constraints and fill the gap
Let’s work through an example.
You have chosen this outcome indicator (in bold), and need to set the target (to fill in the blank).
When you conduct your gap analysis, you find that none of the HIV clinics have a policy, but all need them.
You face significant constraints in getting there: there is no sample policy in your country the clinics can use, there are few IC focal persons identified in the HIV clinics, and you have limited capacity to train them on adopting the policy.
To set the target, you think through how well you can address these constraints by 2015, and thus, how much of a scale up is feasible.
You’ll judge for your own country what is the appropriate target.
Citation
Global Fund to Fights AIDS, TB and Malaria. Monitoring and evaluation toolkit, 2009 version.

20. Summary
Monitoring is the routine tracking and reporting of priority information about a programme
Use global indicators to allow measurement of impact across countries
Chose additional indicators that provide strategic information needed to make good decisions for managing and improving programme performance
Adapt standardized IC indicators to your country context, and use them for supervision at each level
[Review slide]