1. Monitoring and Evaluation
A Trade Union Training on Occupational Health and Safety and HIV/AIDS (Inter-regional Course: Africa and Caribbean) (Turin, 27 September to 8 October 2010)
Monitoring and Evaluation
International Training Centre of the ILO

2. Objective of the Session
To provide information that will enable participants to monitor and evaluate HIV/AIDS programmes in the workplace.
To identify indicators that will help participants in monitoring and evaluating HIV/AIDS programmes in the workplace.
International Training Centre of the ILO

3. International Training Centre of the ILO
What is Monitoring?
Monitoring is the systematic and continuous assessment of a programme over a period of time.
On-going monitoring allows us to see how and where the programme must change.
Monitoring entails routine tracking of the key elements of programme/project performance, usually inputs and outputs, through record-keeping, regular reporting and surveillance systems as well as health facility observation and client surveys.
International Training Centre of the ILO

4. International Training Centre of the ILO
What is Evaluation?
Evaluation is the episodic assessment of the change in targeted results that can be attributed to the programme or project/project intervention.
Evaluation attempts to link a particular output or outcome directly to an intervention after a period of time has passed.
Evaluation entails measuring the impact of an intervention programme to see if it has an effect or not.
Evaluation is usually conducted after the programme has been in existence for some time.
International Training Centre of the ILO

5. Why are monitoring and evaluation important?
Ascertain the appropriateness and effectiveness of the programme.
Assess how well the programme is running.
Assess the effect of the programme on the workplace.
Understand the effectiveness of the programme in changing people’s behaviour with regards to HIV/AIDS and STIs.
International Training Centre of the ILO

6. Two aspects to be monitored and evaluated
The impact of HIV/AIDS in the workplace.
The effectiveness and impact of the HIV/AIDS and STD programme.
Monitor
Baseline Information Evaluate
HIV/AIDS Programme
Report Feedback
International Training Centre of the ILO

7. Questions to address in monitoring and evaluation
Has the programme changed behaviour, reduced the prevalence of HIV/AIDS and STIs and minimized or eradicated discrimination against people living with HIV/AIDS?
What is the prevalence of HIV infection among workers?
How appropriate and effective is the HIV/AIDS workplace programme?
What is the efficiency and progress of the HIV/AIDS workplace programme?
International Training Centre of the ILO

8. International Training Centre of the ILO
What is an Indicator?
An indicator is a direct or indirect measure of change.
Behaviour change with regard to safer sex practices is the change that is of ultimate interest in an HIV/AIDS and STI programme.
Indirect indicators:
Prevalence of STIs and the number of condoms distributed: indicate behaviour change with regard to safer sex practices.
Number of employees volunteering to become peer educators: indicate levels of awareness and motivation around HIV/AIDS and STIs in the workplace and is an indirect indicator of behaviour change.
International Training Centre of the ILO

9. International Training Centre of the ILO
What is an Indicator?
Direct indicator
Number of condoms distributed: indicate the effectiveness of an education programme on the use of condoms.
Number of employees attending and participating in HIV/AIDS and STI education programme: indicate whether the programme is reaching those targeted.
International Training Centre of the ILO

10. Examples on Indicators
Establishment of HIV/AIDS working committee and/or structures.
Adoption of an HIV/AIDS workplace policy.
Number of workers who are aware of the policy.
Number of trained peer educator/counsellors.
Number of workshops conducted by peer educators.
International Training Centre of the ILO

11. Examples on Indicators
Number of workshops conducted by peer educators.
Number of support groups in the workplace.
Number of workers participating in support groups.
Number of HIV/AIDS-related deaths and burials.
International Training Centre of the ILO

12. Examples on Indicators
Absenteeism due to HIV/AIDS.
People’s attitudes/perceptions/behaviours.
Number of claims for occupational exposure to HIV.
Number of medical claims for HIV-related illnesses.
Number of times the Management talks about HIV/AIDS as a workplace issue.
International Training Centre of the ILO

13. Examples on Indicators
Number of collective bargaining agreements with progressive clauses that protect the rights of people living with HIV.
Changes in the budget for HIV/AIDS.
What else?
International Training Centre of the ILO

14. Key aspects to identify indicators
Valid – they should measure the condition or event they are intended to measure.
Reliable – they should produce the same results when used more than once to measure the same condition or event.
Specific – they should measure only the condition or event they are intended to measure.
Sensitive – they should reflect changes in the state of the condition or event under observation.
International Training Centre of the ILO

15. Key aspects to identify indicators
Operational – it should be possible to measure or quantify them with developed and tested definitions and reference standards.
Affordable – the costs of measuring the indicators also should be reasonable.
Feasible – it should be possible to carry out the proposed data collection.
International Training Centre of the ILO

16. Monitoring the impact of HIV/AIDS in the workplace
Maintaining records on sickness or absence and looking at the causes of the absence.
How often employees take special or compassionate leave.
Staff turn-over by reason for departure.
Death by cause.
Decide which indicators you will measure (how will behaviour change be measured?).
International Training Centre of the ILO

17. Monitoring the impact of HIV/AIDS in the workplace
Decide how the indicators will be measured e.g. in-depth interviews or use of questionnaires.
Calculate the costs of the monitoring process.
How often will monitoring take place to give the maximum amount of information with the least disruption and costs?
International Training Centre of the ILO

18. Developing an integrated and comprehensive evaluation plan
Identify programme goals and objectives.
Examine existing data and past evaluation studies (baseline).
Identify internal and external evaluation resources and capacity.
Determine evaluation questions, their feasibility and appropriate designs and indicators.
Plan for disseminating and using evaluation findings.
International Training Centre of the ILO

19. Examples of process measures
Participants:
- Number, demographic information, frequency of attendance, turn-over rate, etc…
Services that are provided:
- Classes, programmes, workshops, publications, other services, communications (number, type, content and frequency).
International Training Centre of the ILO

20. Examples of outcome measures
Changes in policies – new or modified policies.
Changes in collective bargaining agreements.
Changes in practices – new or modified practice.
Changes in programmes – new or modified service programmes.
Changes in behaviours – new or modified behaviours.
International Training Centre of the ILO

21. Main Outcomes Expected
Accountability
Learning Build Capacities Revise
from programmes
experiences as needed
Make more informed decisions
International Training Centre of the ILO

22. International Training Centre of the ILO
Summary
Impact of the HIV/AIDS and STI epidemic in the workplace must be monitored.
Effectiveness and impact of the HIV/AIDS and STI programme on the epidemic must be monitored and evaluated.
Baseline information on key indicators must be collected during the need analysis component of the policy and programme development process.
Subsequent indicator information can then be compared with the baseline information to assess the impact of the epidemic and the prevention programme.
International Training Centre of the ILO