Monitoring & Evaluation (M&E) 1

What is Monitoring? A continuing function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives in the use of allocated funds. It allows to continuously improve the program at each stage 2

What is Evaluation? Evaluation is judgement: Made of the relevance, appropriateness, effectiveness, efficiency, impact and sustainability of development efforts Based on agreed criteria and benchmarks among key partners and stakeholders Evaluation: Involves a rigorous, systematic and objective process in the design, analysis and interpretation of information to answer specific questions Provides assessments of what works and why Highlights intended and unintended results and provides strategic lessons to guide decision-makers and inform stakeholders It is the periodic assessment about the impact/outcome and effectiveness of the project 3

Introduction to M&E System A good M&E system will help: Clarify what impact the project is expected to have Decide how progress and impact will be assessed Gather and analyse the necessary information for tracking progress and impact Explain the reasons for success and failure; agree on how to use this information to improve future actions 4 Contd…

Introduction to M&E System Effective M&E can: Provide managers with information needed for day-to-day decisions Provide key stakeholders with information to guide the project strategy Provide early warnings of problems Help empower primary stakeholders, especially beneficiaries, and involve them more Build understanding and capacity amongst those involved Assess progress and so build accountability In summary a good effective M&E system is a tool for managing impact 5 Contd…

Introduction to M&E System Evaluators of projects frequently have five standard questions that need to be answered. The M&E system should thus be designed to provide answers to those questions Relevance: Did the project address priority problems faced by the target areas and communities Was the project consistent with policies of both donors and recipient governments (or agencies) Effectiveness: ● Have activities, outputs and outcomes been achieved 6 Contd…

Introduction to M&E System Efficiency: Were inputs (staff, time, money, equipment) used in the best possible way to achieve outputs; could implementation been improved/was there a better way of doing things Impact: What has been the contribution of the project been to the higher level development goals; did the project have any negative or unforeseen consequences Sustainability: Have the necessary systems been put in place to ensure the project itself and more particularly, will the project benefits continue once the project and its (foreign) funding has ended 7

What is an Indicator? Indicators are realistic and measurable criteria of project progress. They should be defined before the project starts, and allow us to monitor or evaluate whether a project does what it said it would do Example Proportion of IDUs on NSEP = Number of IDUs on NSEP/ number of population contracted to the TI x 100 8

Types of Indicators Output indicators show that the project or activity has successfully taken place and that the pre-conditions are in place to achieve the outcomes/objectives of the program Output indicators measure activities directly realized within programs These activities are the first step towards realizing the operational objectives of the intervention and are measured in physical or monetary units Example No. of IDUs on NSEP, No. of IDUs on OST, No. of condoms distributed/socially marketed 9

Developing Monitoring System Developing monitoring systems involves: Identifying the desired objective (result) Establishing baselines for these objectives from which progress can be measured Developing indicators (which serve both to interpret the objectives as well as measure progress) Identifying monitoring information sources, systems and frequency of data collection 10 Contd…

Developing Monitoring System Establishing responsibility both for data collection and oversight of monitoring process Establishing regular feedback mechanisms on lessons learned/good practices, and updating interventions in the light of this feedback Noting underlying assumptions and potential risks that highlight potential obstacles to achieving the desired objectives and that can contribute to collating lessons learned 11

Monitoring at PM’s Level At NGO level (on regular basis) PM should ensure that: Monthly field visit plan is prepared, based on the monthly meeting for PM, ORW, PE All the project-based registers are available at the project office and updated regularly IDU master list, event register, advocacy register, condom register, medicine stock register is updated Network analysis is available for every site for tracking the number of IDUs 12 Contd…

Monitoring at PM’s Level Agendas for weekly and monthly meetings are circulated before conducting the meetings and meetings are documented N/S, condom and drug procurement procedures are followed for TI Data collected is analysed for tracking the performance of project progress Scheduled training programs for the TI for the month are completed as per plan 13 Contd…

Monitoring at PM’s Level At Outreach/Site level, ensure that: IDU: Peer ratio of 40:1 is maintained All the PEs are of same age group as IDUs in a given site ORWs fill in registers within 24 hours of the activity conducted ORWs have filled in the ‘IDU registration form’ for the new IDUs identified in the area Sufficient IEC/BCC material is available with ORW for distribution at site level Once a week, interact with selected IDUs (8-10 members) to know the program inputs and quality services given Visit referral centres to know status on patients referred with regard to those availing services at referral centres 14 Contd…

Monitoring at PM’s Level At Clinic level: Observe the doctor’s approach to patients (if he/she is friendly and follows SCM guidelines) Check clinic patients records to ensure they are as per prescribed format and procedures are followed Check whether the eligibility criteria for initiating OST has been followed Monitor the dosage of OST Ensure that the NACO protocol of waste management is being followed Ensure counsellor maintains confidentiality and proper documentation of counselling as per prescribed formats 15 Contd…

Monitoring at PM’s Level Ensure that the consumables/stock are available and no-stock out of medicines is reported Stock registers are updated on a daily basis Ensure weekly meeting with the peers, to plan the week ahead and review the weekly achievement Develop weekly plan of activities for each PE Prepare weekly movement plan for self and share with PM Meet all the PEs at site level and provide field- level support Check daily diaries and types of services being given during every visit 16 Contd…

Monitoring at PM’s Level Ensure the PEs are conducting activities as per weekly activity plan Ensure all the new IDUs identified by the PE during the month have been registered with the program Collate the site-level information on weekly basis from the PE daily diaries Take feedback from the community regarding the project service Ensure that the PEs distribute condom and N/S to the community as per their requirement Gather information about the field challenges faced by the PEs and resolve the issues 17 Contd…

Monitoring at PM’s Level Ensure that sufficient IEC/BCC materials are available with each PE Give feedback on the performance, based on the field observations during weekly and monthly meeting Conduct group meetings at each site during the month Conduct mid-event activities as per plan during the month Support Project Director in conducting advocacy meeting Contact all the listed IDUs at the site regularly with project service Identify new IDUs and ensure he/she is registered under the site Distribute N/S to each IDU and ensure that adequate no are distributed as per the requirement 18 Contd…

Monitoring at PM’s Level Distribute condoms to each IDU and ensure enough condoms are distributed as per requirement Report incidents of violence at the site level to the ORW Ensure that the IDUs on OST are not dropped out from the service Ensure that the IDUs with STI complaints visit the clinic for treatment Ensure all the IDUs have been tested at ICTC twice in a year Accompany the IDUs for testing at ICTC Ensure all the IDUs have been screened for Syphilis twice in a year 19 Contd…

Monitoring at PM’s Level Ensure all the IDUs, who are under ART, are taking regular treatment at the ART centre Ensure that the referred IDU is taking treatment at TB/DOT centre 20

Conducting Participatory Review Involve the community in the review Involve team members, donors and other stakeholders Provide them information about project’s goal, objectives, performance indicators, targets and achievement Discuss qualitative impact of the program Identify gaps in the program Discuss the solutions Prepare an action plan 21

Conclusion Having a good monitoring system is very important for the success of any program PM should regularly monitor the work of all staff members and periodically review the achievements of the program Feedback from the review should then be incorporated in program implementation 22

23