Introduction to Monitoring and Evaluation

Objectives of session Definitions: Monitoring and Evaluation Rationale for M&E Review elements of an M&E plan Steps toward constructing the M&E plan Implementing the M&E plan

Monitoring - Evaluation Evaluation is the use of specific study designs and special studies to measure the extent to which changes in desired health outcomes are attributable to a program’s interventions. Monitoring is the routine process of data collection and measurement of progress toward program objectives. Interventions Outcome

Rationale for M&E M&E help to make informed decisions regarding ongoing programs They facilitate effective and efficient use of resources They determine whether a program is right on track and where changes need to be considered M&E help stakeholders conclude whether the program is a success M&E preserve institutional memory

Three Primary Uses of Evaluation Findings Rendering judgments Summative evaluations of program’s overall effectiveness e.g., audit, renewal, quality control, accreditation Facilitating improvements Formative evaluation to improve program e.g., program’s strengths/weaknesses, progress Generating knowledge Conceptual use of findings e.g., generalization, theory building

What is an M&E Plan? An M&E Plan is a document that describes a system which links strategic information obtained from various data collection systems to decisions that will improve health programs. Fundamental document to ensure: Accountability Measure of success Speaker Notes The M&E plan describes the strategic information your program will gather and use for decision making that will lead to improved health programs and ultimately to improved health status. It is also the fundamental document that will hold the program accountable and tell you whether you succeeded or not.

Functions of an M&E Plan State how the program is going to measure what it has achieved (ensure accountability) Document consensus (encourage transparency and responsibility) Guide M&E implementation (standardization and coordination) Preserve institutional memory M&E plan is a living document and needs to be adjusted when a program is modified. Speaker Notes In addition to stating how the program is going to measure what it has achieved, an M&E plan functions to document consensus, guide implementation, and preserve institutional memory. Remember that an M&E plan is a living document and needs to be adjusted when a program is modified or new information is needed.

Functions of an M&E Plan: by level of the health system Facility (primary data collection): Manage patients/clients Supervise providers Manage logistics Report to district HIS and vertical programs and community District: Allocate resources Report to provincial/national National (Comprehensive vs. Vertical Programs) Collect national data Report to international donors and other stakeholders

Elements of M&E Plan Brief project description Purpose(s) of M&E plan Brief history of M&E plan development Evaluation framework Indicator system Information system (data sources) Impact evaluation design Dissemination and utilization plan Adjustments to M&E plan

1. Brief project description Include: objectives/results implementation responsibility dates/duration geographic focus resources

2. Purpose(s) of M&E plan Specify purpose for: performance (process) monitoring impact evaluation

3. Brief history of M&E plan development Helps to define: shared vision (internal/external) source and extent of interest/commitment

4. Develop Evaluation Framework (Log Frame, Results, Conceptual) Show: Primary components Levels and units of action Causal sequence Component-related indicator(s)

5. Develop Indicator system Organized by framework component Attributes Name of indicator Description/definition Unit of measurement Data source (primary/secondary) Baseline/target value(s) by year(s) Frequency of data collection Responsibility Reporting plan and frequency

6. Information Systems (Data sources) Develop: Information systems for collecting data for indicators Diagram data collection, processing, analysis, and reporting system Data collection tools (routine vs. non- routine) patient records registers survey instruments

7. Select impact evaluation design Decide on design: Experimental vs. observational Quasi-experimental (post-test, pre- and post-test with no control) Implementation plan Execution responsibility (internal/external)

8. Dissemination and utilization plan Decide on: Databases for information storage Clearly defined users Schedule and timeliness Dissemination methods Reports (schedule and audience) Media Speaking events Information-utilization models

9. Adjustments to M&E plan Program changes can occur, affecting the original M&E plan for both performance monitoring and impact evaluation M&E plan is a living document and needs to be adjusted when a program is modified.

Steps toward constructing the M&E plan Preparing to build the plan Constructing the evaluation framework Identifying potential data sources Selecting and defining indicators and their values Designing the impact evaluation Developing a reporting schedule and system Planning for mid-course adjustments

In implementing the M&E plan Utility - serve practical information needs of intended users Feasibility - be realistic, prudent, diplomatic and economical Propriety - conducted legally, ethically, and with regard to those involved in and affected by the evaluation Accuracy - reveal and convey technically accurate information In implementing the M&E plan think of the following important issues: Utility - serve practical information needs of intended users. Feasibility - be realistic, prudent, diplomatic, and economical. Propriety – conduct M&E legally, ethically, and with regard to those involved in and affected by the evaluation. Accuracy - reveal and convey technically accurate information. Propriety- conduct M&E in a respectful and honest manner.