Download presentation
Presentation is loading. Please wait.
1
SOCIAL SCIENCES &TECHNOLOGY
INTRODUCTION to M&E Lecturer D. S. BOWASI ATD; BED. MAED EXECUTIVE DEAN SCHOOL OF EDUCATION, SOCIAL SCIENCES &TECHNOLOGY
2
19 February 2015 University of LUSAKA Bpa 450 D.S. BOWASI (DEAN – SEST
Introduction to M&E 19 February 2015 University of LUSAKA Bpa 450 D.S. BOWASI (DEAN – SEST
3
Session Objectives Define Monitoring Define Evaluation Purposes of M&E
Describe the relationship between monitoring and evaluation Define Program Components Identify the different types of M&E
4
Learning Objectives By the end of this session you will be able to:
Define key M&E terms Identify components of programs to evaluate Describe different purposes for M&E State why Monitoring & Evaluation are important in programming Describe characteristics of good M&E system
5
Defining Programs, Projects & Interventions
Program: organized effort to respond to a broad social problem (typically organized at national level) Project: specific set of activities with linked objectives, that contribute to the overall objective of a program (typically organized at sub-national level) Intervention: often used in same was as “project”, sometimes a specific sub-set of program or project activities
6
Program Planning Involves Setting…
Goals & Objectives based on intended Impact Intended Outcomes Intended Outputs or Deliverables Planned Activities (Processes) Inputs or Resources
7
Inputs What you need to implement the program
e.g. the financial , human and material resources used to implement an HIV counseling and testing program Trained personnel Protocols and guidelines Training materials HIV test kits and other supplies Money
8
Activities What the program provides to accomplish its objectives
Training workshops on for personnel and site managers Providing pre-test and post-test counseling to clients Curriculum development Recording & reporting Dissemination of IEC materials Supervision
9
Outputs What the program delivers Condoms distributed
Clients receiving pre-test counseling, HIV tests, post-test counseling Materials distributed People reached Intervention sessions completed
10
Outcomes The results of the program or changes that occur both immediately or some time after activities are completed Changes in knowledge, behavior, attitudes and skills. Quality of personnel and managers improved. . Clients develop and adhere to personalized risk-reduction and treatment strategy
11
Impact The wider effect of the program on long-term results
HIV transmission rates decrease HIV incidence decreases Changes in HIV mortality and morbidity Decrease in deaths due to HIV-related TB
12
Example: Components of an HIV program
Programme level Population level Input Process Output Outcome Impact Financial resources Staff Supplies Capacity building BCC interventions Community mobilization Policy development Improved skills of providers Increased knowledge Mobilized communities Sound policies available Reduced risk behaviors Declining HIV/AIDS morbidity and mortality Economic growth
13
So how does this relate to Back to our definitions…
M&E? Back to our definitions…
14
Set of procedures & analytical tools to examine
What Do We Mean by M&E? Set of procedures & analytical tools to examine how programs are conducted (inputs & activities) their level of performance (outputs) whether they achieved what they were intended to achieve (outcomes & impact) Types of evaluation monitoring (including process evaluation) evaluation (outcome and impact evaluation)
15
Evaluation: What have we achieved?
Monitoring: What are we doing? Tracking inputs and outputs to assess whether programs are performing according to plans (e.g., people trained, condoms distributed) Evaluation: What have we achieved? Assessment of impact of the programme on behaviour or health outcome (e.g., condom use at last risky sex, HIV prevalence)
16
What is Monitoring? Is a continuous systematic process of collecting, analyzing and using information to track the efficiency of achieving program goals and objectives Provides regular feedback that measures change over time in any of the program components such as costs, personnel and program implementation An unexpected change in monitoring data may trigger the need for a more formal evaluation of activities
17
Key M&E Questions Is the program being implemented as planned?
Are things moving in the right direction? Did the program achieve its objectives? Can results be attributed to program efforts? Which program activities were more (or less) important/effective? Did the target population benefit from the program? At what cost?
18
What is Evaluation? Is a systematic process limited in time of collecting, analyzing and using information to assess the effectiveness, relevance and impact of achieving your program’s goals. Requires study design; sometimes a control or comparison group; often measurement over time. Often involves measuring changes in knowledge, attitudes, behaviors, skills, community norms, utilization of health services, & health status at population level Provides regular feedback that helps programs analyze the consequences, outcomes and results of its actions
19
What is Evaluation? (con’t)
Rigorous research design need for: Establishing a causal link between program effort and desired outcomes Isolating program effect from other, non- program influences on the outcome of interest Not undertaken routinely; usually reserved for specific situations, such as determining the success of a project for scale-up or replication
20
Outcome Evaluation Time With program Attribution “Impact”
Program Indicator: Without program Program Start Program End Time
21
Impact Evaluation Time With program Attribution “Impact”
Program Indicator: Fertility or HIV prevalence Without program Program Start Program End Time
22
Illustration of Program Evaluation/ Impact
With program Change in program outcome Without program Program impact Program start Program end TIME
23
Comparison of Potentially Confusing Terms
Outcome Impact Monitoring Simple Tracking of: Results that occur both immediately and some time after the activities are completed, such as changes in knowledge, attitudes, beliefs, skills, behaviors, access, policies, and environmental conditions. Clients reduced risk to HIV infection Increased prevalence of immediate breastfeeding in hospitals Decrease in newborn hypothermia Increased use of modern methods of contraception Simple Tracking (over multi-year periods) of: Long-term results of one or more programs over time, such as changes in morbidity, and mortality HIV prevalence TB incidence Maternal morbidity Infant mortality Evaluation Rigorous Scientific Analysis (including attribution of effects): of: Program outcomes (same definition as above) Note: Most people consider this to be “Impact Evaluation” Program impacts (same definition as above)
24
Key M&E Questions Did the program achieve its objectives?
Did the target population benefit from the program? At what cost? Can improved health outcomes be attributed to program efforts? Which program activities were more (or less) important/effective? What would have happened in the absence of the program? How can we know or measure this (the counterfactual)?
25
Why Monitor & Evaluate? To make decisions about project management and service delivery To ensure effective and efficient use of resources and provide accountability to donors To assess whether the project has achieved its objectives - has the desired effects To learn from our activities, and provide information to design future projects Programme Improvement Share with Partners Reporting/ Accountability
26
Purposes of Monitoring and Evaluation
Determine whether a plan or program is on schedule with planned activities Assess whether a policy, plan or program has produced desired impacts Generate knowledge: Identify factors (individual, community, programmatic) that influence health outcomes Help inform policy, planning or program decisions: new services, resource allocation, corrections, etc.
27
What Decisions are Guided by M&E Results
How much funding should be allocated to a program What types of programs should be funded Which programs approaches should be presented as models New strategic objectives, activities or results packages Replication and scaling up of successful
28
Reasons to Monitor & Evaluate Different Needs for Different Stakeholders: 1. Funding Agencies & Policy Makers What M&E Measures Evidence of achievement of program objective Program outcome and impact Program cost-efficiency Data about the target population What M&E Results Identify Priorities for strategic planning Programs that qualify for donor assistance Best practices Impact of donor assistance
29
Reasons to Monitor & Evaluate- Different Needs for Different Stakeholders: 2. Communities & Youth
What M&E Measures Youth behavior related to reproductive health Young peoples’ needs How program funs are being spent The process and impact of community participation What M&E Results Identify Actual and potential benefits of youth programs Need for new and better youth services Community resources that can be used to support ARH programs Need for local support for ARH issues and actions
30
2. Communities & Youth Cont…
What Decisions are Guided by M&E Results Degree to which community members and youth should participate in and support the program How to better coordinate community actions to address identified programs How many and what type of local resources should be allocated to identified programs
31
3. Programme Managers & Staff
What M&E Measures Quality of activities and/or services Why some sites are more successful Program coverage What M&E Results Identify Priorities for strategic planning Training and supervision needs How to improve reporting to funding agency Feedback from clients Why program is not accomplishing what it set out to do
32
Replication and scaling up of intervention Fund-raising
What Decisions are Guided by M&E Results (Programme Managers and Staff). Resource allocation Replication and scaling up of intervention Fund-raising Motivating staff Policy advocacy
33
What decisions are Guided by M&E Results
Agreement on the scope and objectives of M&E plan with stakeholders Selection of Indicators Systematic and consistent collection of information on the selected indicators Analyze the information gathered Compare results with program initial goals and objectives Share results with stakeholders
34
Defining the Scope of M&E Effort
Scope refers to the extent of the activity you will undertake in a M&E effort. Scope is determined by several factors (questions) What should be monitored and evaluated? When should health programs be monitored and evaluated? How much will M&E cost? Who should be involved in M&E? Who should carry out the evaluation? Where should M&E take place?
35
What Should be Monitored & Evaluated?
M&E can measure each stage of your program development: design, system development and functioning, and implementation. After goals, objectives and activities are developed, decision on about M&E at each stage is needed. M&E effort can measure each stage to determine how the program is working and its impact on the target population.
36
What Should be Monitored & Evaluated?
Program Design is measured by process evaluation: Assessing how well the program has been designed Documenting the success/challenges with program design System development and functioning is measured by monitoring and process evaluation: Document the development of support systems and determine if they are actually operating once program implementation begins Assess the performance of support systems Measure how effective the preparatory activities are in readying program personal for program implementation Implementation is measured by monitoring, process evaluation, and outcome/impact evaluation: Reveal how program implementation is occurring Determine whether program is achieving its objectives by measuring the changes in outcomes in your target population
37
When Should Programs be Monitored & Evaluated?
Monitoring and process evaluation should occur throughout the life of a program Outcome and impact evaluations are usually done near the end of a program (baseline gathered to measure change) Starting M&E at the beginning of a program is ideal Some activities can still be measured if M&E is started in the middle of a program Even Fewer activities can be measured if M&E is started towards the end of the program
38
When Should a Program be Monitored & Evaluated?
Stage of Monitoring Process Outcome/Impact Program Early - Set up monitoring system; identify indicators and tools, plan for tracking program, data analysis, and reporting. - Assess systems development and functioning. Provide early feedback. Assess if program is responsive to target. - Identify objectives and indicators. Take baseline measurements. Crate an outcome or impact evaluation plan. Middle - Assess MIS and data. Modify original system is inadequate. If program is not performing, launch process evaluation. - Conduct more formal midterm process evaluation to assess quality of program performance. Determine coverage, or whether program is reaching target. - Take mid-term measurements. Analyze short term outcome measures, such as changes in knowledge, increase in use of programs and changes in contextual factors. Provide feedback to program. Late - Analyze data from tracking system to conclude if you conducted the program as planned. Prepare and submit reports. - Analyze end of program measurements. Determine what was done to improve quality of program implementation. Make recommendations for program replication or expansion. - Take end of program measurements. Examine evidence of changes in outcomes. Depending on study design conduct impact analysis to conclude whether outcomes are attributable to program activities. Report to donor and other stakeholders.
39
Where are the levers in a working health systems
6 building blocks of health systems strengthening Service delivery Health workforce INFORMATION Medical products ,Vaccines and technologies Financing Leadership and governance WHO: Nellie Bristol www. globalhealthmagazine.com
40
Reference M&E Websites
MEASURE Evaluation: Health Metrics Network: John Snow Inc.: HIV Global Partners:
41
END OF PRESENTATION
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.