1. Monitoring and Evaluation
PISA INSET Training
Corporate slide master
With guidelines for corporate presentations
Module 4
Monitoring and Evaluation 1
Module 3 - An Introduction to Thinking Skills 1

2. The project cycle ASSESSMENT EVALUATION Monitoring PROGRAMME DESIGN
Disaster
The project cycle
ASSESSMENT
EVALUATION
Monitoring
PROGRAMME DESIGN
IMPLEMENTATION
source: IFRC 3Ps/SPHERE
18-Sep-18

3. Definition Monitoring It is NOT only about PROCESS
‘The systematic and continuous assessment of the
progress of a piece of work over time….’
‘To continuously measure progress against programme
objectives and check on relevance of the programme’
It involves collecting and analysing data/information
It is NOT only about PROCESS
There are various definitions but they all aim at the same: we measure progress and changes need to be made if progress is not made. Obviously ‘progress’ has all to do with what our programme originally aimed for, i.e. the objectives. Are we in the process of reaching them? If not, we need to make adjustments to ensure we reach these objectives eventually.
‘The systematic and continuous assessment of the progress of a piece of work over time….It is a basic and universal management tool for identifying the strengths and weaknesses in a programme. Its purpose is to help all the people involved make appropriate and timely decisions that will improve the quality of the work.’
Gosling and Edwards, 1995 cited in ALNAP Review of Humanitarian Action 2003 *

4. Purpose of monitoring to document progress and results of project
to provide the necessary information to Management for timely decision taking and corrective action (if necessary)
to promote accountability* to all stakeholders of a project (to beneficiaries, donors, etc)
Source: MTV for UCL + Makerere University School of Public Health, Uganda
*accountable = being responsible to someone for some action
Monitoring and evaluation, though two distinct activities are very closely linked. Monitoring is a routine activity with data collected on a regular e.g. daily or monthly basis. Its basic purpose is to keep track of programme activities and improve the efficiency of interventions. Evaluation tends to be episodic, undertaken at critical points in a project cycle and its basic purpose is more to do with improving effectiveness and informing future programming. Monitoring data provides essential inputs into more episodic evaluation. Monitoring data may highlight specific issues in the programme’s implementation that require deeper investigation through evaluation to be resolved. In turn, evaluation can help to identify what needs to be monitored in the future. In a well designed M&E system, data routinely collected through monitoring activities can contribute greatly towards evaluation *

5. Information collected for monitoring
must be:
Useful and relevant
Accurate
Regular
Acted upon
Shared
Timely
Source: MTV for UCL + Makerere University School of Public Health, Uganda *

6. Definitions Evaluation It involves the objective assessment of an
The aim is to determine relevance and fulfilment of
objectives, as well as efficiency, effectiveness, impact
and sustainability of a project.
It involves the objective assessment of an
ongoing or completed project/programme, its
design, implementation and results.
These are all terms we will look at more in detail.
Evaluations are also very important to identify LESSONS LEARNT *

7. Evaluations are done when / because:
Monitoring highlights unexpected results
More information is needed for decision making
Implementation problems or unmet needs are identified
Issues of sustainability, cost effectiveness or relevance arise
Recommendations for actions to improve performance are needed
Lessons learning are necessary for future activities
Additionally, the donor (often from one or various Governments) that funds the project is accountable to its tax payers, and the donor wants to know what exactly is done with the money and whether the objectives were reached. And if not, why not.

8. To evaluate projects well is a real skill! And you often need a team…
Source: UCL + Makerere University School of Public Health, Uganda *

9. M&E in emergencies?
YES
Any project without Monitoring and/or Evaluation is a BAD project
Source: MTV for UCL + Makerere University School of Public Health, Uganda *

10. Examples of data collection methods for M&E
Quantitative Methods
Qualitative methods
Administering structured oral or written interviews with closed questions
Semi structured interviews e.g. key informant
Population based surveys
Focus group discussion
Reviewing medical and financial records
Observing
Completing forms and tally sheets
Case studies
Direct measurement (anthropometry, biochemical analysis, clinical signs)
Mapping, ranking, scoring
Lot quality assessment
Problem sorting, ranking

11. Indicators
An indicator is a measure that is used to show change in a situation, or the progress in/results of an activity, project, or programme.
Indicators:
enable us to be “watchdogs”;
are essential instruments for monitoring and evaluation.
are objectively verifiable measurements
Source: UCL + Makerere University School of Public Health, Uganda

12. What are the Qualities of a Good Indicator?
Specific
Measurable
Achievable
Relevant
Time-bound
The Sphere Project provides the most
accepted indicators for nutrition and
food security interventions in emergencies: see Module 21.
Source: MTV for UCL + Makerere University School of Public Health, Uganda
What are the characteristics of a good indicator?
A good indicator is SMART:
S for SPECIFIC:
By specific, we mean that it specifies the magnitude of the attribute that we’re measuring in a particular time frame and for a particular population
In other words, that they measure what they’re supposed to measure… blood retinol measures vitamin A status (not iron status, not any other vitamin…)
Can you think of another word starting with “S” that describes a good indicator? Good! it’s SIMPLE: clearly and precisely defined
M for Measurable:
Measurable indicators are objective: we are able to quantify them
You can measure the temperature, you can count how many correct answers students get on a test, you can observe if someone is washing their hands before they handle food
Can you think of other examples of indicators and how to measure them?
A for Achievable:
Achievable indicators means that we can actually obtain them: you need to have the resources and capacities to collect, store and process the information obtained from that indicator
R is for relevant:
Relevant to you project, your situation
And T is for Time bound:
Always stating the time period in which that indicator was measured
Beginning and end of projects are commonly used time-marks
Seasons and seasonality too: amount of grain produced in the month of July, total sales in the pre-holidays season, malnutrition rates in the lean (or hungry) season preceding the harvest.
Can you think of other examples of indicators that are time-bound?
SMART Initiative[1]
The Standardised Monitoring and Assessment in Relief and Transition (SMART) Initiative - is an interagency initiative, begun in 2002, to improve the M&E of humanitarian assistance interventions through:
• The development of standardised methodologies for determining comparative needs based on nutritional status, mortality rate and food security.
• Establishing comprehensive, collaborative systems to ensure reliable data is used for decision-making and reporting
A Standardised Training Package (STP) for SMART methodology has recently been released.
[1] More information available at
And there is also the SMART initiative….
Standardised Monitoring and Assessment in Relief and Transition
Initiative - interagency initiative to improve the M&E of humanitarian
assistance

13. Types of indicators Examples?
Indicators exist in many different forms:
Examples?
Direct
Direct indicators correspond precisely to results at any performance level.
Indirect or "proxy" indicators demonstrate the change or results if direct measures are not feasible.
Indirect / proxy
Indicators are usually quantitative measures, expressed as percentage or share, as a rate, etc.
Indicators may also be qualitative observations.
Qualitative
Quantitative
Source: UCL + Makerere University School of Public Health, Uganda
Direct - Number of children with acute malnutrition
Proxy – diet composition for nutritional status
Global / standardised
Standardised global indicators are comparable in all settings.
Other indicators tend to be context specific and must be developed locally.
Locally developed

14. Impact Outcome Output Input
Source: UCL + Makerere University School of Public Health, Uganda
Input

15. Impact Outcome Output Input Related to Goal
Related to Objectives (or Purposes)
Output
Related to Outputs
Source: UCL + Makerere University School of Public Health, Uganda
Input
Related to Activities/Resources

16. Impact Outcome Output Input * Malnutrition rates amongst young
children reduced
Related to Goal
Outcome
% of young children
getting appropriate
complementary food
Related to Objectives (or Purposes)
X number of mothers
know about good
complementary food and
how to prepare that
Output
Related to Outputs
Source: UCL + Makerere University School of Public Health, Uganda
Nutritional education to mothers on complementary food
Input
Related to Activities/Resources *

17. What is a Log Frame?
The logical framework or logframe is an analytical tool
used to plan, monitor, and evaluate projects. ? ? ?
The logical framework or logframe derives its name from the logical linkages set out by the planner(s) to connect a
project’s means with its ends. ?
Victim of a log frame?

18. Log Frames IMPACT OUTCOME INPUTS
Log frames appear in different shapes and with different names sometimes, but overall they look like this.
INPUTS

19. PART 1: What’s the issue you want to solve?
Using this space, please respond to the following: - Write about your team’s topic. - Explain why your team selected this topic. - Explain what issue you will address in your action plan.

20. PART 2: What’s your action plan?
Using this space, please respond to the following:
What are the step-by-step actions your team will take to address the environmental issue?
List the specific responsibilities of each team member toward achieving your plan.
How will your team measure success?
How will you quantify impact? (e.g. tallying how many gallons of water were saved, by tracking how many cans were recycled.)

21. PART 2: What’s your action plan? (continued)

22. PART 3: How did you implement your ideas?
Using this space, please respond to the following:
What did your team do?
Did you organize a school event or a community wide event to raise awareness of the issue? Tell us about it.
Did you work with any local environmental groups, governmental agencies, schools, etc? Tell us about it.
What did you do to get the word out about your project? (e.g. create a Web site, TeacherTube video, press release, write a blog, etc.)
Please provide a list of any internet links to your website, online videos, blog etc.

23. PART 3: How did you implement your ideas? (continued)

24. PART 4: Evaluate your plan. What are the results?
Using this space, please respond to the following:
In what ways was your plan successful?
What challenges did you face?
How did you raise awareness in you school communicate as well as the greater community?
What were the quantifiable results?
What did you learn from this experience?

25. PART 4: Evaluate your plan. What are the results? (continued)