2. Monitoring & Evaluation: The concepts and meaning Day 9 Session 1

3. Session objectives By the end of the this session the CEO’s will:  explain the meaning of monitoring and evaluation  discuss the purpose of monitoring and evaluation

4. A range of approaches to Monitoring and Evaluation in development work What is useful? What isn’t? What approaches /tools/frameworks are appropriate in different contexts?

5. Experiences of monitoring evaluation What is your experience of monitoring and evaluation as a Municipality Officer? CEO? Deputy CEO? Head of department?

6. Rate the following statements Agree/Disagree/Note Sure Monitoring and evaluation (M & E) are one and the same Agree/Disagree/Note Sure Monitoring can be separated from evaluation Agree/Disagree/Note Sure M & E is the responsibility of QUASO’S Agree/Disagree/Note Sure M & E is the responsibility of school heads Agree/Disagree/Note Sure M & E is linked to strategic planning

7. Some context… trends and changes in evaluation over the last 30 years.. A changing field of practice Marco Segone (1998) Stages of M&E Objective of M&EFocus of M&E First generation 1950s-1970s Measurement by comparison (before & after – and control) Measuring results Second generation 1980s Transparency/ Accountability to supervisors Measuring results Third generationUnderstanding/ learning/decision making/positive accountability (360 degree) Results/process/ utilization of learning from M&E

8. Essential changes in evaluation practice? Moving fromCurrently Evaluation more of an audit function. If you used the `tools’ you’ll be able to `measure’ changes. Evaluation more of a learning function – aim to improve the quality of work. In the 1980s a concern with `attribution’ – trying to measure your schools ‘ contribution. Less emphasis on attribution, more on questions of whether you have the right `strategy’ or not. 1990s. An emphasis on indicators. Which were to be monitored. Not fixated on indicators. Trying to get a holistic look at what is changing.

9. Moving fromCurrently Assuming a linear relationship between cause and effect. Input-output-outcome-impact More emphasis on looking at the wider learning. The intended as well as the unintended. Evaluation largely done by technocrats and seen as a science Evaluation seen more as a learning tool. An art. Emphasis on understanding process. Few techniques and toolsA plethora of techniques and tools – and more tools, approaches & frameworks developing all the time… Essential changes in evaluation practice? Cont..

10. Monitoring Monitoring is the regular observation of activities taking place in a school/ institution It is a process of systematically gathering information on all aspects of the school/ institution It is systematic and purposeful observation Monitoring is part and parcel of all good strategic planning

11. The purpose of monitoring Monitoring provides information useful in: Analyzing the situation in the school/ institution Determining whether the inputs are well utilized Identifying problems facing the school/ institution and finding solutions

12. The purpose of monitoring Cont... Monitoring provides information useful in: Ensuring all activities are carried out properly by the right people and in time Using lessons from one experience on to another Determining whether the way something planned (e.g. a building project or a reading project) is the most appropriate way of solving the problem at hand.

13. Evaluation Evaluation is a process of determining the value of what has been achieved in relation to activities planned and overall objectives It involves value judgments (failed or succeeded? Done or not done? Improved or did not improve; can read or cannot read? ) and hence it is different from monitoring which is the collection of information.

14. Purpose of Evaluation Evaluation enables schools to identify the constraints or bottlenecks that hinder the institution from achieving its objectives. Evaluation answers the question “what have we achieved?”

15. Approaches to M & E ? Logical framework analysis Results based management Outcome mapping Participatory monitoring & impact assessment Programme logic Most significant change (tool)

16. 1) Planning - knowing where you (& your partners & the people with whom you are working) are going? 2) Devising how will you (and they) get there? 3) Assessing progress along your path 4) Identifying, how will you (and they) know when you’ve arrived? We do this all the time.. Approaches are essentially for?

17. Types of evaluation formative (taking place during the life of a project or an institution, with the intention of improving the strategy or way of functioning of the project or institution). summative (drawing learning from a completed project or an institution that is no longer functioning). Summative evaluation is the difference between a check-up and an autopsy!

18. M & E is essentially a way of dealing with the richness/diversity of the development experience & synthesizing information into forms we can understand. Not necessary for everyone to know the different approaches But is necessary for each institution to consciously understand what approach (or set of approaches) it is using and – very importantly – to consciously evolve its own approach which is in line with its institutional values.

19. The evaluation planning triangle Specified evaluation instruments match to performance indicators Performance indicators matching objectives Specific agreed objectives Evaluation Planning Triangle

20. Group activity Ministry Officials group: Is it necessary to monitor and evaluate curriculum implementation in your schools? Explain why. Head Teacher group: Is it necessary to monitor and evaluate curriculum implementation in your schools? Explain why. Deputy Head Teacher and Senior Teachers group: Is it necessary to monitor and evaluate the implementation of a school development plan that involves the physical expansion of the school? Explain why.

21. 20

22. Monitoring and Evaluation Monitoring and evaluation helps you to learn from what you are doing and how you are doing it, by focusing on: – Efficiency – Effectiveness – Efficacy (Impact)

23. Efficiency Efficiency tells you that the input into the work is appropriate in terms of the output. This could be input in terms of money, time, staff, equipment and so on. When you are concerned replicability [doing the same thing elsewhere], then it is very important to get the efficiency element right.

24. Effectiveness Effectiveness is a measure of the extent to which a development programme or project achieves the specific objectives it set. For example, we set out to improve leadership skills and knowledge of 22 Head Teachers, 1 Deputy Head Teacher, 2 Ministry officials and 2 Head of departments in Mombasa Municipality - did we succeed? If yes, we were effective. If no, we were ineffective. But, how can we know?

25. Efficacy (Impact) Impact tells you whether or not what you did made a difference to the problem situation you were trying to address. –In other words, was your strategy useful? –Did ensuring that Ministry officials, Head teachers, Deputy head teacher and Head of departments were trained improve the pass rate in the final year of school? Improve the quality of leadership in our schools? Before you decide to replicate something elsewhere, you need to be sure that what you are doing makes sense in terms of the impact you want to achieve.

26. The cycle of M&E Evaluate/ learn/ decide PlanImplement Monitor Reflect/ learn/ decide/ adjust Implement Monitor Reflect/ learn/ decide/ adjust Implement

27. Benefits of Monitoring and Evaluation Monitoring and evaluation (M&E) of development activities provides Head teachers, teachers with better means of:  learning from past experience  improving service delivery  planning and allocating resources  demonstrating results as part of accountability to key stakeholders

28. Benefits of Monitoring and Evaluation cont.. Within organizations like schools there is a strong focus on results so M & E is an important way of ensuring: 1.results 2.quality and effectiveness 3.progress against targets 4.achievement of goals 5.consistency in delivery

29. Session objectives By the end of the this session the CEO’s will:  explain the meaning of monitoring and evaluation  discuss the purpose of monitoring and evaluation