1. Monitoring and Evaluation
Dr. Raghu Bista
Anil Kumar Gupta

2. Outline Basics of M&E framework
Result orientation in practice: Monitoring tools, techniques and issues in selected projects

3. Implementation of Project/Programs
Once preparation is completed, finance is assured and design is finalized, project/program is forwarded for implementation.
During implementation phase, Project/program is put into action. Roads will be contracted and built, project activities will be started and completed, project output will be produced. So this is the investment phase of the project. Here, inputs are converted into outputs

4. Outputs of Project/Programs Implementation
The output of the implementation phase is the new road, new hydropower capacity, new bridge, new hospital department, new irrigation system, trained manpower, enhanced capacity, new product etc.

5. Project Implementation
To create expected outputs, projects are divided into work packages and activities, activity time and activity cost are estimated, time schedules are prepared. Likewise budget, resources and procurement plan are efficiently utilized

6. Project Control
During project/program implementation, control is management function to corrective action to address gap between and actual and planned performance. Clearly, the need to exert proper control mandates the need to monitor the proper activities and elements of the project

7. The Project Control Cycle
1. Setting a Goal
4. Taking Corrective Actions
2. Measuring
Progress
3. Comparing Actual
With Planned

8. Monitoring
“Monitoring is the collection, recording, and reporting of project information that is of importance to the project manager and other relevant stakeholders”
(Mantel, Meridith, Shafer, Sutton & Gopalan, 2005).
Monitoring is a continuous function that uses systematic collection of data on specified indicators to provide management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds.
(World Bank, 2011)

9. Purpose of Monitoring
To indicate at the earliest instance any shortcomings with regard to achieving intended objectives so that measures can be undertaken in good time
To monitor the development of the project as a whole, and its component projects, in relation to changes in the context and circumstances of their implementation.
To implement a rapid problem identification system as well as a system for internal communications to the various stakeholders.

10. Purpose of Monitoring Cont..
To provide forecasts on cost,
To forecast resources and schedule
To facilitate evaluation process during and after activities, through the definition of specific indicators.

11. Process of Monitoring Identification of Indicators
Collection of information
Analysis of information
Use of information for Action

12. Work Schedule and Monitoring

13. Project S-Curve as Tool for Cost Monitoring
Project S curve consists of presenting costs against time for both budgeted and actual expenditure.

14. The Project S-Curve Cumulative Cost ($ in thousands)
Elapsed Time (in weeks) 10 5 45 40 35 30 25 20 15 50 60
$20,000 Negative Var
Cumulative Budgeted Cost
Cumulative Actual Cost

15. Evaluation
Evaluation can be defined as a process by which general judgments about quality, goal attainment, project/program effectiveness, impact and costs can be determined. It can therefore be conceptualized as a review of the whole project/program in order to assess its overall value and effects.
Evaluation is a systematic and purposeful undertaking carried out by internal or external evaluators to appraise relevance, effectiveness, efficiency, impacts as well as sustainability generated by the policies, plans, programmes, and projects under/after implementation.
(National Planning Commission, 2013)

16. Evaluation at Different Level
Evaluation is also classified by subject such as project level, program level, sector Level, policy level.

17. Evaluation Type Ex-ante Evaluation Mid Term Evaluation
Completion Evaluation
Ex-post Evaluation (Impact Evaluation)

18. M & E During Programme/Project Period
(National Planning Commission , 2013)

19. Implementation of projects
Type of Evaluation
Start of Implementation Completion of Implementation 2.
Implementation of projects 4. 1. 3.

20. Ideal situation-M& E Complementary
Monitoring Evaluation

21. Difference Between M & E
Evaluation is assessing the entire project cycle.
Evaluation looks for lessons to be learned from both success and lack of success, and also looks for best practices which can be applied elsewhere.
It is concerned with the assessment of progress against predetermined goals, outcomes and their effects and impacts on stakeholders.
It is generally conducted by an independent outsider
Monitoring is a continuing function that takes place throughout the implementation of a project/programme
It focuses on the implementation of the project comparing what is delivered with what is planned
It is directly related with inputs, processes and outputs.
It is generally conducted by the implementing agency.

22. Result based M& E
Video

23. Result Based Management
Result based management is perspective that focus on achievement of development results.
It is a management strategy that focuses on the processes, products and services contributing to the achievement of development results
(NPC, 2013)

24. How the Results Come
Once a project is designed and implemented, it will produce outputs (For example 25 trained math teachers)
The trained math teachers will use improved methods to teach, the result will be increased math solving competency of students
In exam, number of students to pass exam will be increased with better marks as ultimate results and will be able to get scholarship

25. Results Chain
Impacts
Activities
Inputs
Outputs
Outcomes

27. The Power of Measuring Result
If you do not measure results, you cannot tell success from failure
If you cannot see success, you cannot reward it
If you cannot reward success, you are probably rewarding failure
If you cannot see success, you cannot learn from it
If you cannot recognize failure, you cannot correct it
If you can demonstrate results, you can win public support
(Osborne & Gaebler 1992)

28. Result Base M & E
Results-based monitoring is a process of incessantly collecting and analyzing information on the extent to which results from the implementation of a given plan, policy, programme or project have been achieved so as to compare it with the expectations.
Results-based Evaluation is an undertaking that appraises relevance, efficiency, effectiveness, and sustainability of a given plan, policy, programme or project by making comparison of the outcomes of implementation with the intended results.
(NPC, 2013)

29. Essence of Result Based M & E
Traditional M & E is more focused on completion of assignment/program/policy
Result Based M & E gives emphasis on achievement of ultimate results from the intervention
So it focus on continuous monitoring and evaluation of impacts during project implementation and after project implementation to assure results

30. Typologies of Result Based Monitoring
Continuous
Sustainability

31. Evaluation Criteria Relevancy Effectiveness Efficiencies Impact
Sustainability

32. Impact Evaluation
The objective of this type of evaluation is to determine whether or not a given project is achieving the intended impacts
Impact project performance with training of project staff
Impacts on income of people by apple farming project
Impacts of irrigation on increment of income and reducing poverty

33. Impacts Assessment: Impacts vs. Counterfactual

34. Selection of Indicators of Project Outcomes
Indicators are “the variables used to measure progress towards goals”
(Stem , Margoluis, Salafsky, & Brown, 2005)
Indicators are required to be selected for Monitoring and Evaluating Results

35. Indicators
An indicators indicates progress, success, and failure of a policy, program and project
Indicators are related:
How do I know the intended positive change has taken place?
What changes? In what way? (quantity, quality, time description of targets)

36. Types of Indicators I Input indicators Process indicators
Output/outcome indicators
Impact indicators
Time related indicator
Quantity related indicator
Cost indicators
Quality indicator

37. Steps to be Taken to Select Indicators
Precisely identify what we want to monitor or examine
List down the questions that need answers
Identify the kind of indicators that can be used
Review the methodological basis and robustness of these indicators
Find out what data will be needed to derive each indicator, and where and how to get this data

38. SMART Indicator Examples of Indicators Specific Measurable Achievable
Relevant
Time-bound
Examples of Indicators
Drop out ratio
Enrolment percentage

39. Targets and Baseline Target is the intended results
Baseline is the existing scenario or reference point against which performance is compared
Baseline is the existing scenario

40. M & E Plan
The plan is managerial tool that: that specifies indicators, targets, sources of data, schedules, resources and responsibilities for M & E activities

41. Exercise

42. Methods of Evaluation
Quantitative
Qualitative
Mixed

43. Quantitative Tools
Questionnaire and survey
Existing database

44. Qualitative Direct Observation Focus Group Discussion Interview
Case Studies
Document Studies
Analysis of SWOT

45. Timing of Research
When and How often to collect data necessary to provide required information

46. Data Analysis and Reporting
Once data are collected, they are analyzed or processed for reporting. This may include averaging the data, calculating ratio, developing trends etc. Finally, the analysed data are reported in some form.

47. Essential Elements of Evaluation Report
Executive Summary
Main Body
Outline of evaluation background (Objective, Strategy, Time/Duration, Role/
Introduction, Evaluator/s)
Review of expenditure along with a narrative overview
Evaluation log-frame and descriptive summary
Results of each of the dimensions evaluated : Relevance, Effectiveness, Efficiency, Outcomes and impact and Sustainability
Conclusion
Summary of results, learning and recommendation
Utility of results for management
Justification of evaluation plan
Changes in implementation modality at different points in time

48. Essential Elements of Evaluation Report Contd..
Annexes
Initial outline used for implementation
Evaluation log-frame and matrix demonstrating progress status
Evaluation questions and results analyzed through the measurement of indicators
Unprocessed data obtained from interview, survey, direct communication and other means
Reference