1. GEF Expanded Constituency Workshop Kigali, rwanda February 2019
Sara El Choufi - Evaluation Analyst
Photo by Karsten Würth

2. Overview Monitoring and Evaluation in the GEF
Independent Evaluation Office (IEO)
M&E Policy
Terminal Evaluation Guidelines
Recent Evaluation Findings
Ongoing Evaluations

3. Is our activity on track?
Monitoring
Is our activity on track?
Monitoring uses systematic collection of data to keep activities on track.
Forms of monitoring:
Monitoring of environmental conditions and stressors
Monitoring of progress toward project/program outcomes
Monitoring of project/program performance

4. Evaluation Are we doing the right thing?
Are we doing things right and efficiently?
Are there better ways of doing it?
Evaluation is a systematic assessment of an activity (program, strategy, etc.) that assesses relevance, effectiveness, efficiency, results and sustainability.
Project/Program Evaluations:
Mid-term
Terminal
Other forms of evaluation: impact, thematic, performance, country, corporate, comprehensive

5. M&E in the GEF Two overarching objectives:
Promote accountability for the achievement of GEF objectives through the assessment of results, effectiveness, processes, and performance of the partners involved in GEF activities.
Promote learning, feedback, and knowledge sharing on results and lessons learned among the GEF and its partners as a basis for decision making on policies, strategies, program management, programs, and projects; and to improve knowledge and performance.

6. Separate reporting lines for Monitoring (through Secretariat) and Evaluation (through IEO)
GEF Council
Annual evaluation reports
Overall Performance Study
(to Assembly)
Annual Work Program and Budget
Annual Monitoring Report
Evaluation Management Response
Programming documents and indicators
Results Based Management
Agency evaluation units
GEF Independent Evaluation Office
GEF Secretariat
Project and Program Implementation Reports
Agency Portfolio Reports
Project documents with M&E plans
Project and Program
evaluations
Corporate evaluations
Project and Program
Independent evaluations
Agency GEF coordination units
Project and Program Implementation Reports
Project and Program monitoring documentation
Terminal evaluations
GEF projects and programs

7. GEF Independent Evaluation Office
Mission:
Enhance global environmental benefits through excellence, independence, and partnership in monitoring and evaluation.
Functions:
Accountability
Promote knowledge sharing and Learning
Brief history:
1996 — Initially established as an M&E unit within the GEF Secretariat
2003 — The M&E unit was made autonomous of the GEF Secretariat
2005 — The unit was renamed as GEF Evaluation Office
2013 — The office was renamed as GEF Independent Evaluation Office
GEF IEO Functions:
Independent Evaluation: The main function of the IEO is to independently evaluate the GEF’s efficiency, effectiveness, relevance, impact, and sustainability.
Normative function: IEO sets minimum monitoring and evaluation standards within the GEF to ensure improved and consistent measurement of the GEF results .
Oversight function: IEO provides quality control for the minimum requirements of monitoring and evaluation practices in the GEF, in full cooperation with relevant units in the GEF Agencies, and tracks implementation of Council decisions related to evaluation recommendations.
Knowledge management and dissemination function: IEO supports knowledge sharing and follow-up of evaluation recommendations. It participates in the development and maintenance of a comprehensive knowledge system based on evaluation findings and lessons.

8. GEF IEO Stakeholders Clients with governance role: GEF Council
GEF Assembly
The replenishment group
Clients that carry out decisions of the governing bodies
GEF Secretariat
GEF Agencies
Executing agencies at the country or regional level
Country Clients
Clients involved in monitoring and evaluation
Wider audience: environmental entities, academia, research institutions, civil society, general public

9. The GEF M&E Policy
Defines the concepts, role, and use of monitoring and evaluation within the GEF
Defines the institutional framework and responsibilities
Indicates the GEF minimum M&E requirements covering:
project design
application of M&E at the project level
project evaluation
engagement of Operational Focal Points in M&E

10. The GEF M&E Policy: 4 Minimum Requirements
1. Design of M&E Plan
Concrete and fully budgeted M&E plan
Align with GEF focal area results frameworks
Baseline data
Mid-term reviews and Terminal Evaluations
Organizational set up and budget
2. Application of M&E Plans
SMART indicators
Baseline for the project is fully established
Organizational set up for M&E is operational and its budget is spent as planned
3. Project and Program Evaluation
All FSPs will be evaluated and evaluations are
Independent
Apply evaluation norms and standards
Assess, outputs and outcomes, likelihood of sustainability
4. Engagement of Operational Focal Points
OFPs should be informed on M&E activities
GEF Agencies keep track of the application of this requirement

11. Forthcoming Updates to the M&E Policy
Separation of M&E policies consistent with GEF’s evolution
Expanded coverage of GEF Priorities based on IEO Gap Analysis (2015) gender, safeguard standards, stakeholder engagement, other
Focus on programmatic approaches requires attention to program-level M&E (OPS6)
Evolution: GEF formally introduced M&E in the TORs in 2003, highlighting the independent nature of these functions.
The M&E Policy followed in 2006 and was updated in 2010, to reflect the evolution of the GEF as an institution. In both occasions IEO and SEC worked jointly at the update.
IIEO Gap Analysis: in its AWPB for GEF-6 (2014), IEO indicated it will update the M&E policy. One year later, IEO provided an analysis of current gaps in the 2010 policy. The analysis identified several gaps…
OPS6: the GEF is different from what it was 8 years ago. It increasingly invests in integrated programming through programs and multifocal projects.
Most of the time, program-level M&E is not addressed, and when it is, it does not report on how child projects contribute to the program’s objectives.
Current reporting requirements for multifocal projects increase operating costs; at the same time, synergies generated and tradeoffs mitigated are not captured

12. Underpinning IEO Work: Terminal Evaluations
Terminal Evaluations Include:
Results: Outputs, outcomes and progress to impact
Implementation, execution, and project cycle related information
Project finances including co-financing
Recommendations and Lessons for the future
GEF M&E Policy (2010): Minimum Requirement 3
Terminal evaluations mandatory since 1995
Required for full-sized and medium-sized projects
Terminal Evaluation Guidelines:
Available at aluations/guidelines-gef- agencies-conducting- terminal-evaluation-full- sized-projects
What are terminal Evaluations?
Terminal evaluations are evaluations at the end of the project.
How are TEs used?
They are an important sources of information on a project: please see the slide
TEs are also used for:
Reporting at the project portfolio level (APR, AMR)
Input to other evaluations
STAR’s performance index
More than a 1300 terminal evaluations are completed so far.
Terminal evaluation may be accessed at:
GEF website:
Through PMIS

13. Characteristics of a good terminal evaluation
Quality
Outcomes
Consistency and comprehensiveness
Sustainability
Lessons and recommendations
Project finances
M&E
Other characteristics
Transparency and timeliness
Candor
Balance
Utility
Characteristics of a good terminal evaluation
Quality of terminal evaluations
The GEF IEO, along with the evaluation offices of some of the GEF agencies, track quality of terminal evaluations. The GEF IEO approach assesses quality on following parameters:
Outcomes: how well the outcomes and other results of the project have been discussed and compared with the expectations.
Consistency and comprehensiveness: to what extent information provided in the terminal evaluation is consistent, comprehensive and complete.
Sustainability: how well the terminal evaluation report discusses concerns related to future progress and continuation of the progress made so far.
Lessons and recommendations: the extent to which a terminal evaluation provides valid recommendations and lessons that are based on the project experience.
Project finances: the extent to which information on project finances including information on co-financing is complete.
M&E: the extent to which the terminal evaluation provides information on implementation of the M&E plan, and use of monitoring information.
Among these criteria, the GEF IEO gives more weightage to discussion on outcomes, and consistency and comprehensiveness of information.
There are several other characteristics of the good terminal evaluation that are not covered in the GEF IEO criteria. These may also play an important role in preparation of terminal evaluation of good quality:
Transparency: the extent to which the process through which terminal evaluation was conducted adds to its credibility.
Timeliness: timeliness of the terminal evaluation ensures that the evaluators had access to relevant information sources and it ensures that the information coming out of the evaluation is readily available to the GEF stakeholders.
Candor: Candor is very important in ensuring that the weaknesses identified in the project results, implementation process, and design, are documented, analyzed and presented so that these may be avoided in future.
Balance: the evaluation should provide information on different perspectives on project performance. It should provide a balanced coverage to these perspectives based on their relevance.
Utility: a good terminal evaluation should also be backed up with efforts to ensure its utility. Thus, it should be part of a larger knowledge management system that ensures it utility to the partnership.

14. Independent Evaluations and Methods
Semi-Annual Evaluation Report
Impact, Thematic, Performance, Corporate, Country Clusters
Comprehensive Evaluations every 4 years
Methods
Qualitative
Quantitative including GIS, Remote Sensing, Big Data Analytics

15. RECENT EVALUATION FINDINGS

16. Performance and Impact
81%
62%
Satisfactory outcomes
of projects have outcomes that are likely to be sustained
Outcomes:
Performance and sustainability of outcomes is higher in MICs ((Brazil, China, India, Mexico, Russian Federation)
A lower percentage of projects implemented in Africa, Small Island Developing States (SIDS), and Least Developed Countries (LDCs) have satisfactory outcomes as compared with the portfolio average.
Africa – 74%
SIDS – 64%
LDCs – 71%
When projects are divided into two categories – those approved from GEF-pilot trough GEF-2 and those from GEF-3 and later, projects implemented in fragile states, regional projects, and in Europe and Central Asia show statistically significant improvement over time. In these groups we see improvement of 10% or more.
Fragile states – 58% (upto GEF-2); 77% (GEF-3 onwards)
Regional projects – 76% (upto GEF-2); 85% (GEF-3 onwards)
Europe and Central Asia – 76% (upto GEF-2); 86% (GEF-3 onwards)
Sustainability:
62 percent of completed GEF projects are rated likely for outcome sustainability. Sustainability have improved over the GEF replenishment periods. Some categories of projects such as projects in LDCs, fragile states, global projects, climate change and biodiversity focal area, show statistically significant improvements in sustainability ratings through GEF-2 to GEF-3 and onwards.
Likelihood of outcome sustainability at project completion is influenced by
the quality of project preparation,
country context,
support from the government,
the quality of implementation and execution, and
materialization of co-financing.
Cofinancing commitments for GEF-6 exceed the target
Project cycle efficiency gains are slow
8.8:1
6:1 target

17. Broader adoption and transformational change
61%
59%
of projects achieved broader adoption
of projects achieved environmental stress reduction
GEF interventions have contributed to reducing environmental stress and broader adoption
✔ Level of ambition ✔ Effective transformational mechanism ✔ Quality of implementation and execution
✔ Harnessing market forces
✔ May be achieved by projects of different size
Mechanisms for broader adoption Mainstreaming and replication Scaling-up and market change
Success factors for transformational change:
Clear ambition in designs
Addressing market reforms through policies
Mechanisms for financial sustainability
Quality of implementation and execution
May be achieved by projects of different size

18. GEF interventions yield positive returns on investment
$1:1.08
$1:1.04
Land degradation
Biodiversity
43.52
tC/ha
Picture of land degradation
Picture of biodiversity
$ amount on top of the picture
Land Degradation (take from old ppt) $1.08/$1
Value for Money of Biodiversity Projects
$1.04 return per $1 invested (likely underestimate)
Positive impacts on forest cover, vegetation productivity, and carbon sequestration
Factors (BD)
Project Size

19. GEF’s Comparative advantage: OPS6
RELEVANCE
Serves multiple conventions and broad range of environmental issues
Strong Support to LDCs and SIDS
PERFORMANCE
Long history of good performance
Ability to address linkages and synergies between focal areas
TRANSFORMATIONAL
Ability to create an enabling environment through legal and regulatory reforms
Delivers innovative financial models and risk-sharing approaches
Picture of land degradation
Picture of biodiversity
$ amount on top of the picture
Land Degradation (take from old ppt) $1.08/$1
Value for Money of Biodiversity Projects
$1.04 return per $1 invested (likely underestimate)
Positive impacts on forest cover, vegetation productivity, and carbon sequestration
Factors (BD)
Project Size

20. Sharing Evaluative Knowledge
Events and Products
Website
Earth-Eval
Partnerships
Expanded Constituency Workshops

21. EVALUATION WORK IN PROGRESS
Photo by Jeremy Cai on Unsplash

22. EVALUATION WORK IN PROGRESS Evaluations Underway
June 2019 Annual Performance Report with a focus on Transportation Evaluation of GEF Support to Scaling Up Impacts Value for Money Analysis of GEF Support to Sustainable Forest Management and REDD+ projects The GEF Evaluation Policy
Fall 2019
Strategic Country Cluster Evaluations:
African Biomes
LDCs
SIDS

23. SUSTAINABILITY
Photo by David Clode on Unsplash

24. Sustainability at Project Completion
62%
of projects have outcomes that are likely to be sustained
GEF’s definition of sustainability is consistent with other multilateral organizations: continuation of benefits after the [project] implementation and risks to their continuation.
At project completion, 62% of projects are likely to be sustained.
Outcomes:
Performance and sustainability of outcomes is higher in MICs ((Brazil, China, India, Mexico, Russian Federation)
A lower percentage of projects implemented in Africa, Small Island Developing States (SIDS), and Least Developed Countries (LDCs) have satisfactory outcomes as compared with the portfolio average.
Africa – 74%
SIDS – 64%
LDCs – 71%
When projects are divided into two categories – those approved from GEF-pilot trough GEF-2 and those from GEF-3 and later, projects implemented in fragile states, regional projects, and in Europe and Central Asia show statistically significant improvement over time. In these groups we see improvement of 10% or more.
Fragile states – 58% (upto GEF-2); 77% (GEF-3 onwards)
Regional projects – 76% (upto GEF-2); 85% (GEF-3 onwards)
Europe and Central Asia – 76% (upto GEF-2); 86% (GEF-3 onwards)
Sustainability:
62 percent of completed GEF projects are rated likely for outcome sustainability. Sustainability have improved over the GEF replenishment periods. Some categories of projects such as projects in LDCs, fragile states, global projects, climate change and biodiversity focal area, show statistically significant improvements in sustainability ratings through GEF-2 to GEF-3 and onwards.
Likelihood of outcome sustainability at project completion is influenced by
the quality of project preparation,
country context,
support from the government,
the quality of implementation and execution, and
materialization of co-financing.

25. Discussion Based on your experience, discuss
What are the factors that affect sustainability of project outcomes?
Sustainability:
62 percent of completed GEF projects are rated likely for outcome sustainability. Sustainability have improved over the GEF replenishment periods. Some categories of projects such as projects in LDCs, fragile states, global projects, climate change and biodiversity focal area, show statistically significant improvements in sustainability ratings through GEF-2 to GEF-3 and onwards.
Ask participants to discuss their experience with GEF and other environmental projects. What are the factors that affect sustainability? Give participants 10 minutes for small group discussion. Ask several groups to share their reflections with the whole group. Make notes on the flipchart. Then, discuss the IEO findings:
GEF IEO sustainability analysis (June 2018, part of the Annual performance Report). The following factors affect the likelihood of sustainability of outcomes at project completion:
the quality of project preparation,
country context,
support from the government,
the quality of implementation and execution, and
materialization of co-financing.

26. Discussion - continued
What factors affect sustainability after the project completion?
We discussed some of the factors that affect project sustainability before and during the project, such as quality of project design, project implementation and execution, government support, and country context. Now, based on your experience, what are some of the factors that may affect sustainability of outcomes once the project is completed?
Discussion in small groups – 5 minutes, followed by a plenary discussion. Make notes on the flipchart, then discuss the IEO findings (GEF IEO sustainability analysis (June 2018, part of the Annual performance Report):
53 projects were evaluated in-depth for sustainability after the project completion.
The key factors that contribute to higher sustainability of outcomes after the project completion include
high stakeholder buy-in,
political support,
availability of financial support for follow up, and
sustained efforts from the executing agency.

27. VISIT US AT www.gefieo.org
FOR MORE
VISIT US AT
Photo by Karsten Würth