Towards a framework for assessing climate change interventions through impact evaluation Martin Prowse, ODI

Key messages 1.The applicability of impact evaluation to assessing climate change interventions has not been widely considered 2.However, some mitigation and adaptation measures show potential in the short term Mitigation: Biofuel production, forest protection initiatives, and environmental labelling Adaptation: Community-based adaptation 3.Other measures show potential in the longer term (for example, projects within NAPAs)

Overview I 1.To what extent has impact evaluation been applied to climate change interventions? 2.How can we assess if IE could be applied to climate change interventions? 3.What are the generic shortcomings of impact evaluation?

Overview II 4. Four mitigation measures (green growth strategies, environmental labelling initiatives, biofuel production, forest protection) 5. Four adaptation measures (community- based adaptation, disaster risk screening, GEF LDCF, GEF SCCF) 6. Conclusions

1. To what extent has impact evaluation been applied to climate change interventions? ‘In the context of impact evaluations of GEF projects, it is clear that the rigorous impact evaluation model is neither appropriate nor affordable’ Mixed experience from using IE-type approaches GEF’s alternative: Bamberger’s ‘Shoestring Methodologies’ and a ‘Theory of Change’ approach

2. How can we assess if IE could be applied to climate change interventions? Costly and time consuming Suited to small-scale interventions, not large-scale policy reforms Direct budget supports limits scope for ex ante IE How does IE intersect with country ownership?

Institutional inertia Moral and ethical concerns Technical capacity and institutional compliance 3. What are the shortcomings of impact evaluation?

4. Four mitigation measures Green growth strategies (carbon credits from offset projects in non-Annex I countries) Environmental labelling initiatives (which illustrate the ‘carbon footprint’ of products) Biofuel production (of second-generation biofuels such as jatropha) Forest protection schemes (such as Reduced Emissions from Deforestation and Degradation – REDD)

4. Four mitigation measures

5. Four adaptation measures Community-level adaptation (an autonomous, bottom- up approach) ORCHID (disaster risk screening tool) GEF Least Developed Country Fund (provides adaptation finance to the poorest countries, especially through National Adaptation Plans) GEF’s Special Climate Change Fund (again has a primary focus on adaptation)

5. Four adaptation measures

4. Community-based adaptation Autonomous bottom-up approach to adaptation Builds on local technical knowledge and coping strategies Incongruence with IE? (i.e. participatory standpoint vs. ultra- positivism) But ‘with and without’ comparisons of CBA are possible

ORCHID Mainstreaming climate risk management through appraising projects and programmes Probably not desirable to randomise organisations or programmes Might be possible to use quasi-experimental methods

6. GEF’s Least Developed Country Fund Provides finance to the poorest countries, mainly for adaptation Supported completion of NAPAs Common priorities: water resources; food security; agriculture; infrastructure. NAPAs: variable quality, with focus on conventional development projects IE can assess such projects

7. GEF’s Special Climate Change Fund SCCF has a primary focus on adaptation Again, a focus on water resources, agriculture, infrastructure Plans a reminiscent of conventional development projects

Conclusions 1.The applicability of impact evaluation to assessing climate change interventions has not been widely considered 2.However, some mitigation and adaptation measures show potential in the short term Mitigation: Biofuel production, forest protection initiatives, and environmental labelling Adaptation: Community-based adaptation 3.Other measures show potential in the longer term (for example, projects within NAPAs)