An Evaluation Agenda for USAID’s Global Climate Change Initiative Kit Kernan, Ph.D., Technical Expert Marc D. Shapiro, Ph.D., Project Leader Global Climate Change Monitoring and Evaluation Project Development and Training Services (dTS) Contact: MShapiro@OnlineDTS.com
Outline Context and GCC results frameworks Evidence in the literature USAID’s climate change portfolio Near-term evaluation agenda Long-term evaluation agenda
GCC Context and Result Frameworks USAID’s 2011 Evaluation Policy Impact and performance evaluations Presidential Global Climate Change Initiative (GCCI) and USAID’s GCC Office GCC funding GCC “pillars” Sustainable Landscapes Mitigation, Clean Energy Mitigation, and Adaptation Global Climate Change Monitoring and Evaluation Project
GCC Result Frameworks GCC RFs are the theoretical foundation for USAID investment to achieve objectives 36 specific cause-and-effect links 8 fundamental causal hypotheses
GCC Evaluation Agenda Concept Objective is to improve the efficiency and effectiveness of USAID climate change investment Uncover or generate rigorous evidence to test RF hypotheses systematically Sustained over time Integrates many sources of information Employs several complementary approaches Systematic literature reviews Meta-analysis of evaluations Review hypotheses and theories of change after accumulate evidence
Evidence in the Literature Sampling of literature evidence bearing on hypotheses Findings: GCC hypotheses approximately capture cause-and-effect concepts in literature No rigorous tests found Validity of hypotheses often assumed Emphasizes hypothesis interactions Suggests success dependent on design quality and implementation
GCC Portfolio Analysis 159 USAID activities assessed (fiscal years 2011-’13) 40% Adaptation 33% Sustainable landscapes 27% Clean energy Investment patterns across causal hypotheses reflects USAID practitioner beliefs in relative efficacy and context
GCC Portfolio Analysis GCC investment in hypotheses 21% developing data and analytical tools 20% Individual and institutional capacity building 14% Testing, demonstrating, and disseminating CC technologies 13% improve mitigation and adaptation planning 10% supporting policies, laws, regulations 10% lowering financial barriers 7% social and environmental safeguards 7% establishing mechanisms for international cooperation and coordination
Near-term Evaluation Agenda Selection over time Selection criteria include All “climate change integration pilots” Variety across and within pillars Hypothesis-specific (frequently funded) Data and analytical tools Capacity building Demonstrating and disseminating technologies Mitigation and adaptation planning Supporting policies, laws, regulations (enabling environment)
Near-term Evaluation Agenda Selection criteria (continued) Implementation-specific Scale of project Early in implementation process Mission time availability and capacity for involvement Feasibility of rigor Coverage by other funders/organizations Ultimate selection decisions made by country/regional missions jointly with Washington Level of rigor varies along the spectrum of impact and performance evaluations (IE or PE) Most, not all, under GCCM&E
Near-term Agenda thus Far General/cross-pillar – 1 Building social capital through participatory climate change problem identification (IE, Macedonia) Clean energy – 2 to 4 Micro hydro-power and agriculture in DRC (IE) Private finance networking (PE) Potential State-level top-down planning on energy efficiency & renewables in South Asia (IE) Bottom-up energy efficiency & renewables in Latin America (IE)
Near-term Agenda Possibilities Adaptation - 8 Disaster risk reduction (DRR) and Water management DRR in semi-nomadic area (PE, Angola) DRR and water management (IE, Ecuador) Groundwater recharge and DRR (IE+PE, Indonesia) Agriculture-related Measuring emissions from urea deep placement for rice and potential policy impact (PE, Bangladesh) Climate forecasting and agricultural resilience (PE, Kazakhstan) Crop index insurance (IE, Dominican Republic) Agricultural adaptation capacity building through tertiary education (PE, Guinea) Conflict resolution and prevention in semi-nomadic areas (I/PE, Ethiopia)
Near-term Agenda Possibilities Sustainable landscapes - 1 Reducing Emissions from Deforestation and Degradation (REDD+, IE, Africa) Likely test aspects of 7 fundamental hypotheses Add more over time for learning agenda focused on RF
Feedback sought What do you believe should be priorities regarding Hypotheses to focus on? Topics deserving rigorous evidence? Types of activities most relevant to evaluate for impact in climate change milieu? Most compelling evaluation questions? Other comments or questions