1. Evaluating adaptation
Simon Anderson
Contents
Key challenges for the evaluation of adaptation
Emerging results frameworks, incl. ICF, PPCR
TAMD framework and indicators

2. Key challenges for the evaluation of adaptation - attribution or contribution?
Assessing attribution, establishing baselines and targets, contending with long time horizons, and addressing accountability
Attribution assessed through impact evaluation.
Setting counterfactuals through experimental and quasi-experimental design
More meaningful to consider contribution than attribution in complex contexts and a range of techniques can be used to infer counterfactuals when they are not available.

3. Key challenges for the evaluation of adaptation - baselines and targets
Data availability challenges and changing underlying contexts
Techniques to estimate or reconstruct baselines can be useful here, as well as normalisation of outcome data.
Utilising sequential targets can also help a programme adapt to changing contexts and remain relevant to adaptation objectives over time.
Adaptation investments often long term and results may only be judged when climate effects are fully experienced and efficacy of adaptation can be measured.
Limiting factor analysis and statistical modelling are techniques that may support long term evaluations.
There is the need to fund and support longer term evaluation, such as a longitudinal approach to a thematic area of adaptation work.

4. Key challenges for the evaluation of adaptation - addressing accountability
Public disclosure of information
Inclusion of beneficiary satisfaction scores and participatory techniques such as ‘most significant change’.
Downward and upward accountability can also be improved by integrating learning into programme design, so that a flexible, iterative approach allows periodic reflection and improvement.

5. Emerging results frameworks – challenges identified through a review of the EU GCCA programme
Formulating indicators that accurately measure effectiveness of adaptation can be problematic – theories of change and an adaptation pathway required.
Devising indicators to effectively track improvements in climate finance management systems, or changes in climate expenditure can also prove challenging.
Existing data systems may need supplementary climate change information to enable the characterization of climate challenges.
Lack of clarity regarding implementation and institutional arrangements for M+E.
Limited participation in the design and implementation of M+E frameworks.
Lack of specific focus and activities on M+E of adaptation.
Short term and programme/ project centred evaluation .

6. The TAMD approach - indicators, and use of ‘normalised’ development indicators

7. TAMD Indicators
Indicators using vulnerability as proxy for developmental outcomes plus conventional development parameters – emphasis on using what’s available.
Examine:
Effectiveness of targeting the climate vulnerable poor;
Based on theories of change and an understanding the objectives of different stakeholders, etc.
A set of indicators used to examine the performance of Climate Risk Management functions including adaptation delivery

8. TAMD Steps Theory & Design Indicators Methodology Data Gathering2 3 4 5 6 7
Theory &
Design
Indicators
Methodology
Data Gathering
Analysis
Lessons
Learned
Climate risk management
Development performance 1
Scope

9. Purposes for applying the TAMD approach
CLIMATE RISK MANAGEMENT
CLIMATE IMPACTS ON DEVELOPMENT
INTERVENTIONS
What are the comparative advantages of approaches in different interventions
How well do investments in CRMgt address the increases in vulnerability
How well does a single intervention enable adaptation
How much are investments in CRMgt leading to better adaptation
How well is climate managed across on sector

10. TAMD framework