The value of the Outcome Harvesting tool in evaluating advocacy organizations Barbara Klugman with Ricardo Wilson-Grau AEA Conference 2012

Meaning of Outcome Harvesting Judging the merit, significance or worth of the advocacy effort by exploring the depth and meaning of patterns among the actual outcomes harvested, in relation to intended outcomes of that group ‘Harvesting’ or identifying and formulating actual outcomes of the advocacy interventions of an organization or network – Description of a change in a policy or practice – Role of other actors in that contribution – Contribution of the organization or network – Substantiation of these outcomes by those who made the change happen, or who contributed

Why use Outcome Harvesting? – Enabling a shift from intuitive assessment of progress to evidence-based assessments of progress – Naming the outcomes in very specific, measurable terms (who changed what, when, where) To bring rigour to organizations’ ability to assess to what extent their advocacy strategies are contributing towards their long-term intended outcomes through – Enabling a conversation among activists about which strategies are working most effectively and which are not … To identify outcomes that occur in complex environments reflecting contribution of many actors

OH is especially useful when groups know what they are trying to achieve When organizations have a theory of change against which to make sense of actual outcomes and Some means of documenting outcomes in an ongoing way OH can be an especially powerful aid for understanding what is being achieved

OH clarifies the relationship between activity, output and outcome Pushes organizations to be honest about effectiveness And be precise about what has changed Validates process outcomes which can build confidence in contexts where outcomes are unpredictable

‘Exposure’ visit of Thai land rights activists to India - AJWS* resulted in outputs – Shifts in knowledge, motivation and ideas including recognising each others’ struggles as part of a single movement which lead these activists to make changes which delivered outcomes e.g. – Created organizational mechanisms to increase women’s opportunities for leadership – Strengthened their cross-organizational and cross- regional alliances; acted in solidarity with each other *AJWS - American Jewish World Service

OH accumulates outcomes to show progress over time e.g. ‘Exposure’ visit of Thai land rights activists to India Output: participants recognising value of building movements across issues & geographies Outcomes from this learning: On returning participants formalized a forum of NGOs, academics and public opinion leaders in the southern region of Thailand to build a shared analysis of the multiple issues facing communities. Just over two months later when a community in x area was fined for intruding on private land and their leader was taken to court, other organizations from the newly consolidated group came to court in a show of support. Those involved indicate that this would not have happened before the exposure visit.

Where to find outcomes? Organizations are unlikely to have recorded outcomes fully or rigorously. Conversely, if they have done so, They are likely to want more from OH than a reiteration of their outcomes  Requires identifying sources & their quality before negotiating terms of an evaluation because of major time and money implications  Requires ensuring the organization fully understands what OH will and will not deliver

What OH will not do OH does not identify opportunities missed OH does not explain the ‘why’ an outcome materialises OH can show what current strategies delivered, but is not forward-looking; it does not identify current opportunities and obstacles

Conclusions: OH is a tool that delivers an evidence-based assessment of strategic effectiveness reveals the correlations between these and an organization’s or a network’s strategies and intended outcomes; delivers rigorously stated outcomes In addition, these insights can provide an excellent grounding for broader organizational or strategic evaluations