1. AEA conference 2016, Atlanta
Making Sense of Outcomes Using the signification process of SenseMaker to enrich Outcome Harvesting
Goele Scheers
AEA conference 2016, Atlanta
3:30 PM–4:15 PM
Oct 28, 2016
Room A701

2. Outcome Harvesting
A tool to identify, formulate, analyse and interpret outcomes to answer useable questions.
How did a programme or initiative contribute to outcomes?

3. Behavioural change outcomes
An observable and significant change in a social actor’s behaviour, relationships, activities, actions, policies or practice that has been achieved and that has been influenced by the change agent.

4. 6 steps Design the harvest Review documentation and draft outcomes
Engage with change agents
Substantiate
Analyse, interpret
Support use of findings

5. Outcome changes / fragments / statements
SenseMaker
Outcome changes / fragments / statements
SenseMaker is a complexity-aware monitoring and evaluation approach
Collection of large numbers of micro-narratives (experiences, anecdotes, events, moments, etc.)
Self-signified by the storyteller (at the point of origin)
SenseMaker® comes with software for collection and analysis (Cognitive Edge)
Steff Deprez (April 2016)

6. Please, tell us more about your story …
Think about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.
Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?
Please, tell us more about your story …
Steff Deprez (April 2016)

7. Please, tell us more about your story …
Think about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.
Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?
Please, tell us more about your story …
Steff Deprez (April 2016)

8. Please, tell us more about your story …
Think about a specific recent moment or event when you felt particularly encouraged or concerned about being a coffee producer and selling your produce to the ‘company’.
Think about what it is like being a girl? What happened recently to yourself or another girl? Please share the story of what happened?
Please, tell us more about your story …
In your experience, attitudes revealed were…
wanting things to change
Keep things as they are
Steff Deprez (April 2016)

9. Cases
Case 1: The signification process applied during an Outcome Harvesting Evaluation of a Global Network dedicated to improving the life of vulnerable children
Case 2 : Collective signification during an outcome harvesting process in a Cereal Value Chain project in Mali (USAID).

10. The signification process applied during an Outcome Harvesting Evaluation of a Global Network dedicated to improving the life of vulnerable children

11. Family for Every Child is a global alliance of national civil society organisations working together to mobilise knowledge, skills and resources to build a world where every child grows up in a permanent, safe and caring family, and to provide quality alternative care where needed

12. Outcome Harvesting for Family for Every Child
OH Evaluation: outcomes
Integrate OH into the M&E system of the network

13. 6 OH steps applied to Family for Every Child
1. Design the Harvest
2. Review documentation and draft outcomes
3. Engage network members, board and staff
6. Support use of findings and make strategic decisions
5. Substantiation
4. Analyse and interpret

14. Harvesting the outcomes
Outcomes were gathered during focus groups with secretariat staff, board and members  95 outcomes

15. Harvesting the outcomes

16. Example of an outcome from Family
Outcome description: In June 2015, The Ministry of Women and Child Development of India invited Butterflies (Family’s Indian member) to lead the Foster Care Committee in developing guidelines, training modules and monitoring tools on foster care and included their contribution in the Foster Care guidelines that were launched in October Significance: This was a key moment in the development of state policy and guidance in India, when not only international NGOs and experts, but also national NGOs, including Butterflies were asked to contribute. Previously, there had been piecemeal and patchy development of foster care in India, led by internationally supported NGOs, but not strongly or consistently coordinated by the government. Contribution: The exposure of Butterflies to foster care through its membership of Family enabled it to respond to a request it received from its government, and thereby make a significant contribution to the government’s development of policy and guidance for foster care providers. Butterflies was able to respond within tight deadlines because of the help of network members finding and sharing with them the policy and practice examples from various countries.
Network contribution can be secretariat as well as members, board, etc

17. Harvesting the outcomes

18. Tagging of outcomes: Nature of the outcome

19. Tagging of outcomes: Significance

20. Tagging of outcomes: Contribution

21. Tagging of outcomes: Effects of the outcomes

22. Analysis: triad

23. Analysis/Triads filtered

24. Analysis: contribution dyad

25. Analysis: Dyad filtered
CONTRIBUTION OF THE NETWORK
Internal outcomes
68%
External outcomes
10 outcomes
14 outcomes
Change has happened
without influence of the network
Change only happened because
of the interventions of the network

26. Analysis: dyads filtered

27. Analysis: useless triad
Better Outcomes Children
55%
STRATEGIC OBJECTIVES
Example where the triad was not useful
37% 8%
Governance Membership
Income

28. Collective signification of outcomes Outcome Harvesting Evaluation Cereal Value Chain project in Mali

29. The Cereal Value Chain Project
Focus area’s
(1) Improve Farmer Cooperative Management
(2) Improve Production Quality and Quantity
(3) Increase Market Sales of Millet/Sorghum and Rice
Five-year program ( )
101 communes: Sikasso & Mopti
One key implementer: Bamako
Two local partner organisations: Mopti, Sikasso

30. OUTCOME HARVESTING WORKSHOP
Harvesting of Outcomes
DOCUMENT REVIEW
OUTCOME HARVESTING WORKSHOP
Outcome Formulation
Outcome description
Significance description
Contribution description
Signification
2-days
> 60 outcomes formulated

32. Banks
On 10/10/2014 the Mopti BMS financed the Fédération pour le Développement Rural de Youwarou (FDRY) for the sum of 246,500,000 FCFA.
Farmer Cooperatives
From 2014 until April 2016, the cooperative Rizicultrice de Niéna (512 members) has applied System Rice Intensification (SRI) on an area of 40ha, while the original forecast was 10 ha.
Farmer Cooperatives
In 2014, the cooperative of Zébala 7 (12 members) has managed to do a group sale of 3.300T of sorghum and 7,24T millet for an amount of FCFA. The extra revenues were used to start building a warehouse for cereals.
OMA
The ‘Office de Observation du Marché Agricole’ (OMA) expanded the zones in 2014 in which market information (weekly price and quality of cereals sold in the Mopti, Sikasso and Ségou regions) is distributed
District Agriculture Authority
In 2015, the DRA used 7 local radio stations to broadcast at large-scale agricultural production technologies and the use of pesticides. The radio stations are ORTM Sikasso, Radio Kenedougou de Sikasso, Yeredon de koutiala, Wassoulou de Yanfolila, Folona de Kadiolo and Kafo Kan de Bougouni.
Research Institutes
As of June, 2014, the Regional Center for Agricultural Research (Centre Régional de Recherche Agricole CRRA) in Mopti introduced 4 new varieties of ameliorated rice adapted to the village parameters. These four varieties are: WAPMO, SUTURA, SAKU and SK In 2014, seed production for 4 varieties were 4 tonnes (base seeds of G4). During the two last years (2015, 2016) the CRRA self-funded to pursue these activities of producing seeds of the 4 varieties. The seeds were sold in seed vendor stores.

33. OH template Guides discussions Collective Indexing
Facilitates documentation
OH template

38. Collective tagging & signifcation

39. Analysis

40. Comparison of the nature of outcomes in 2014 and 2015
In 2015, Faso Kaba of Bamako, a seed company, expanded its distribution network in Koutiala in the region of Sikasso by including the producer organization, Yeredon of Kadiaradougou (commune of Loulouni), and Benkadi of Parasso (commune of Loulouni).
Comparison of the nature of outcomes in 2014 and 2015
The CMDT invested in the capacity building of cooperatives in 2014 and 2015, to help prevent erosion. This was done through the restauration of the soil and the increase in agricultural yield across the 117 cooperatives and more precisely in N’gounkan and Bandiagara in the Koutiala subsidiary.
In 2015, three cooperatives (Kendé, Ogodiré, Souley) finalized the process for formalizing their cooperatives according to the law of the Organization for the Harmonization of Business Rights in Africa (OHADA)

41. Contribution of the CVC Activity to the outcomes
Analysis
Contribution of the CVC Activity to the outcomes
16 outcomes
7 outcomes

42. Analysis Agro-Dealers Farmer Organisation Processing Companies
Grain Traders
Banks

43. Reflections on Sensemaker for OH - Strenghts
COLLECTIVE SIGNIFICATION
Stimulates discussion and joint reflection during harvesting
Common understanding about program, outcomes and the role of the implementer
Triads are easy to understand, useful, possibility to make nuances
ANALYSIS
More depth, new layers of meaning/connections
Easy visualization of patterns
Change agents can relate to the analysis

44. Reflections on Sensemaker for OH - Strenghts
SUPPORTING USE OF FINDINGS
Feedback of the results of the OH evaluation
Further collective sensemaking on the outcomes
Generate actionable insights

45. Reflections on Sensemaker for OH - Challenges
Three corners need to be well-defined
Triads have to be connected to the use and the evaluation questions
Need to have a substantial amount of outcomes

46. Thank you!
Goele Scheers Independent consultant