Harvesting outcomes from a global network dedicated to improving the life of vulnerable children worldwide Goele Scheers EES-2016 Conference Maastricht Outcome Harvesting- Principles in Practice Thursday 29 September 2016 15:45 – 15:15 PM Room 0.11

Family for Every Child is a global alliance of national civil society organisations working together to mobilise knowledge, skills and resources to build a world where every child grows up in a permanent, safe and caring family, and to provide quality alternative care where needed

Outcome Harvesting for Family for Every Child OH Evaluation: outcomes 2012-2016 Integrate OH into the M&E system of the network

6 OH steps applied to Family for Every Child 1. Design the Harvest 2. Review documentation and draft outcomes 3. Engage network members, board and staff 5. Substantiation 6. Support the use of findings 4. Analyse and interpret

Principle 1: Ensure usefulness throughout the evaluation 1. Design the Harvest

Users and uses as defined during design The EveryChild Board of trustees will use the findings to decide on a full transfer of its assets and resources to Family for Every Child and thereby complete the transition process. Family for Every Child members, Board and Senior Management Team will use the findings as input for the Strategic Plan.

Useful evaluation questions What are the emerging external outcomes telling us about the added value of the network, in particular in relation to Family’s conceptual framework goals? What are the emerging internal and external outcomes telling us about the vibrancy and sustainability of the network, in particular in relation to Family’s strategic objectives? What does this mean for the asset transfer decision and the strategic plan?

During the harvest Deadline for asset transfer process was moved forward, design had to be adapted After the harvest had been done: decision together with users not to substantiate. Outcomes deemed to be credible enough for internal use. Users and evaluator decided not to do additional harvesting, because the results were needed quicker than anticipated to inform the strategic planning process

2. Review documentation and draft outcomes Principle 2: Harvest social change outcomes Principle 3: Formulate an outcome as an observable change Principle 4: Establish plausible influence or contribution, not control or attribution 2. Review documentation and draft outcomes

Document Review Due to the strong focus on activity and output reporting, there were only a small number of outcomes generated from documents

2 types of network outcomes Internal outcomes: changes in the behaviour, relationships, or actions of the Family for Every Child members and bodies which strengthen and develop their collective capacity to achieve the network’s purpose. External outcomes: changes in the behaviour, relationships, actions, policies or practices of individuals, groups or organisations outside of the Family for Every Child network that represent significant contributions towards the network’s purpose.

Example of an external outcome from Family Outcome description: In June 2015, The Ministry of Women and Child Development of India invited Butterflies (Family’s Indian member) to lead the Foster Care Committee in developing guidelines, training modules and monitoring tools on foster care and included their contribution in the Foster Care guidelines that were launched in October 2015. Significance: This was a key moment in the development of state policy and guidance in India, when not only international NGOs and experts, but also national NGOs, including Butterflies were asked to contribute. Previously, there had been piecemeal and patchy development of foster care in India, led by internationally supported NGOs, but not strongly or consistently coordinated by the government. Contribution: The exposure of Butterflies to foster care through its membership of Family enabled it to respond to a request it received from its government, and thereby make a significant contribution to the government’s development of policy and guidance for foster care providers. Butterflies was able to respond within tight deadlines because of the help of network members finding and sharing with them the policy and practice examples from various countries. Network contribution can be secretariat as well as members, board, etc Pitfalls: wanting to formulate ‘more aware’ as an outcome

Example of an internal outcome from Family Outcome Description: At the 2015 Assembly of Members in South-Africa, network members were leading the sessions. Significance: This outcome demonstrates that there has been a shift in power, decision-making and representation away from the secretariat to the members. It is an indication, that Family for Every Child is becoming a member-led network. Contribution: The secretariat, together with the board, have put the necessary structures in place that enabled network members to lead sessions. For example: the approach to having co-chairs for the Multi Country Initiatives meant that there were members who had been involved in the decision-making on the development and delivery of the projects. They were therefore informed enough and participating in the leadership of the projects enough to be able to lead sessions.

3. Engage network members, board and staff Principle 5: Facilitate the identification and formulation of outcomes Principle 6: Nurture appropriate participation 3. Engage network members, board and staff

Outcomes were gathered during focus groups with secretariat staff, board and members  95 outcomes

The evaluator facilitated the harvesting of outcomes and followed up through shared google documents, e-mail and skype afterwards Emphasise the time required for ping-ponging

Example google docs

Example outcome format

Principle 7: Rigorously aim for credible-enough outcomes 5. Substantiation

No substantiation with third parties, but: The outcome descriptions concern observable facts and are specific and concrete enough to be verifiable. They were provided by informants who were knowledgeable about the outcomes. The informants agreed to go on record with the information they provided about outcomes. 
 The relationship between how Family for Every Child contributed and the outcomes was judged by the evaluator to be plausible.
 There was cross-validation between secretariat staff and members After the Outcomes had been gathered, the evaluator together with the users decided that substantiation with third parties was not necessary, since the outcomes were only going to be used internally. Outcomes were the contribution was not clear or the description insufficient were deleted

Principle 8: Strive for less because it will be more Principle 9: Reveal patterns of social change 4. Analyse and interpret

22 outcomes indicating a clear and successful transition of responsibilities from the secretariat and EveryChild to the members of the network represented in the Board

NATURE OF THE CHANGE Change in Relationships Change in Activities Majority of outcomes are about changes activities and capacity (67%) 19% directly linked to changes in relationships 10% between changes in activities and relationships 19% 10% Change in Activities Change in Capacity 67%

Principle 10: Learn Outcome Harvesting experientially 5. Support use of findings and make strategic decisions 3. Engage network members, board and staff

Family members and staff: Were introduced to the Outcome Harvesting methodology through workshops Learned to formulate outcomes ‘while doing’ during the Outcome Harvest

Family for Every Child members and staff: Need further guidance in harvesting their outcomes until the network is able to continue on its own

Thank you! Goele Scheers Independent consultant goelescheers@gmail.com www.goelescheers.be