1. Outcome Mapping: from alternative to mainstream
Outcome Mapping: from alternative to mainstream? The OM usefulness barometer OM Master Class - 26 March 2013, Brussels Jan Van Ongevalle

2. High expectations of PME
Address a growing call to show results
Satisfying multiple accountability needs
To show results that are difficult to measure
Stimulating internal learning processes
Learning about unexpected results

3. Extra challenging when dealing with processes of complex change?
Multiple actors, different perspectives and different forms of interaction
Multiple accountability needs
Unpredictable and non-linear link between cause and effect, therefore important to learn quickly what works and what doesn’t.
When dealing with complex change you need to be able to change as well.

4. 10 NGOs in action-research (2010-2012, PSO & HIVA
Vredeseilanden
ETC Foundation
Personal goal exercises
Client satisfaction instruments
- In all cases it will be hybrid forms, often still with elements of logframes

5. How can actor focused PME help in dealing with processes of complex change
“How can PME contribute to dealing with complex processes of social change?”
1. How does the PME approach help to clarify relationships, roles and expectations?
3. How does the PME approach help to strengthen the own internal adaptive capacity?
4. How does the PME approach help to strengthen upward, downward and horizontal accountability needs?
2. How does the PME approach help to learn about the progress towards the development objectives?

6. OM usefulness barometer (2012-2013) - Methodology
Launch of websurvey on the outcome mapping learning community
Websurvey respondents (N=43)
15 interviews
2 additional case studies

7. General info about survey respondents

8. OM and multiple actors and relationships (N=43)
One respondent disagreed: “being both a capacity builder and donor made roles and responsibilities confusing despite OM”

9. How does OM help to deal with multiple actors?
1. Providing a practical framework for actor focused PME
3. A framework for building trust
2. Stimulating conversation and dialogue

10. OM and learning
Learning about small incremental changes at the level of boundary partners (new result area).
Becoming more specific about effects you hope to contribute to.
Motivating to see that you can monitor these small incremental changes
OM no garantee for learning

11. Monitoring effects that are difficult to quantify….
Monitoring changes in behaviour (Global Water Partnership):
On 8 September 2009, officials representing key Vietnamese government organizations agreed to adopt a recommendation on strengthening River Basin Organisation in Vietnam following an Integrated Water Resource Management approach in basins
The recommendation was adopted by government officials after having discussion in a Dialogue on River Basin Management in Vietnam organized by GWP’s affiliate the Vietnamese Water Partnership.

12. Monitoring unexpected effects
E.g.: Sustainable Agriculture Programme - Pacific Islands and Territories One of the love to see progress markers for the targeted villages read as follows: “sharing lessons and experiences with the other villages”. It was an unexpected surprise for the programme team to learn during the monitoring process that it was not the targeted village communities that took the initiative to share experiences but instead the surrounding villages invited them to come and share new technologies for improving their agriculture production. This happened without any support from the programme.

13. Satisfying accountability needs

14. Accountability towards the donor
Monitoring reports more focused on a programme’s outcomes instead of its outputs.
helps to report on how outcomes were obtained and how the programme was able to contribute to them.
OM can complement other PME approaches that are mandated by the donor.
But OM offers no garantee that donor’s information needs will be met:
donors don’t always want info about capacity development.
Appetite for quantitative information
Effort to translate OM information in info that donors need

15. Innovative ways of visualising OM information

16. Satisfying accountability needs of…
boundary partners In case BPs are actively involved in the monitoring process…… but may take considerable effort and not always possible.
final beneficiaries
Need to go beyond the OM tools and methods to strengthen downward accountability

17. Adaptive capacity Challenges: Time & resources
Stimulating informal learning spaces
Capacity to facilitate reflection

18. Potential challenges for OM implementation
head: agenda for OM
arms: concepts, methods, tools of OM
spine: values, principles of OM
legs: support for the implementation of OM

19. What do we hear in the corridors about PME policy?
We are not using alternative PME approaches because it is not allowed by our donor.
Why experimenting with alternative PME approaches? Start by implementing the logframe approach more effectively.
Policy makers are not interested in learning. They just want to know if we achieved our indicators 19

20. Space for experimentation within existing PME policy
Space for experimentation within existing PME policy? (2010 Survey: )
Indirect aid (NGO’s)
Direct aid (Belgian Technical Cooperation, BTC)
Special evaluation unit (Belgian Ministry of Development Cooperation)
OESO/DAC
Use of Logical framework
Logical Framework is mandatory
Logical framework is not mandatory but standard PME tool.
Is not mandatory as evaluation instrument. Follows OESO/DAC evaluation criteria
Predominant reference to the logical framework as PME framework (e.g. managing aid 2005,2009).
Use of alternative PME approaches
Experimentation with alternative PME approaches at operational level.
No policy but negotiated case by case
BTC pilots outcome mapping in selected programmes since 2010.
Spheres of influence featuring strongly in new PME manual of BTC
Specific approaches are not prescribed but importance of both quantitative and qualtitative methods are recognised.
Growing recognition of the importance of methodological diversity, e.g.
When dealing with cap. dvpt. (OECD, 2006).
Civil society & Aid Effectiveness (OECD,2009)
Legal frameworks do not explicitely refer to a specific PME method.
Logframe becomes mostly mandatory through the subsidy guidelines that are based on linear results based planning models.
Need for tangible results at beneficiary level measured through SMART indicators. 20

21. Recommendations Practitioners
From results based management towards results based learning
Considerable effort to nurture a learning culture: time, resources, managment support
Don’t use donor requirements as excuse for not investing in learning about results
Donors
Make funded programs accountable for their learning about results
Use wider notion of results
Ask for actor focused theories of change and accept that these can change over time
Develop own PME capacity

22. Development is not only about….
.... problems that can be fixed
...but also about complex processes of social change; where a diversity of actors challenges current practice and experiments with alternatives

23. Opportunities for further research
Develop more insight in the percieved usefulness of OM among donors and boundary partners.
More longitudinal case study research.
Explore how using OM contributes to bringing the complex story of development and how this may help towards development education of the general public in donor countries.

24. So what is your position on the OM usefulness barometer?
Thank you!!