1. outcomemapping.ca Introduction to Outcome Mapping Ziad Moussa, Lebanon Jan Van Ongevalle, Belgium Heidi Schaefer, Canada 22 September 2014 – Dar Es Salaam – OM Lab

3. Workshop Objectives Clarify what Outcome Mapping is and does Introduce and apply Outcome Mapping concepts and tools Consider if and how Outcome Mapping could be useful in our work

4. Agenda Overview Key concepts Vision Mission Boundary Partners Progress Markers Strategy Map Organisational Practices M&E planning and journals

5. Proposed process guidelines: Everyone can participate One speaks, all listen Value each other’s ideas No smoking or mobile devises Respect each other by being on time Questions any time (answers may come later) Enjoy... Network... Laugh... Sing..!...Other?

6. Everyone has process roles: Shaker and Waker Shaker and Waker Time Keeper

7. Key concepts in Outcome Mapping

8. Acknowledgements This presentation makes use of various materials that were shared by members of the global OM community. Without being exhaustive, special thanks goes to Terry Smutylo, Steff Deprez, Jan Van Ongevalle, Robert Chipimbi, Daniel Roduner, Kaia Ambrose and many others.

9. Source: A guide for project M&E: IFAD

10. Time Source: Ricardo Wilson-Grau (inspired by Jeff Conklin) Conventional thinking… ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS

11. Time … clashes with relationships of cause and effect that are unknown Source: Ricardo Wilson-Grau (inspired by Jeff Conklin) ACTIVITIES OUTPUTS OUTCOMES IMPACT INPUTS

12. Social change can be… Complex: involve a confluence of actors and factors Unstable: independent of project duration Non-linear: unexpected, emergent, discontinuous Two-way: intervention may change Beyond control: but subject to influence Incremental, cumulative: watersheds & tipping points Source: Terry Smutylo

13. OUTPUT OUTCOME ACTIVITY INPUTS ACTIVITY INPUTS ACTIVITY INPUTS OUTPUT ACTIVITY OUTPUT OUTCOME Source: Ricardo Wilson-Grau Time INPUTS

14. Evaluation challenges 1.Establishing cause & effect in open systems 2.Measuring what did not happen 3.Reporting on emerging objectives 4.Justify continuing “successful” interventions 5.Timing – when to evaluate 6.Encouraging iterative learning among partners 7.Clarifying values 8.Working in ‘insecure’ situations Source: Terry Smutylo

16. Brief definition of OM A participatory method for planning, monitoring and evaluation Focused on changes in behaviour of those with whom the project or program works Oriented towards social & organizational learning

17. OM’s answer Source: Terry Smutylo / OM Lab 2012 Support people to build their own well-being Enable interventions to adapt as they engage Apply a systems understanding Start from observable behaviour change Recognise that all interventions have limited influence Embrace different perspectives

18. A brief history of OM 1990s: post-Rio need to demonstrate ‘sustainable’ results 1998: Barry Kibel and Outcome Engineering 1999: Methodological collaboration with projects 2000: Publication of manual in English 2002: Training, facilitation & usage globally 2006: OM Learning Community 2008: CLAMA 2010: East Africa and beyond

21. “Being attentive along he journey is as important as, and critical to, ariving at the destination ” Michael Quinn Patton

22. Three key concepts in OM: 1.Sphere of influence 2.Boundary Partners 3.Outcomes understood as changes in behaviour

23. There is a limit to our influence Project People the project works with/through People who benefit from the project Sphere of control Sphere of influence Sphere of concern

24. There is a limit to our influence Inputs, activities, outputs Outcomes: Changes in behavior Impact: Changes in state Sphere of control Sphere of influence Sphere of concern

25. Participatory research on demonstration farms to develop approaches to drip irrigation Farmers participate in field trials Participating farmers learn how to use drip irrigation equipment Extension workers visit demonstration farms Training of extension workers Publication of performance of different set- ups Increased knowledge of techniques Extension workers promoting drip irrigation Farmers adopting drip irrigation methods Reduced numbers of new wells Greater quantities of groundwater available Source: Terry Smutylo

26. Who are your boundary partners? Programme BeneficiariesStakeholdersBoundary Partners

28. The Problem with “Impact” Impact implies… The reality is… Cause & effect Open system Positive, intended resultsUnexpected positive & negative results occur Focus on ultimate effectsUpstream effects are important Credit goes to a single contributor Multiple actors create results & need credit Story ends when program obtains success Change process never ends Source: Terry Smutylo

29. Focus of Outcome Mapping Outcome Mapping Community ownership increases Program influence decreases InputsActivitiesOutputsOutcomesImpacts

31. ✓ OM is a guide to the journey we take with our partners. We co-create the map. ✓ It focuses on the intention and what happens along the way ✓ The map is not the territory, it shows the route taken ✓ “The only real voyage of discovery exists, not in seeing new landscapes, but in having new eyes” (Marcel Proust) Where is the map?

32. Why? Who? What? How? Vision Boundary Partners Outcomes Challenges, Progress Markers Mission, Strategy Map, Organizational Practices 4 Key Planning Questions

33. Step 1: Vision

34. improved human, social, & environmental wellbeing

35. I have a dream! Martin Luther King, Jr. August 28, 1963

36. A vision statement.. Guides Motivates and inspires Is an ‘accountability-free zone’

37. 37 Vision in graphic form, Nagaland (India)

38. Vision statement in narrative form Local authorities, communities, and international organizations in developing countries in Africa recognize the value of HIV/AIDS intervention as an integral part of social & economic development. Municipal, regional, and national governments actively support HIV/AIDS prevention activities by formulating and implementing effective public health policies. Using research findings, they have developed a comprehensive public health strategy to slow down the infection rate. Formerly marginalized groups (e.g. women and youth) are organized into advocacy groups that can effectively formulate their needs to policy makers. All groups have access to reliable and relevant technical information about HIV/AIDS prevention and are able to make informed choices. In essence, there are healthier, happier, and wealthier communities.

39. Vision facilitation question Imagine that, 5-10 years from now, the program has been extremely successful. Things have improved beyond your most ambitious dreams. What changes have occurred? What (& how) are your intended beneficiaries doing? What are your partners doing? Describe the better world you are seeking.

40. Step 2: Mission

41. The mission is that “bite” of the vision statement on which the program is going to focus.

42. How the program intends to apply its resources in support of the vision The areas in which the program intends to work How the program will support the achievement of outcomes by its direct partners A mission statement describes:

43. Example Mission Statement In support of this vision and on behalf of its donors, the program will work in the areas of research, dissemination, capacity building, & coordination. It will contribute to the production, synthesis, & dissemination of research data, position papers, & other information that will sensitize local & international actors to HIV/AIDS prevention. The program will seek to expand the range of disciplines involved in HIV/AIDS research. It will enhance HIV/AIDS research capacity in order to produce credible information for local, national, & international policy-making & program development. It will promote an interest in HIV/AIDS research among new researchers by providing research fellowships, mentorship, & training opportunities. The program will contribute to the development of linkages between Northern & Southern researchers & encourage partnerships between research organizations, advocates, & decision makers. It will increase its visibility & credibility among the donor community & will convince them of the utility of supporting HIV/AIDS prevention.

44. Your mission is your “business” What do you do to support the vision? What areas do you need to work in? Who are your principle collaborators? How do you work with them?

45. Summary Vision ✓ About the future ✓ Observable ✓ Idealistic ✓ Not about the program Mission ✓ Feasible ✓ Identifies activities and relationships ✓ About the program

46. Exercise : Correcting Vision and Mission Statements

47. 1.Across rural India, women and girls are contributing to and sharing equitably in the benefits of development. 2.They utilize and benefit from appropriate health care, education, food and water security. 3.They are free from violence in the home and in the community. 4.The project seeks to improve their well being through a wide range of interventions. 5.Women are able to earn their own livelihoods, accessing freely the markets, credit, banking and municipal services they need to pursue their economic goals. 6.Villages are fully served by public transport, are well lit at night and have police forces that enforce all laws fully and equitably. 7.The consortium of donors responds favorably to the project’s progress reports and decides to continue funding for a second phase

48. 1.The Swayamsiddha Project works with government, non- government and community-based organizations to strengthen their individual and combined efforts towards improving women’s health and empowerment. 2.The development of women’s self help groups is facilitated by the project, providing women with moral, material and educational support in identifying and taking action to make community and government services more responsive to their health and livelihood needs. 3.Primary and reproductive health care at the local level is directly accountable for, and responsive to, the needs of women and girls. 4.Enrolment of girls in primary schools increases by 35% as a result of project interventions 5.Research is undertaken in the application of ecosystem approaches to human health in agriculture and in the provision of health and sanitation services.

49. Step 3: Boundary Partners

50. Step 3: Boundary partners Outcome mapping focuses on outcomes as changes in behaviour or relationships of those with whom the program or project is working directly (= boundary partners)

51. Boundary Partners are... Those individuals, groups, & organizations with whom a program: interacts directly to encourage change as a contribution to the vision. can anticipate some opportunities for influence engages in mutual learning

52. Who are your boundary partners? Programme Beneficiaries Stakeholders Boundary Partners Strategic partners

53. CIDAIDRCBAIF State NGO State NGO State NGO State NGO State NGO State NGO SHGPolice Community Leaders Families Banks PHCs Swayamsiddha Boundary partners have their own boundary partners - Often many actors and results at different levels

54. Facilitation questions In which individuals, groups, or organizations is your program trying to encourage change as a contribution to the vision? With whom will you work directly? Are you choosing boundary partners because you want to influence the ways they help or influence others? On whose actions does your success depend?

55. Strategic partners Selected for their potential to contribute to the mission A person or group with whom the program works directly to achieve the mission, without necessarily wanting to change the partner’s behaviour as part of the mission E.g. Donor agency, contracted service providers, NGOs doing similar work, media agencies

56. Facilitation questions In which individuals, groups, or organizations is your program trying to encourage change as a contribution to the vision? With whom will you work directly? Are you choosing boundary partners because you want to influence the ways they help or influence others? On whose actions does your success depend?

57. Screening of boundary partners A program has normally not more than 4 or 5 types of boundary partners What if too many BP’s? What if you are not sure whether it is really a BP? – Some boundary partners might be boundary partners of boundary partners! – Discuss how the boundary partners are possibly contributing to the vision – Can we realistically influence the boundary partners? – Where will we put most efforts and resources? – Sometimes similar actors can be clustered as one type of boundary partners – Who are your ultimate beneficiaries / target group? There are not your boundary partners – Maybe some partners are actually strategic partners?

58. Exercise : Boundary partners

59. Boundary partners: exercise Read the handout with the case of the VVOB teacher education and vulnerability programme On the basis of the case of the VVOB programme (see handout), try to position the various actors in (or out) of the various influence spheres on the flip chart provided. Solution: see handout! 2-11-201559

60. Developing an actor focused TOC (case of VVOB Zimbabwe) Inputs, activities, outputs Sphere of control Sphere of influence Sphere of interest Final beneficiaries Pupils College admins Student peer educator clubs Student support structures Boundary partners = Direct target groups Staff devpt committees Earch Childhood depts VVOB support team College students College Lecturers Indirect target groups OutcomesImpact Unicef DGD PLAN International Local resource persons

61. Step 4: Outcome Challenge

62. Outcome Challenge Describes behaviour of a single boundary partner Sets out the ideal actions, relationships activities Describes the boundary partner’s contribution to the vision

63. Facilitation questions Ideally, in order to contribute to the vision, what would the boundary partner be doing? With whom would they be interacting? How could this boundary partner contribute maximally to the vision?

64. Women’s self help groups are taking action to make community and government services more responsive to the health and livelihood needs of women and girls. They influence banks, police, health and social service providers, local officials and state and national government agencies in relationships of mutual respect and joint action to improve women’s well being. Women’s self help groups arrange bank loans for members and for life skills training for girls to be included in the school curriculum. They influence local, state and national government policies and expenditures on community improvement and transportation and support women candidates to run for election to local government office.

65. Step 5: Progress Markers

66. (Deep transformation) (Active engagement) (Early positive responses) Love to see Like to see Expect to see Progress Markers

67. Love to see Like to see Expect to see Like to see Expect to see Like to see Expect to see

68. Why Graduated Progress Markers? Articulate the complexity of the change process Allow negotiation of expectations between the program and its partners Permit early assessment of progress Encourage the program to seek the most profound transformation possible Help identify mid-course improvements

69. ? Greater awareness… Empowered women… Community ownership… Reduced conflict… Increased collaboration… Governmental commitment… Gender sensitivity… Equal access… Budgetary transparency… Active participation… Poverty alleviation… Strengthened capacity… How can we measure:

70. Examples Project expects to see Tanzania Forest Service – Consult with communities on the new forest policy and the national REDD strategy; – Develop management plans for all Nature Reserves; – Continue a policy that is supportive of participatory forest management.

71. Examples Project would like to see Tanzania Forest Service – respond promptly to information from communities about illegal activities and other forest management issues and take action to resolve the issue. – incorporate recommendations from communities and civil society organistions in the revised forest policy and the national REDD strategy; – approve voluntary social and environmental timber standards; – implement and monitor management plans for nature reserves and other high biodiversity forest reserves with the participation of communities; – approve independent forest monitoring and cooperate fully in its implementation; – incorporated Biodiversity indicators in NAFORMA

72. Examples Project would love to see Tanzania Forest Service – successfully advocate for wider application of good timber standards throughout Tz gov’t systems. – consistently and effectively apply the Forest Act and regulations within the timber and charcoal trade thereby increasing the revenue that they capture and reducing illegal timber harvesting and charcoal production; – incorporated social and environmental standards in revised harvesting regulations and other documents related to the national forest programme; – Applying good social and environmental practices in respect of reserve management

73. Facilitation questions How can the programme know the boundary partner is moving toward the outcome? What would they be doing? What milestones would be reached as the boundary partner moves towards their intended role in contributing to the vision?

74. Progress Marker Checklist Each Progress Marker: Describes a changed behaviour by the boundary partner Can be monitored & observed As a set, Progress Markers: Are graduated from preliminary to more profound changes in behaviour Describe the change process of a single boundary partner

75. Exercise : Arranging Progress Markers

76. Step 6: Strategy Maps

77. Strategy Maps For each Boundary partner: What will the Project do to SUPPORT desired changes in the Partner towards the Outcome Challenge? Diverse range (an array) of ACTIVITIES: -Aimed at Partner -Aimed at Partner’s environment -Cause, Persuade, Support

78. Strategy Map causalpersuasivesupportive Partner Environment

79. 6 types of support strategies causalpersuasivesupportive I aimed at individual boundary partner direct Influence (e.g. funding, prepare a report, …) arouse new thinking; build skills, capacity (e.g. skill enhancement, methodological workshops, training) on-going support supporter/mentor who guides change over time,involvement is more frequent and sustained, nurturing for self- sufficiency (e.g. program member who provides regular guidance and input, coaching, expert, …) E aimed at boundary partner’s environment alter the physical or regulatory environment (e.g. technical transfer, policy change, internet access, terms of reference, rules, guidelines) broad information dissemination/ access to new info (e.g. Radio, TV, website, publications, conferences, workshops) create / strengthen peer networks partners working together and collectively supporting each other) (e.g. research network, …)

80. Example strategy map: for extension agents 80 CausalPersuasiveSupportive Directed to the boundary partner  Provide research and training funds  Run technical training workshops for extension agencies and activities (e.g. ISFM)  Coordinate training on farmer participatory action research approaches to extension agencies, service providers and research teams  Conduct training on protocols development and project monitoring  Contract a farmer-training specialist to work with extension on farmer mobilization, action research, data collection and documentation  Conduct quarterly monitoring and review workshops Directed to the environment of the boundary partner Building model storage facility for grain and backstopping  Develop extension materials and policy brief on adaptation to climate change  Facilitate establishment of exchange forums for sharing of information, knowledge and experience among partners  Establish a network of extension agents and researchers that facilitate knowledge and information sharing  Engage researchers and policy makers as conveners for discussions on climate change impact and alternative measures

81. Exercise : Strategy Map

82. Exercise on strategy maps Instructions Your table will be given a number of cards with activities written on them. Discuss which type of strategy each of the activities represents. The Outcome Challenge has been provided so that you are able to think about what the program is contributing to. Select someone from your table to plot the activities in the appropriate section of the strategy map on the wall. Be prepared to explain your reasoning for plotting the different strategies as I-1, I-2, I-3, E-1, E-2, or E-3. !Remember, a strategy map describes what the program will do to try to encourage change(s) in a boundary partner. It breaks down depending on whether they are aimed at the individual boundary partner or the environment and whether they are causal, persuasive, or supportive. Time: 30 minutes

83. mission vision Girls & Women Community Leaders Women’s Self Help Groups Families Police State NGOs Banks Public Health Clinics Strategic Partners Strategies Project’s Outcomes Boundary Partners BP’s outcomes BAIF

84. Step 7: Organisational Practices

85. Organizational Practices How does your team or organization stay relevant, viable and effective?

86. You: keep learning foster creativity & innovation seek better ways to assist your partners maintain your niche maintain high level support build relationships

87. 8 practices 1.Prospecting for new ideas, opportunities, and resources 2.Seeking feedback from key informants 3.Obtaining the support of your next highest power 4.Assessing and (re)designing products, services, systems, and procedures

88. 8 practices 5.Checking up on those already served to add value 6.Sharing your best wisdom with the world 7.Experimenting to remain innovative 8.Engaging in organizational reflection

89. M&E planning and journals

90. 5 kinds of M&E Information Program Partner outcomes (behaviour changes in the partners) implementation (interventions by the program) relevance & viability (actions of the program) C o n t e x t u a l I n f o r m a t I o n State, status or situational data Strategies

91. M&E Planning Worksheet What information will be collected? How will it be collected, from what sources and when? Who will collect it? Who will make sense of the info & how? Who will manage the collection-to- usage processes? What resources are needed & who contributes?

92. Outcome journal What, when, who, how much Contributing factors Sources of evidence Unanticipated change Lessons, required changes, action

93. Strategy journal What did you do, with whom, when (How) did it influence change Outputs Suggested changes/follow up Lessons

94. Performance journal Progress against internal commitments – e.g. organisational practices Reviewing the logic of the programme

95. Journal for Monitoring Outcomes

96. Name of the boundary partner Work dating from/to: Name(s) of the person(s) who compiled the journal: Outcome Challenge:. Progress MarkersRemarkable facts, what happened Follow up / corrective measures Narrative of a success story: Unanticipated changes : Which support strategies from the ICCO alliance where helpful or require further follow-up or action during the next term? Contributing or limiting factors and actors towards achievement of progress markers: : Summary of lessons learned/recommendations : Outcome Journal

97. Thank you! For more info on OM: Visit the OM learning community: www.outcomemapping.ca Contact: Simon Hearn s.hearn@odi.org.uk

98. Sharing information to improve evaluation

99. outcomemapping.ca