1. Choosing and using Outcome Mapping well
OM Lab 2012
Keynote Address
Patricia Rogers
Good morning. It is an honour to be here with you this week to explore Outcome Mapping together. How appropriate it is to be meeting in Beirut – where East meets West, and where change and rebuilding is underway.
I’m excited to be part of this historic occasion where we will hear from each other about our experiences of Outcome Mapping,
learn from the past and present, and together shape the future.
Before we move to individual cases, I’d like to share some thoughts about what we are seeking to learn from each other. So this morning I’m going to discuss two big questions, some examples and a big idea.
Let’s start with the questions. I’m sure we have brought many questions with us to Beirut. I think there are two big ones.

2. When should we choose Outcome Mapping?
The first one is about choosing Outcome Mapping.
There are lots of ways of doing evaluation. What is it about the situation that makes it appropriate to choose Outcome Mapping? Does Outcome Mapping suit particular purposes? Particular options in terms of who will have control over the evaluation and who will conduct it? Is Outcome Mapping more or less suitable in combination with other particular methods or research designs?
I am going to be listening to the cases to understand why people chose Outcome Mapping for their evaluation, and what we might learn about making this choice.

3. When should we choose Outcome Mapping?
there is a lot of guidance about what impact evaluation looks like
Simon Kneebone ‘Show Me The Change”

4. When should we choose Outcome Mapping?
there is a lot of guidance about what impact evaluation looks like
Simon Kneebone ‘Show Me The Change”

5. How do we use Outcome Mapping well?
The second question is about using Outcome Mapping well.
What are the skills needed to do it well? How do we know if we have those skills in our team? How do we develop them?
Outcome Mapping, like other evaluation methods, is often done in ways that are adapted from its original form. When we’re planning what we will actually do, there are many different ways of proceeding.
But at what point do these adaptations stop being Outcome Mapping? What are the essential elements for it to still be Outcome Mapping?
Jessica Dart, who with Rick Davies developed and popularised another approach to evaluation - Most Significant Change – has talked about how she has seen all sorts of things being called Most Significant Change, even when essential elements of the approach are not present.
When we are adapting Outcome Mapping, what are reasonable adapations? What are necessary adaptations to our situation? How do we know if we have lost the essence?
To answer this, I think we need to learn from the process of knowledge translation in evidence-based policy. We need to be able to distinguish between the theories of change we are trying to activate and the theories of action about what we will do. Let me illustrate this.

6. Does it have to be pizza?
A few years ago I was working on an evaluation of a middle school project that aimed to support kids who were in danger of getting into trouble and dropping out. The project involved mentoring, homework help and referral to specialist services.
One of the elements of the project was a pizza night at the start of the school year, involving kids, parents and teachers where they could meet and start establishing relationships between home and school, and have some shared, fun experiences.
Another school had heard about this program and was planning to implement it in the new school year. I was in the office when a phone call came in from them. They were all set to go for the year. They just had one question. Did it have to be pizza?
This is a common issue when we seek to replicate success – confusing the details of the action with the details of the underpinning theory of how change comes about. It is helpful to have a very clear picture of how we think change will come about, and then adapt the specific activities to be locally suitable. So, changing from pizza to tacos or to falafel might be a useful local adaptation. But changing the feel of the event from a fun, shared event to a serious lecture on student behaviour would not be – even if they then served pizza.
Before moving to

7. Example 1: Stepping away from Outcome Mapping

8. Example 2: Seeking out Outcome Mapping

9. Evaluation

10. Better projects, programs, policies
Evaluation

11. Better projects, programs, policies??
Changed
behaviour –
decision, actions
Better projects, programs, policies
Evaluation

13. Outcome Mapping can provide.. Which can contribute to…
Accurate, timely and credible information that identifies and explains poor performance
Decisions and actions to solve problems

14. Outcome Mapping can provide.. Which can contribute to…
Accurate, timely and credible information that identifies and explains good performance
Decisions and actions to reinforce, repeat, replicate expand good practice

15. Outcome Mapping can provide.. Which can contribute to…
Clear signals about what is important
More focus (attention and resources) on priority issues

16. Outcome Mapping can provide.. Which can contribute to…
Clear accountability and consequences
Increasedmotivation to find ways to improve performance

17. Outcome Mapping can provide.. Which can contribute to…
Increased ability to generate and use information
Ongoing capacity to learn

18. Outcome Mapping can provide.. Which can contribute to…
Compelling narratives of achievements
Improved support and resourcing

19. Outcome Mapping can provide.. Which can contribute to…
Accurate, timely and credible information that identifies and explains poor performance
Decisions and actions to solve problems
Accurate, timely and credible information that identifies and explains good performance
Decisions and actions to reinforce, repeat, replicate expand good practice
Clear signals about what is important
More focus (attention and resources) on priority issues
Clear accountability and consequences
Increasedmotivation to find ways to improve performance
Increased ability to generate and use information
Ongoing capacity to learn
Compelling narratives of achievements
Improved support and resourcing