Impact Pathways: An Approach for Understanding, Fostering and Evaluating Research-for-Development Outcomes Boru Douthwaite, Technology Policy Analyst Sophie Alvarez, Consultant Simon Cook, Leader BFPs Rick Davies, M&E Specialist, Pamela George, CPWF Program Manager, John Howell, M&E Specialist, Ronald Mackay, Professor Emeritus, Jorge Rubiano, National University of Colombia CIAT Seminar, 1 st November 2006

Impact Pathways Matter

How change happens “Improvements in poverty alleviation, food security and the state of natural resources result from dynamic, interactive, non-linear, and generally uncertain processes of innovation.” EIARD, 2003 EIARD represents a group of European donors 15 EU Countries plus Norway and Switzerland

Impact Pathways Approach People plan and implement projects on the basis of their change models - their implicit theories about how the world works If you can improve these theories you can improve the practice, making impact more likely Impact Pathways Approach – A participatory approach for: 1.Making practitioners’ theories explicit about how they will achieve adoption and impact (impact pathways, program theory); 2.Improving these theories 3.Using those models / frameworks for M&E and impact assessment 4.As a result, contributing to project and program “adaptive management” and thus likelihood of impact

History and Current Work Past Work in Nigeria on Striga –Douthwaite, B., T. Kuby, E. van de Fliert and S. Schulz Impact Pathway Evaluation: An approach for achieving and attributing impact in complex systems. Agricultural Systems 78 pp –Douthwaite, B., Schulz, S., Olanrewaju, A., Ellis-Jones, J Impact pathway evaluation of an integrated Striga hermonthica control project in Northern Nigeria. Agricultural Systems. Published on-line Current Work (since Oct. 2005) –CPWF-supported, CIAT-led impact assessment project in 9 river basins ($900,000) –Douthwaite, B., Alvarez, B.S., Cook, S., Davies, R., George, P., Howell, J and Mackay, R The Impact Pathways Approach: A Practical Application of Program Theory in Research-for-Development. For submission to an Evaluation Journal Future Work –EU-funded, Wageningen-led “eco-system approach for co-innovation of farm livelihoods” project (Euro 1.8 million with 6 PhDs and 4 PostDocs) –Phase II of Knowledge Sharing for Research Project (with Simone Staiger) –PRGA INIS Project (CIAT and CIP led) See

Impact pathways – two conceptualizations…. Logic model Network maps

Impact Pathways Two complementary conceptualizations of a project, a program or an organization’s impact pathways: 1.A visual description of the causal chain of events and outcomes that link outputs to the goal (logic model); and 2.Network maps that show the evolving relationships necessary to achieve the goal Implementing organizations; stakeholders; ultimate beneficiaries Shows the project rationale; its logic Foundation of ex-ante (and ex-post) impact assessment

Foundations of the IP Approach Synthesis of concepts and tools from: –Program Evaluation Renger and Titcomb (2002) – problem trees Chen (2005) – program theory Mayne (2004) - performance stories –Innovation histories Douthwaite and Ashby, 2005 –Appreciative Inquiry Whitney and Trosten-Bloom, 2003 –Social network analysis Cross and Parker, 2004

The Process of Developing Impact Pathways

Participatory Development of Impact Pathways

Example of a Problem Tree

Turning a problem tree into an objective tree

The Process of Developing Impact Pathways – The Workshop

Scaling Out and Scaling Up Scaling up - an institutional expansion, from adopters and their grassroots organizations to policy makers, donors, development institutions Scaling out - spread of a project outputs (i.e., a new technology, a new strategy, etc.) from farmer to farmer, community to community, within the same stakeholder groups

Develop a vision of project success two years after the project ends Work in project groups Take 5 minutes to individually answer the question –You wake up 2 years after your project has ended. Your project has been a success and is well on its way to achieving its goal. Describe what this success looks like to a journalist: What is happening differently now? Who is doing what differently? What have been the changes in the lives of the people using the project outputs, and who they interact with? How are project outputs disseminating (scaling-out)? What political support is nurturing this spread (scaling-up)? How did that happen? Discuss and develop a common vision Keep it realistic Workshop

Example of a Vision

The Process of Developing Impact Pathways – The Workshop

Develop a project timeline from when your project started until 2 years after it will end Build a timeline of activities, outputs and outcomes that take you from the beginning of the project to achieving the vision It is a story of adoption of project outputs (scaling-out) and the political support that helps it along (scaling-up) Workshop

Example of a Timeline

The Process of Developing Impact Pathways – The Workshop

Family ties Friendship ties Workplace ties

Today’s tasks….. 1.Identify relevant actors & relationships 2.Develop network diagrams for Your project now Residual network 2 years after project has finished 3.Identify key extension (scaling out) and political support (scaling up) linkages 4.Identify differences between the two networks and discuss implications Workshop

Differences between the MUS project’s maps

The Process of Developing Impact Pathways – After the Workshop

Impact Narrative Text description of the project impact pathways Achieves the integration between the logic and network models Helps with colligation (tracing of logical steps, Roberts, 1996), making hidden assumptions explicit Helps with the plausibility of ex-ante impact assessment

IP Logic Model (Draft 1)

IP Logic Model (Final)

Network Maps

Separate relationships

Final maps based on answers

The theory behind the IP approach “Stakeholders' implicit theories are not likely to be systematically and explicitly articulated, and so it is up to evaluators to help stakeholders elaborate their ideas.” (Chen, 2005, p. 14)

Impact of IPs 1 From Workshops –“I will use Impact Pathways in future design of projects” –“The dynamics of the networks is useful to envision the future” – “It helps show gaps” –“It is good for planning” –“It helps explain impact of my project” –“Constructing impact pathways should not be one- shot” –“The impact pathways should be a living document”

Impact of IPs 2 Significant Change Stories resulting the Volta IP Workshop 1.Locally-organized IP Workshop to build basin program 2.Exploiting an opportunity for political lobbying 3.Network concepts hybridized with influence mapping to become main PhD research methodology 4.Problem and objective trees used to explain project to primary stakeholders

Impact of IPs 3 Science Council review of CPWF 2007 – 2009 Medium Term Plan –‘The CPWF has introduced the use of “objective trees” at the MTP project and CP level, a useful and innovative complement to the MTP logframe. In addition to providing a useful overview, the process of preparing these flow charts has clearly helped the CP provide the necessary focus, clarity and cohesion that now exists in the research plans at all levels.’

Impact Pathways Evaluation Monitoring and evaluating progress along impact pathways –Regularly updating objective tree, timeline and network maps –Most Significant Change to pick up unexpected consequences –Provides the information needed for “adaptive management” Impact Pathways Evaluation = Action research –Is publishable; raises the status of M&E

IPs and ex-post Impact Assessment Ex-post impact assessment should identify and describe (EIARD, 2003): –The concept or model of innovation; –The logic model underlying a project or program –In other words, the normative and causative program theory

Research Questions What types of network should R4D programs attempt to foster to achieve impact? Food security, poverty alleviation, improved health, environmental security How does improving the congruence between implicit and explicit stakeholder theories improve research-for-development effectiveness? Are impact pathways generalizable? –Can one project’s impact pathways help planning and implementation of others?

IPs – the elevator conversation What do you do? –Work to mainstream an impact orientation in agricultural R4D projects and programs What is impact orientation? –That the people implementing projects are clearer about how their research will make a difference, and take responsibility that it does How do you do that? –Work with practitioners to make explicit their impact pathways –Carry out research to understand which impact pathways work, when

Summary Change Models matter: People plan and implement projects on the basis of their change models - their implicit theories about how the world works If you can improve these theories you can improve the practice, making impact more likely The Impact Pathways Approach – A participatory approach for: 1.Making practitioners’ theories explicit about how they will achieve adoption and impact (impact pathways); 2.Improving these theories 3.Using these models / frameworks for M&E and impact assessment IP Evaluation = Action research 4.As a result, contributing to project and program “adaptive management” and thus likelihood of impact A work in progress, including research on network structures and impact pathways

Impact Pathways Matter