© 2011 Regents of the University of Minnesota. All rights reserved. Ripple Effect Mapping: A Tool for Evaluating the Impacts of Complex Interventions Presented at the 19 th Annual MESI Spring Training Conference Scott Chazdon, Ph.D., Evaluation and Research Specialist Extension Center for Community Vitality Kit Alviz, M.A., Research Fellow, Extension Center for Family Development

© 2011 Regents of the University of Minnesota. All rights reserved. Session Overview Background – 20 minutes Demonstration – 40 minutes Want to try mind mapping? Download from Discussion – 45 minutes

© 2011 Regents of the University of Minnesota. All rights reserved. Session Learning Objectives Describe the elements of the Ripple Effect Mapping process. Identify the skills needed to facilitate a Ripple Effect Mapping session. Give examples of the types of situations in which Ripple Effect Mapping would make sense. Produce the first stage of a Ripple Effect mind map (optional). Weigh the benefits and limitations of Ripple Effect Mapping as an evaluation tool.

© 2011 Regents of the University of Minnesota. All rights reserved. Ripple Effect Mapping Purpose – to better understand intended and unintended results of a program, intervention or collaborative for individuals, groups, sectors or communities. Can be post-program (more summative) or mid-program (more developmental)

© 2011 Regents of the University of Minnesota. All rights reserved. Direct vs. Indirect Impacts Collaboratives and high engagement programs often build social capital, but dont take credit for it. People do not act in isolation -- strengthened social capital is a necessary pre-condition for other impacts Other impacts may occur that were not foreseen in program theory

Mind Mapping – Radiant Thinking Pictorial Method –Note taking –Brainstorming –Organizing –Problem solving –Evaluation Image: Mindmap, Graham Burnett, For more on mind mapping, see Buzan, T. (2003). The mind map book. London: BBC Books.

© 2011 Regents of the University of Minnesota. All rights reserved. Concept Mapping (Trochim, 1989) Mind Mapping (Eppler, 2006) Outcome Mapping (Outcome Mapping Learning Community, 2011) Participatory Impact Pathway Analysis (Douthwaite et al, 2008) Most Significant Change (Davies 2005) Appreciative Inquiry (Preskill & Catsambas, 2006) Ripple effect mapping: related approaches

© 2011 Regents of the University of Minnesota. All rights reserved. How Does it Work? Identify the intervention Schedule the event and invite participants Group mapping session held Follow-up interviews Cleaning, Coding, Analysis

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Method –Identify the intervention High engagement program or position Cross-sector initiative Collaboration –Invite stakeholder group Participants Non-participant stakeholders 12 to 20 participants Two moderators

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Method –Appreciative Inquiry interview Conducted among pairs of participants Examples of questions: –Tell me a story about how you have used the information from the program? –Is there anything that resulting from the program that you are proud to share? –List an achievement or a success you had based on what you learned.

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Method –Starting the Map On wall or using Mind Mapping software with data projector Floating topics generated from Appreciative Inquiry –Different approaches

© 2011 Regents of the University of Minnesota. All rights reserved. - Building the Map Theming -Group identifies which items are closely related -Group generates initial theme names -Floating topics moved and organized Rippling -Group cross-validation -Potential for probing using the Community Capitals Framework THEMING AND RIPPLING

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Examples of Contexts where REM makes sense Kit: State systems building to improve quality and quantity of child care Scott: Community leadership development

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Demonstration of Mapping Process Think back to your first program evaluation experience. –Is there anything that you gained from this experience that you are proud to share? –List an achievement or a success you had based on what you learned or who you met. –Did the experience lead to other important developments in your personal or professional life? Floating topics Beginning to categorize

Example: Ripple Effect Map of Community Gardening in Frogtown & Rondo neighborhoods

© 2011 Regents of the University of Minnesota. All rights reserved. Example: Ripple Effect Map of Hugo, MN Business Retention and Expansion program

© 2011 Regents of the University of Minnesota. All rights reserved. Example: Ripple Effect Map of Child Care Expansion Initiative

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Cleaning, Coding, Analysis –Organize map to better identify pathways or combine pathways –Download data to Excel for coding –Code using relevant thematic framework and type of outcome KASA = something learned Behavior change = action taken Impact = change in system –Follow-up interviews if more clarity is needed

The Community Capitals Framework (Emery and Flora, 2008)

© 2011 Regents of the University of Minnesota. All rights reserved. RIPPLE EFFECT MAPPING Coding Demonstration

First order (core outputs) Second order ripples Third order Fourth order Human capital effects (knowledge and behavior change) Social capital effectsCivic effects Financial effects Built capital effects Health, Food and Nutrition Effects Cultural effects Natural environment effects Market the City of Hugo City identity workshop -XX How to attract residents and biz (coninuing work)XX Have identified key attributes about the CityXXX Create, Coordinate, and Encourage Events New position at City for park & rec. planningXXX ~10 new recreation programsXX Hanifl Fields attracted over 20,000 kidsXX? Entrepreneurial BootcampXXX Businesses have used City resourcesX? Provide promotion opps. for bizX Coupons at football tourneyX Host Business and Breakfast Workshops 5 business breakfasts were heldXXX attendees Lots of business networkingX?? Address Highway 61 Access Issues and Improve Downtown Hugo Installed a traffic light at 61 and 147thX Removed 4 blighted bldgs on 61X Coding Example

Reporting Example (Hugo BR&E program)

Reporting Example (Child Care Expansion) Ripples for Theme 1: Updating regulations Individuals volunteer their time and energy to Health and Safety Regulations Work Group No funding This is their passion; many have been working in the trenches for over 30 years Passion to make child care settings as safe as possible Prevention and keeping parents aware Grounded in the reality of day to day Children that fell through the cracks- reason for being here White papers, living document that can be used daily Completed and submitted Safe Sleep Paper to Dept. of Social Services Community Care Licensing Parent group member presented the paper Within one month, the Licensing Division responded and said they would start moving on the changes Babies will be safer when they sleep Appendix – Ripple Effect Tables

© 2011 Regents of the University of Minnesota. All rights reserved. Benefits Simple and cheap tool Captures impacts of complex or evolving work Captures intended and unintended impacts Participatory and appreciative approach that engages stakeholders Group validation of results

© 2011 Regents of the University of Minnesota. All rights reserved. Limitations Risk of bias in participant selection and data collection Participants may not have complete information about a program or program outcomes Potential for inconsistency in implementation

© 2011 Regents of the University of Minnesota. All rights reserved. Suggestions Use same facilitator, recorder and mapper Develop a facilitator guide with ideas for probes. Make decision prior to mapping whether to use a pre-existing framework as probes during group interviews May need to recognize that one organization isnt trying to take all credit for all change It is important to probe for negatives

© 2011 Regents of the University of Minnesota. All rights reserved. Lessons Learned Thus Far Find the right balance between breadth and depth Schedule the event along with another activity Put much effort into recruitment and explaining the process Choose a good setting – not too informal Use external facilitators, not program staff

© 2011 Regents of the University of Minnesota. All rights reserved. Q&A and Discussion

© 2011 Regents of the University of Minnesota. All rights reserved. References Baker, B., Calvert, M., Emery, M., Enfield, R., & Williams, B. (2011). Mapping the impact of youth on community development: What are we learning? [PowerPoint slides]. Retrieved from pdf pdf Buzan, T. (2003) The Mind Map Book. London: BBC Books. Douthwaite, B., Alvarez, S., Thiele, G., & MacKay, R. (2008). Participatory impact pathways analysis: A practical method for project planning and evaluation. ILAC Brief 17. Emery, M., & Flora, C.B. (2006). Spiraling-up: Mapping community transformation with community capitals framework. Community Development: Journal of the Community Development Society 37(1), Eppler, M.J. (2006). A Comparison Between Concept Maps, Mind Maps, Conceptual Diagrams, and Visual Metaphors as Complementary Tools for Knowledge Construction and Sharing. Information Visualization 5: Hansen Kollock, D.A., Flage, L, Chazdon, S., Paine, N., and Higgins, L. (2012). Ripple Effect Mapping: A Radiant Way to Capture Program Impacts. Kollock, D. A. (2011). Ripple effects mapping for evaluation. Washington State University curriculum. Pullman, WA. Outcome Mapping Learning Community website. (2011). Preskill, H. & Catsambas, T.T. (2006). Reframing evaluation through appreciative inquiry. Thousand Oaks: Sage Publications.

© 2011 Regents of the University of Minnesota. All rights reserved. Contact information Scott Chazdon, Ph.D. Evaluation and Research Specialist Center for Community Vitality Kit Alviz, M.A. Research Fellow Center for Family Development

© 2011 Regents of the University of Minnesota. All rights reserved. The University of Minnesota is an equal opportunity educator and employer. This PowerPoint is available in alternative formats upon request. Direct requests to Thank you!