RIPPLE EFFECT MAPPING (REM) February 11, 2015 Scott Chazdon, PhD, University of Minnesota Alyssa Chiumento, University of Minnesota Gretchen Nicolls, Twin Cities LISC

Twin Cities: Building Sustainable Communities Target Areas Evaluating Impact: Qualitative / Quantitative Moving Beyond the Gap: An Assessment of Neighborhood Health (Quantitative) Ripple Effect Mapping: Creating a Narrative of Change (Hybrid) BSC Evaluation: Overall and Site Specific Reports (Qualitative) Gretchen Nicholls, Twin Cities LISC

Today’s speakers Scott Chazdon, PhD Evaluation and Research Specialist Center for Community Vitality University of Minnesota Extension Alyssa Chiumento Community-Based Research Assistant The Center for Urban and Regional Affairs University of Minnesota Gretchen Nicholls Program Officer Twin Cities LISC

Webinar Overview What is Ripple Effect Mapping? Why REM? How does it work? Examples from Twin Cities LISC What does it offer as an assessment tool? Slide: Scott

Why Ripple Effect Mapping? The challenge of capturing impact Limitations of traditional evaluation methods Slide: Scott The challenge of capturing impact Socially complex interventions Requiring collaboration among stakeholders or sectors Highly emergent/evolving contexts Limitations of traditional evaluation methods

Why Ripple Effect Mapping? Purpose: To better understand intended and unintended results Post-program as part of impact evaluation Part of developmental evaluation Slide: Scott Purpose – to better understand intended and unintended results of a program, intervention or collaborative for individuals, groups, communities and regions. Completed post-program as part of impact evaluation May also be completed during program as part of developmental evaluation

Direct vs. Indirect Impacts Unrecognized social capital People do not act in isolation Unforeseen impacts Slide: Scott Collaboratives and high engagement programs often build social capital, but don’t take credit for it People do not act in isolation -- strengthened social capital is a necessary pre-condition for other impacts Other impacts may occur that were not foreseen in program theory

Mind Mapping and Radiant Thinking Pictorial Method Note taking Brainstorming Organizing Problem solving Evaluation Slide: Scott For more on mind mapping, see Buzan, T. (2003). The mind map book. London: BBC Books. Image: Mindmap, Graham Burnett, http://en.wikipedia.org/wiki/File:Mindmap.gif

How Does it Work? Identify the intervention Schedule the event and invite participants Group mapping session held Follow-up interviews Cleaning, coding, analysis Slide: Scott

Identify the Intervention Not all interventions are right for REM. Are there likely both intended and unintended effects? Is the intervention something people can attribute influence to, or at least identify contributions made? Do people already talk about the “ripples” from the intervention? Slide: Scott

Identify the Intervention Community leadership development Business Retention & Expansion program Tourism Assessment program Engaged scholars program Ramsey County Master Gardeners LISC’s Building Sustainable Communities (BSC) Slide: Scott Examples of contexts where REM made sense Community leadership development Business Retention & Expansion program Tourism Assessment program Engaged scholars program Ramsey County Master Gardeners LISC’s BSC

Appreciative Inquiry Conducted among pairs of participants Examples of questions: Tell me a story about how you have used the information from the program? Is there anything that resulting from the program that you are proud to share? List an achievement or a success you had based on what you learned. “Floating Topics” Slide: Scott Conducted among pairs of participants Participants matched with non- participants Examples of questions: Tell me a story about how you have used the information from the program? Is there anything that resulting from the program that you are proud to share? List an achievement or a success you had based on what you learned. Responses mapped as “Floating Topics”

Theming and Rippling Theming Rippling Group identifies which items are closely related Group generates initial theme names Floating topics moved and organized Rippling Group cross-validation Potential for probing using the Community Capitals Framework Slide: Scott

Twin Cities LISC’s Ripple Effect Mapping Slide: Alyssa

Examples from BSC Blake Road Corridor Collaborative Native American Community Development Institute Slide: Alyssa

The Blake Road Corridor Collaborative Slide: Alyssa

Slide: Alyssa

A Major Theme: Reducing Crime Slide: Alyssa

Native American Community Development Institute Slide: Alyssa

Slide: Alyssa

A Major Theme: Building Power in the Community & in Individuals Slide: Alyssa

Cleaning, Coding, Analysis Organize map Follow-up interviews Excel/Google Drive Sheets Thematic framework and type of outcome Something learned Action taken Change in system Slide: Alyssa Organize map to better identify pathways or combine pathways Follow-up interviews if more clarity is needed Download data to Excel/Google Drive for coding Code using relevant thematic framework and type of outcome KASA = something learned Behavior change = action taken Impact = change in system

Improvement of structures, (e. g. housing, retail, etc Improvement of structures, (e.g. housing, retail, etc.) and infrastructures that contribute to the well-being of organizations, individuals, and communities. Increased private and public wealth that is invested in the well-being of organizations, individuals and households, and communities. Strengthened ability of organizations or communities to protect landscape, air, water, soil, and biodiversity of both plants and animals. Increased ability of organizations or communities to mobilize public engagement or influence the distribution of public resources. Slide: Alyssa Strengthened ability of organizations or communities to support diverse worldviews and to transmit knowledge of spiritual, cultural heritage, artistic expression, and traditions to future generations; recognize and build on cultural strengths; systems accepting that there may be different frames. Strengthened or expanded trust or connections among people, groups, and organizations and their resources. Change in knowledge, attitudes, or skills among organization or community members; leadership.

Coding Example: FRAN Slide: Alyssa

The Blake Road Corridor Collaborative Category of Effects Count of Reported Effects (96 total) Percent of Reported Effects Examples Human 28 29.2% Changed engagement strategy to reach more people. Social 31 32.3% Initiated or revitalized Cops and Kids Fishing and Cops and Kids Shopping Programs. Civic 26 27.1% Expanded National Night Out events; instituted Joint Community Police Partnership (JCPP). Financial 19 19.8% Facilitated improvements, i.e. regional storm water management. Built Influenced siting and design issues regarding Blake Station Park and Ride and Operation and Maintenance facility. Health* 10 10.4% Given Allina funding for gatherings at the garden related to healthy eating and active living. Cultural Held PTO meetings at Westside Village, where many Somali families live. Natural Environment 4 4.2% Environmental remediation of brownfield sites was conducted. Slide: Alyssa Facilitated improvements, i.e. regional storm water management, and facilitated additional future development because of cost savings for potential developers. Held PTO meetings at Westside Village, where many Somali families live; use of pool at Blake School restricted at certain times for women only. * Improvements in the physical, mental, and emotional health and well-being of community residents.

Percent of Reported Effects Native American Community Development Institute Category of Effects Count of Reported Effects (65 total) Percent of Reported Effects Examples Human 23 35.4% Community engagement sessions in gallery generated hundreds of ideas about what is wanted for the community and for the city. Social 28 43.1% It is a safe, inviting place; when people need a place to go, they know to go to NACDI. Civic During the Make Voting a Tradition Campaign, we trained leaders in the community about getting out the vote. Financial 4 6.2% NACDI is pulling in more financial support for all sorts of projects. It is a contact list for the community and a pipeline to resources outside of it. Built 5 7.7% Influenced re-siting of the 21st Street Station by leveraging our prior work and our credibility. Health 2 3.1% NACDI makes me feel good. Cultural 17 26.2% Indigenous People's Day established in 2014 with the help of NACDI staff and Native American Indian community members. Natural Environment 1 1.5% Picking up trash with Jay in Phillips. Slide: Alyssa Human Three community engagement sessions in gallery generated hundreds of ideas about what is wanted for the community and for the city; now participation is more constructive; the community iterates its own points of view into the power structure; we have a better understanding of organization structures. Social It is a safe, inviting place; when people need a place to go, they know to go to NACDI; relationships broadened thinking about how NACDI can be involved in the community and how to make more partnerships possible; NACDI is building connections with the Seward neighborhood. Civic During the Make Voting a Tradition Campaign, we trained leaders in the community about getting out the vote; until NACDI joined, there was no indigenous organization within the Alliance for Metropolitan Stability; made the mayoral forum listen to community, held forum on own terms, because we had own power already built. Financial NACDI is pulling in more financial support for all sorts of projects. It is a contact list for the community and a pipeline to resources outside of it; not the usual trend, but young artists supporting young artists is a really wonderful thing. Built Traveling along Franklin Ave, banners and other physical evidence reinforce presence of NACDI in South Minneapolis and instill sense of pride; influenced re-siting of the 21st Street Station by leveraging our prior work and our credibility.

Program Comparisons Human Social Civic Financial Built Health Cultural Natural Environment Total BRCC 28 31 26 19 10 4 96 29.2% 32.3% 27.1% 19.8% 10.4% 4.2% NACDI 23 5 2 17 1 65 35.4% 43.1% 6.2% 7.7% 3.1% 26.2% 1.5% Slide: Alyssa

Community Capitals Highlights Cultural Capital Civic Capital Social Capital BRCC Held PTO meetings at Westside Village, where many Somali families live; use of pool at Blake School restricted at certain times for women only. Expanded National Night Out Events; instituted Joint Community Police Partnership (JCPP). Initiated or revitalized Cops and Kids Fishing and Cops and Kids Shopping Programs. NACDI Indigenous People's Day established in 2014 with the help of NACDI staff and Native American Indian community members. Relationship between community and established power structure has changed; politics feels more visible and accessible; the community iterates its own points of view into the power structure. Until NACDI joined, there was no indigenous organization within the Alliance for Metropolitan Stability. NACDI brought an Indian perspective. Slide: Alyssa

Benefits and Limitations Simple tool Complex work Captures consequences For people familiar and unfamiliar with program Cathartic experience Clarify misunderstandings Potential for bias Incomplete information Inconsistency implementation Not sufficient alone Slide: Scott Benefits Simple and cheap retrospective tool Captures impacts of complex or evolving work Captures intended and unintended impacts Participatory and appreciative approach that engages stakeholders Group validation of results Great supplement to quantitative assessments Wonderful summary of a program to people who are unfamiliar with it. Impetus for a program, its processes for achieving success, and both its intended and unintended consequences in a clear and anecdotal matter Excellent visual tool, they can be well utilized on an organization’s website or social media platforms Useful also to those who are familiar with the program Participants to identify and reconcile any inconsistencies between an organization’s mission statement and its present actions They allow for a re-examination of an organization’s strategy. The REM process can be a cathartic experience for participants, assuring them of their efforts, identifying real effects of their work, and reaffirming their dedication to their organization’s mission. The REM process can highlight participant’s perceptions and understandings, and perhaps equally importantly - it can highlight misperceptions and misunderstandings. Limitations Risk of bias in participant selection and data collection Participants may not have complete information about a program or program outcomes Potential for inconsistency in implementation Not intended as the only evaluation method to be used with a program

Suggestions Facilitator, recorder and “mapper” “Probing” for responses Community capitals Recognition or credit for all change Probe for negatives Slide: Scott Use same facilitator, recorder and “mapper” Understand you will be “probing” for responses – think about some of those probes beforehand Make decision prior to mapping whether to use community capitals as probes during group interviews May need to recognize that one organization isn’t trying to take all credit for all change It is important to probe for negatives

Lessons Learned Thus Far Balance between breadth and depth Scheduling Recruitment and explanation Setting External facilitators Differences in maps Facilitation and evaluation For more information: http://blog-ripple-effect-mapping.extension.umn.edu/ Slide: Scott Find the right balance between breadth and depth Schedule the event along with another activity Put much effort into recruitment and explaining the process Choose a good setting – not too informal Use external facilitators, not program staff Maps can differ significantly; Each organization approaches the process with its own history, experiences, and biases in mind, and as a result, each map will reflect those unique conditions. What does it take? At the same time, the facilitators must use evaluation research skills such as group interviewing and analyzing qualitative data "on the fly," engaging the participants in the process of generating and refining thematic categories, and discovering causal pathways among the "ripples" or effects they report. Finally, it is helpful for session facilitators to have strong technology and typing skills to make the most effective use of the mind mapping software possible.

Contact Information Scott Chazdon, PhD Evaluation and Research Specialist Center for Community Vitality University of Minnesota Extension E-mail: schazdon@umn.edu Alyssa Chiumento Community-Based Research Assistant The Center for Urban and Regional Affairs University of Minnesota E-mail: chium001@umn.edu