Evaluating Advocacy: A Model for Public Policy Initiatives

Slide 2 Innovation Network, Inc. Who We Are: Innovation Network National nonprofit organization Committed to evaluation as a tool for empowerment Work with nonprofits at the grassroots level Build evaluation capacity of nonprofits and funders so they can better serve their communities Practice a participatory approach Innovation Network(202) K St. NWwww.innonet.orgwww.innonet.org Washington, DC

Slide 3 Innovation Network, Inc. Overview/Agenda Discuss challenges inherent to advocacy evaluation Review traditional evaluation models and their fit for evaluating advocacy Share lessons learned from our work Evaluation planning Data collection strategies Feedback loops and reflection Shift in emphasis

Slide 4 Innovation Network, Inc. A Case Study Coalition for Comprehensive Immigration Reform (CCIR) Evaluation funded by The Atlantic Philanthropies Began the evaluation in Fall 2005

Slide 5 Innovation Network, Inc. THE CHALLENGES

Slide 6 Innovation Network, Inc. The Challenges I continue to find my greatest pleasure, and so my reward, in the work that precedes what the world calls success. -- Thomas Edison

Slide 7 Innovation Network, Inc. The Challenges Success is to be measured not so much by the position that one has reached in life as by the obstacles which he has overcome while trying to succeed. -- Booker T. Washington

Slide 8 Innovation Network, Inc. The Challenges Ability is nothing without opportunity. -- Napoleon Bonaparte

Slide 9 Innovation Network, Inc. The Challenges If you don't risk anything, you risk even more. -- Erica Jong

Slide 10 Innovation Network, Inc. TRADITIONAL MODELS

Slide 11 Innovation Network, Inc. Continuous Learning Cycle Logic Model Analysis, Reflection & Improvement Evaluation Planning Data Collection

Slide 12 Innovation Network, Inc. The Logic Model

Slide 13 Innovation Network, Inc. Chain of Outcomes Residents improve basic job skills and identify jobs & career paths of interest KNOWLEDGE/ ATTITUDE Participants use post- training services, e.g. job placement, child care, transportation BEHAVIOR Closer in Time Easier to Measure More Attributable to Project Increased number of community residents obtain and retain jobs that support families CONDITION More Distant in Time Harder to Measure Less Attributable to Project

Slide 14 Innovation Network, Inc. LESSONS LEARNED

Slide 15 Innovation Network, Inc. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

Slide 16 Innovation Network, Inc. 8-Step Advocacy Evaluation Plan 1. Locate issue on the policymaking continuum 2. Identify type of policy change 3. Understand role of the group/network in policymaking 4. Clarify dimensions of success relevant to stakeholders 5. Develop chain of outcomes, leading to the intended change 6. Identify indicators & measurement systems 7. Identify most effective strategies 8. Prioritize

Slide 17 Innovation Network, Inc. Spheres of Influence

Slide 18 Innovation Network, Inc. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

Slide 19 Innovation Network, Inc. Data Collection Strategies Mixed-method approach Deeper focus on “real-time” data Observations/monitoring Frequent opportunities for data collection De-brief protocol Opportunity windows/intense periods Capturing multiple perspectives

Slide 20 Innovation Network, Inc. Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

Slide 21 Innovation Network, Inc. Reporting and Feedback Loops Frequent, informal reporting Learning briefs; research Approximate “real-time” Transparency/inclusion Face-to-face

Slide 22 Innovation Network, Inc. SHIFT IN EMPHASIS

Slide 23 Innovation Network, Inc. Before… Outputs Outcomes

Slide 24 Innovation Network, Inc. Now… Outputs Outcomes

Slide 25 Innovation Network, Inc. Finding Balance… OutputsOutcomes

Slide 26 Innovation Network, Inc. Advocacy Evaluation Project Online resources Indicators for advocacy Piloting tools Advisory committee

Slide 27 Innovation Network, Inc. Any Questions? Jennifer Bagnell Stuart Ehren D. Reed

Slide 28 Innovation Network, Inc. Thanks for Your Participation! Measure results. Make informed decisions. Create lasting change. Innovation Network, Inc K St. NW, 11 th Floor Washington, DC (202)