Evaluating Advocacy: A Model for Public Policy Initiatives

Who We Are: Innovation Network National nonprofit organization Committed to evaluation as a tool for empowerment Work with nonprofits at the grassroots level Build evaluation capacity of nonprofits and funders so they can better serve their communities Practice a participatory approach Innovation Network (202) 728-0727 1625 K St. NW www.innonet.org Washington, DC 20006 info@innonet.org

Overview/Agenda Discuss challenges inherent to advocacy evaluation Review traditional evaluation models and their fit for evaluating advocacy Share lessons learned from our work Evaluation planning Data collection strategies Feedback loops and reflection Shift in emphasis

Coalition for Comprehensive Immigration Reform (CCIR) A Case Study Coalition for Comprehensive Immigration Reform (CCIR) Evaluation funded by The Atlantic Philanthropies Began the evaluation in Fall 2005

THE CHALLENGES

The Challenges I continue to find my greatest pleasure, and so my reward, in the work that precedes what the world calls success. -- Thomas Edison

The Challenges Success is to be measured not so much by the position that one has reached in life as by the obstacles which he has overcome while trying to succeed. -- Booker T. Washington

Ability is nothing without opportunity. The Challenges Ability is nothing without opportunity. -- Napoleon Bonaparte

If you don't risk anything, you risk even more. The Challenges If you don't risk anything, you risk even more. -- Erica Jong

TRADITIONAL MODELS

Continuous Learning Cycle Logic Model Analysis, Reflection & Improvement Evaluation Planning Data Collection

The Logic Model

Chain of Outcomes Residents improve basic job skills and identify jobs & career paths of interest KNOWLEDGE/ ATTITUDE Participants use post-training services, e.g. job placement, child care, transportation BEHAVIOR Increased number of community residents obtain and retain jobs that support families CONDITION Closer in Time Easier to Measure More Attributable to Project More Distant in Time Harder to Measure Less Attributable to Project

LESSONS LEARNED

Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

8-Step Advocacy Evaluation Plan Locate issue on the policymaking continuum Identify type of policy change Understand role of the group/network in policymaking Clarify dimensions of success relevant to stakeholders Develop chain of outcomes, leading to the intended change Identify indicators & measurement systems Identify most effective strategies Prioritize

Spheres of Influence

Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

Data Collection Strategies Mixed-method approach Deeper focus on “real-time” data Observations/monitoring Frequent opportunities for data collection De-brief protocol Opportunity windows/intense periods Capturing multiple perspectives

Continuous Learning Cycle Planning Real-Time Feedback and Reflection Data Collection

Reporting and Feedback Loops Frequent, informal reporting Learning briefs; research Approximate “real-time” Transparency/inclusion Face-to-face

SHIFT IN EMPHASIS

Before… Outcomes Outputs

Now… Outputs The fields of evaluation and philanthropy have seen a shift of focus from measuring outputs to outcomes in the last 10 to 15 years. See Kellogg materials. Outcomes

Finding Balance… Outputs Outcomes

Advocacy Evaluation Project Online resources Indicators for advocacy Piloting tools Advisory committee www.innonet.org/advocacy

Jennifer Bagnell Stuart Any Questions? www.innonet.org Jennifer Bagnell Stuart jabstuart@innonet.org Ehren D. Reed ereed@innonet.org

Thanks for Your Participation! Measure results. Make informed decisions. Create lasting change. Innovation Network, Inc. 1625 K St. NW, 11th Floor Washington, DC 20006 (202) 728-0727 www.innonet.org