Complexity Matters: Aligning the Evaluation of Social and Behavior Change with the Realities of Implementation International Social and Behavior Change Communication Summit 2016: Elevating the Science and Art of SBCC Addis Ababa, Ethiopia I February 10, 2016

How did we get here? Complexity, context and a growing focus on systems thinking and implementation. The “Behavior Change in the Age of Complexity” workshop at the CORE Group meeting in Washington DC, October, 2015

Foundational truths of SBC The change process is sensitive to context, can be highly variable, and difficult (if not impossible) to predict. Many factors (e.g., motivation, knowledge, mood, structural realities, social norms, etc.) influence both social and individual behavior change. Change is emergent—it unfolds as part of a process. Over-reliance on quantitative data not only fails to capture the complexity this process, but actually distorts it. We know that monitoring and evaluation of SBC is necessary to both improve on the intervention at hand as well as inform future interventions. Context matters – and influences the process of social and behavior change There is a reason we know think about change in a social and behavioral way – it’s all parts combined – and how they intertwine We have some good indicators to measure the “larger picture”, but are we measuring in a way that tells the “whole story”? We cannot ignore, but must embrace cultural, socio-economic, and other contextual issues as part of the overall influencing process Participatory approaches matter; the audience(s) have insight we gain from, it’s not simply increasing the knowledge base from a research perspective We cannot ignore sustainability – time and time again, campaigns show results, but then we ask the question – what about a year later?

Basic challenges in M&E of SBC Behavior is not static, it’s a constant moving target What was our impact? How did we measure outcomes? Trials of miles and miles of trials… Project “life cycle” vs. “life” Contextual complexity – determinants beyond the individual – much like we view in the social ecological model Temporal complexity – even within a given program, variables change and can affect outcomes Constructivist complexity – who’s eyes are we really looking through? Is it fair? Unbiased? Attribution vs. Contribution – So many factors to consider that may be beyond one’s control or ability to influence. Being able to understand our contribution is helping in future improvements among our approaches RCTs, yes seen as the gold standard, when looking at standardized indicators. There is nothing really “standardized” about SBC, it is completely reliant on norms, context, political will, but getting closer to the community to gain real-time perspectives on change can again help us improve our processes Projects have a finite cycle. Life and those we work with affect change through SBC projects way outlive these cycles. Short term results give us only so much, there is untapped added value in longer term studies, including longitudinal studies and post-project sustainability studies

Pushing the M&E Envelope Less donor-driven Less proving and more improving Learning from other disciplines Embracing the messiness Participation and engagement

Measuring the Hard to Measure Individual and collective agency, efficacy & empowerment Collective action Increases in community voice and social advocacy Social and human capital enhancement Transformative or lasting change Cross generational outcomes Social norm shifting

Evolving Methods and Tools Transformometer Social network mapping Narrative Photovoice Outcome Harvesting Lives Changed Indices Post-project sustainability studies

Why a “Call to Action”? Don’t we already know that complexity matters? What generates the gap between what we know and what we do? Structural Social Personal

Recommendations for advancing the appreciation of context in programming Implementing organizations can:   Facilitate participation of communities and implementers in the evaluation process  Create rapid assessment tools that gauge the context of implementation as it evolves  Be nimble  Ensure that tracking the project implementation process—not just outcome indicators—is an activity included in proposals and work plans Be humble about the transferability of your findings and expect the humility of others  Donors can:   Recognize that many projects develop emergent goals over time Value alternative and mixed methods for measuring SBC Dedicate resources to strengthen evaluation capacity internally and externally Stipulate that implementing partners should track the implementation process as well as project outputs and outcomes Prioritize the acuity and nimbleness of implementer response   Encourage the humility of implementing partners