1. Complexity Matters: Aligning the Evaluation of Social and Behavior Change with the Realities of Implementation
International Social and Behavior Change Communication Summit 2016: Elevating the Science and Art of SBCC
Addis Ababa, Ethiopia I February 10, 2016

2. How did we get here?
Complexity, context and a growing focus on systems thinking and implementation.
The “Behavior Change in the Age of Complexity” workshop at the CORE Group meeting in Washington DC, October, 2015

3. Foundational truths of SBC
The change process is sensitive to context, can be highly variable, and difficult (if not impossible) to predict.
Many factors (e.g., motivation, knowledge, mood, structural realities, social norms, etc.) influence both social and individual behavior change.
Change is emergent—it unfolds as part of a process. Over-reliance on quantitative data not only fails to capture the complexity this process, but actually distorts it.
We know that monitoring and evaluation of SBC is necessary to both improve on the intervention at hand as well as inform future interventions.
Context matters – and influences the process of social and behavior change
There is a reason we know think about change in a social and behavioral way – it’s all parts combined – and how they intertwine
We have some good indicators to measure the “larger picture”, but are we measuring in a way that tells the “whole story”?
We cannot ignore, but must embrace cultural, socio-economic, and other contextual issues as part of the overall influencing process
Participatory approaches matter; the audience(s) have insight we gain from, it’s not simply increasing the knowledge base from a research perspective
We cannot ignore sustainability – time and time again, campaigns show results, but then we ask the question – what about a year later?

4. Basic challenges in M&E of SBC
Behavior is not static, it’s a constant moving target
What was our impact? How did we measure outcomes?
Trials of miles and miles of trials…
Project “life cycle” vs. “life”
Contextual complexity – determinants beyond the individual – much like we view in the social ecological model
Temporal complexity – even within a given program, variables change and can affect outcomes
Constructivist complexity – who’s eyes are we really looking through? Is it fair? Unbiased?
Attribution vs. Contribution – So many factors to consider that may be beyond one’s control or ability to influence. Being able to understand our contribution is helping in future improvements among our approaches
RCTs, yes seen as the gold standard, when looking at standardized indicators. There is nothing really “standardized” about SBC, it is completely reliant on norms, context, political will, but getting closer to the community to gain real-time perspectives on change can again help us improve our processes
Projects have a finite cycle. Life and those we work with affect change through SBC projects way outlive these cycles. Short term results give us only so much, there is untapped added value in longer term studies, including longitudinal studies and post-project sustainability studies

5. Pushing the M&E Envelope
Less donor-driven
Less proving and more improving
Learning from other disciplines
Embracing the messiness
Participation and engagement

6. Measuring the Hard to Measure
Individual and collective agency, efficacy & empowerment
Collective action
Increases in community voice and social advocacy
Social and human capital enhancement
Transformative or lasting change
Cross generational outcomes
Social norm shifting

7. Evolving Methods and Tools
Transformometer
Social network mapping
Narrative
Photovoice
Outcome Harvesting
Lives Changed Indices
Post-project sustainability
studies

8. Why a “Call to Action”? Don’t we already know that complexity matters?
What generates the gap between what we know and what we do?
Structural
Social
Personal

9. Recommendations for advancing the appreciation of context in programming
Implementing organizations can:
Facilitate participation of communities and implementers in the evaluation process 
Create rapid assessment tools that gauge the context of implementation as it evolves 
Be nimble 
Ensure that tracking the project implementation process—not just outcome indicators—is an activity included in proposals and work plans
Be humble about the transferability of your findings and expect the humility of others 
Donors can:
Recognize that many projects develop emergent goals over time
Value alternative and mixed methods for measuring SBC
Dedicate resources to strengthen evaluation capacity internally and externally
Stipulate that implementing partners should track the implementation process as well as project outputs and outcomes
Prioritize the acuity and nimbleness of implementer response  
Encourage the humility of implementing partners