Thomas Schwandt University of Illinois, USA Evaluation, complexity, equality Presentation in the Technical Seminar: “Towards an Equity-focused and Gender -responsive framework to evaluate the SDGs” UNICEF March 16, 2016 Thomas Schwandt University of Illinois, USA

What is complexity?

Complexity as… Plexus means braided or entwined, from which is derived complexus meaning braided together; the English word “complex” is derived from the Latin

CAS Multiple, interconnected parts Dynamic, adaptive Exhibit diversity Display emergence E.g., ecologies, education and health systems, etc.

Another way of thinking about complexity A characteristic of a situation For example in the planning, implementing and evaluating of an intervention (policy, program, project)

A situation is characterized by… Multiple actors & relationships Multiple understandings and perspectives Conflict—differences in relations of power restrict or limit what is considered to be important regarding: boundaries of facts and norms (what’s in and what’s out) boundaries of perspectives (whose viewpoints are more privileged and whose viewpoints are more marginalised)

AND… Many, if not all, of the situations we are interested in examining in the 2030 Agenda are marked in some way by a context of structural inequality* (i.e., the absence of just and fair inclusion of all) * Unequal access and status with respect to roles, functions, decisions, rights, etc. based on race, gender, ethnicity, class, etc.

CHALLENGES

Challenges to Evaluators Be critically aware of different possible boundaries and their consequences for judgments of whether an intervention is effective; be capable of facilitating boundary critique

Challenges to Evaluators Avoid the fixation on a single vision, e.g., overarching Theory of Change linking intervention and outcome(s) in predictable, controllable, linear, causal pathways. Design evaluations that are flexible, adaptive, iterative

Challenges to Evaluators Develop an expanded universe of evidence —multiple methods working in concert: Most Significant Change Outcome Mapping Social Network Analysis Soft Systems Methodology Qualitative Comparative Analysis Performance Indicators Experimental Trials

Policy Challenges Avoid indicator blindness: Andrew Natsios (2010), ‘’… those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable.” Natsios, A. (2010) ‘The Clash of the Counter-bureaucracy and Development’, Washington DC, USA: Center for Global Development. Available at www.cgdev.org/files/1424271_file_Natsios_Counterbureaucracy.pdf

Policy Challenges Avoid excessive reliance on static, prescriptive, comprehensive planning—rather, treat plans as hypotheses; engage in dynamic, flexible, diversified planning

Policy Challenges Avoid exclusive focus on ex-ante analysis in favor or more attention to monitoring, learning, adaptation

Challenges to Policymakers & Evaluators Develop methodologies to synthesize findings from multiple interventions and across multiple goals (Implementation of the SDGs requires a more complex, integrated policy agenda requiring new ways of thinking of the science-policy nexus, Goal 17)

I look forward to our continuing discussion Thank you! I look forward to our continuing discussion