1. EVALUATING FOR GENERATIVE CHANGE: CITIZEN PARTICIPATION FOR ACCOUNTABILITY Australasian Evaluation Society (AES), International Conference, Friday 2 Sept 2011 Keren Winterford

2. The presentation Short presentation + more time for Q&A / discussion = Slide presentation informed by Pecha Kucha

3. Today’s story

4. KEY MESSAGE 1.We need to think about ‘change’ within development practice in different ways 2.Our standard M&E practices are not helpful in comprehending this type of ‘change’ 3.We need to prioritize and assess our contribution to Generative Change. That is change that propels itself for continued, amplified and unknown outcomes. 4.We need to employ different thinking and practice for evaluating generative change

5. Citizen Voice and Action

6. Shifting participation From Beneficiary Project Consultation Appraisal Micro Consumer / client To Citizen Policy Decision-making Implementation Macro (Gaventa and Valderrama 1999) Co-creator in policy and service delivery A different view of participation redefines ‘change’ and how ‘change’ happens within development practice

7. How ‘change’ happens Development workers enable / influence only Citizen led VOICE ACCOUNTABILITY Change occurs from citizen voice influencing state accountability

8. STATE (government officials)CITIZENS Citizen Led Building and rebuilding relationships Dialogue New Knowledge Citizenship built through iterative cycles of participation, action and change Citizen influence of state Culture of accountabilities Gov response Citizen Action Gov with citizens Motivating and encouraging What I’ve learnt from my research about how ‘change’ happens

9. What I’ve learnt from my research about what ‘change’ is valued Intangible changes - self perception (of citizenship), beliefs, attitudes and relationships Strengthened relationships Increased dialogue Achieving quick wins Multiple accountabilities These valued changes provide a platform for ‘Generative Change’

10. STATE (government officials)CITIZENS Citizen Led Building and rebuilding relationships Dialogue New Knowledge Citizenship built through iterative cycles of participation, action and change Citizen influence of state Culture of accountabilities Gov response Citizen Action Gov with citizens Motivating and encouraging Valuing Generative Change Generative Change

11. How we think about change informs our evaluation practice Informed by development management Predictable Controllable End point ‘System’ known Linear causal strand ‘Change’ is… Really?!... Dissonance between ‘what we think we do’ and ‘what we do’

12. “Those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable”. Former USAID Director, Andrew Natsios, 2010

13. Challenges in evaluating Citizen Voice and Action Evaluating ‘project’ within broader social change process – how to disentangle!? Short term project cycles – for influencing long term change – how to discern impact?! Role of INGO in enabling / facilitating – how to discern causal link for INGO? The value of ‘the intangible in creating change – how to measure?!

14. We can think about ‘change’ differently and our role in it Informed by complexity thinking Emergent Enabled only – citizen led Part of social change Incomplete knowledge Multiple causal strands Change is…transformational and generative Informs ‘different’ types of evaluations

15. Making choices for evaluation For Whom? Whose reality counts? Who knows what? What can we know...about ‘change’? What is the ‘change’? The ‘Gold Standard’ should be ‘methodological appropriateness’ (Patton 2010)

16. Humility see ‘change’ as ‘complex’ Reflexivity Fluency in politics Accountable for why acted Only partial knowledge possible blurring distinctions [M&E / formative- summative / interim with ‘end game’ / process and outcome ‘measures’ theory of change – multiple strand causality Citizen led – learning for social change For Generative Change Real time feedback Emergent planning Valuing building blocks for generative change To evaluate... Valuing quick wins

17. Valuing for Generative Change We need to and can think about how change happens differently We need to value different sets of change outcomes as well as process = which form the building blocks for generative change We need to be humble and honest Set our learning goals towards how best to support individuals themselves to create their own best future

18. Thanks & Questions

19. The value of seeing through a complexity lens Non linear Emergent Feedback processes shape (future) change Interacting and interdependent parts Sensitive to initial conditions Co evolve Adaptive Self organizing Order within chaos ‘Behavior of agents’ ‘Features of system’ ‘Dynamic of change’ Ramalingam et al 2008