1. Is the Accountability, Impact and VFM Debate too Donor Focused? Cathy Shutt November 23 rd INTRAC Workshop

2. Competing M&E and Impact Assessment Agendas? DFID’s PPA quantitative indicator log frame requirements Testing and learning from theories of change using pluralist and participatory M&E methods Organisational demands to choose a set of global or meta-level indicators to assess impact of new programmatic approach, over 10-15 years, according to fixed results based chain –Impact = sustainable changes in people’s lives

4. Key Points from Literature Are causal relationships underpinning CSO theories of change valid: –between transparency and accountability? –between voice and accountability? –between de jure policy decisions and de facto development outcomes? Arguments from Kauffman at the Bank: –Measures of de jure quantifiable indicators - policy change - meaningless without qualitative accompaniment –Assessing governance changes is about perceptions: whose and how? –Links between policy and development outcomes too complex for attribution: need to prove contribution Citizens’ empowerment is more than a means to an end (Gaventa and Barrett) Need for nuanced, contextualised qualitative approaches to governance M&E

5. INGO M&E Practice - trends NOs/HQ desire to improve M&E stemming from multiple drivers Pursuing/piloting pluralist approaches to M&E & impact assessment M&E seen as means to test & learn from theories of change Debates about conceptualisations of impact: can empowerment be more than a means to an MDG end? Focus on changes in power relations: integrating power analysis Interest in finding meaningful ways to assess contribution Organisational pressure to: – Use global or meta-level indicators? – Aggregate? – Produce some numbers – default ‘numbers reached’ Worries about capacity M&E debates producing opportunities for learning

6. Value for Money Conundrum INGOs Want to influence conceptualisation -where to fight battles? Concerns about conceptualisation in complex multi -country ‘governance’ projects part of broader theories of change Numbers reached / £ = VFM? Competition and race to the bottom? Is VFM having procedures that seek to spend less, well and wisely or number crunching – cost/benefit per beneficiary? Many outcomes under a RBA e.g. stopping human torture cannot be reduced to numbers and costs Aid is about accountability to poor people not donors Collective action by the sector needed: –define and articulate the sector's 'unique selling point' –Public education: less simplistic marketing, more nuanced communication about RBA being about changing power relations

7. Questions Are there ways to ensure that personal freedoms and empowerment do not get relegated to instrumental outputs - means to MDG ends? How do we measure them? What do we count? How much When? Is it possible to attach monetary value? What exactly are INGOs trying to ‘quantify’ or ‘aggregate’ beyond donor requirements and why? What are the implications e.g.? –What will the impact be on local partners managing relationships with several INGOs each using different organisational metrics, measures and approaches? –Given methodological challenges and weak M&E capacities is there a danger that ‘numbers reached’ will form the basis for VFM calculations & lead to a ‘race to the bottom’?