1. PADev Participatory Assessment of Development as a method for assessing agencies Presented at the 10 th Biannual Conference of the European Evaluation Society, Helsinki. October 3, 2012 www.padev.nl Wouter Rijneveld and Fred Zaal

2. Contents 1.Brief introduction of PADev 2.Agency Assessment 3.Outcomes in Langbinsi, N-Ghana 4.Uses for this approach 2 “Adding constituency voice to organizational assessment”

3. What is PADevWhat is PADev Developed 2008-2012 universities, research institutes + NGOs from NL, Burkina Faso, Ghana 4 rounds of fieldwork in 11 locations Participatory workshops of 3 days 3

4. backdonor INGO NGO Project a in community x Usual perspective in evaluationUsual perspective in evaluation 4 Activities Results Outputs Outcomes Impact PM&E

5. Community x Very poor – poor – average – rich – very rich Local influences National influences Global influences Natural Physical Economic Human Social Political Cultural Project c Project b backdonor INGO NGO 1 Project a Community x Project c Project b NGO 2 Project a Project c Project b NGO 3 Project a Private initiatives Government Local State National Companies (e.g. telecom) Other INGO’s Other backdonors

6. History Community x Very poor – poor – average – rich – very rich ProjectsActors Changes in context Methodology PADevMethodology PADev 6

7. 3 day workshops3 day workshops 60 people from area of 20,000 Subgroups: men, women, old, young (officials and project staff separate group) Individual life history questionnaire data about participants, parents, siblings, children (total 600 persons per workshop) 4 rounds of 3 workshops each 7

8. Exercises in PADevExercises in PADev Time line events Changes in 25 years Ethnography of wealth classes Inventory of ‘projects’ Best 5 / worst 5 Impacts on wealth classes Perceptions over time Relation between changes and projects Agency assessments 8

9. 9

10. 10

11. 11

12. Agency assessmentAgency assessment workshop Jan 2012 Langbinsi, North Ghana Per subgroup major agencies identified (5-10) Negotiated consensus per subgroup on six statements Rating scale from ‘always’ to ‘usually not’ Debriefing with the agencies

13. Statements (derived from previous workshops)

14. Overview Most critical comments: –Participation (influence on type and manner of projects) –Honesty Most positive: –Relevance –Long term commitment

15. Per agencyPer agency Organisation (code) Long term commitment Realistic Expectations HonestyRelevanceParticipation# groupsOverall a4,0 3,53,83,563,8 e3,74,03,74,03,333,7 f3,33,7 4,03,333,6 d3,54,03,04,03,043,5 b3,42,8 3,2 53,1 h3,02,0 4,03,022,8 i2,0 1,54,02,522,4 g3,02,51,52,0 22,2 c2,82,02,31,81,542,1 k4,0 1 q 3,013,8 r3,04,03,04,0 13,6 s4,0 2,013,6 p3,02,04,0 13,4 j2,04,0 3,0 13,2 n2,03,0 4,0 13,2 m3,04,0 2,0 13,0 o4,0 2,04,01,013,0 l4,02,0 12,4

16. Per agencyPer agency Organisation (code) Long term commitment Realistic Expectations HonestyRelevanceParticipation# groupsOverall a4,0 3,53,83,563,8 e3,74,03,74,03,333,7 f3,33,7 4,03,333,6 d3,54,03,04,03,043,5 b3,42,8 3,2 53,1 h3,02,0 4,03,022,8 i2,0 1,54,02,522,4 g3,02,51,52,0 22,2 c2,82,02,31,81,542,1 k4,0 1 q 3,013,8 r3,04,03,04,0 13,6 s4,0 2,013,6 p3,02,04,0 13,4 j2,04,0 3,0 13,2 n2,03,0 4,0 13,2 m3,04,0 2,0 13,0 o4,0 2,04,01,013,0 l4,02,0 12,4 Blue = government White - NGO Mentioned by more groups Mentioned by 1 group Sorted by overall score

17. Some very popular NGOs Rather good (e.g. Health) and rather poor government agencies (the more political the worse) And some very small, very impopular NGOs Biggest difference Govt-NGOs: honesty and realistic expectations October 3, 201217

18. Differences Difference score officials - non-officials Long term commitment Realistic Expectations HonestyRelevanceParticipation Overall score Governmental0,50-0,550,281,05-0,60+0,13 Non-Governmental0,350,250,500,200,80+0,42 Overall0,34-0,440,000,63-0,23+0,09 Mostly NGO officials who rated –More positive than others about (their) NGOs, esp. about participation (  bias) –More convinced about relevance of Govt (  realistic view of what NGOs can and cannot do)

19. Could be used whenCould be used when Adds constituency voice to organizational assessment (OA) Cannot replace other forms of OA Provides positioning of organization relative to other agencies 2 hours exercise, 3 hours of analysis Careful of bias of just focusing on one organization October 3, 201219