1. Measuring Effectiveness, Melbourne, Sept 2007 1 Measuring effectiveness a network perspective…

2. Measuring Effectiveness, Melbourne, Sept 20072 The questions asked… ► How we assess relationships ► How we need to change as individuals and organisations to be open to feedback ► How we empower communities to hold us and others to account

3. Measuring Effectiveness, Melbourne, Sept 20073 Why bother?  People, and their relationships with each other, are the means and end of development  “…if donors are to achieve their aims of contributing to the reduction of global poverty they need to invest as much or more time in managing relationships as they currently spend in managing their money”[1] [1] [1] “Relationships Matter For Supporting Change In Favour Of Poor People” R. Eyben 2004. IDS [1]

4. Measuring Effectiveness, Melbourne, Sept 20074 Three approaches ►Analysis of individual relationships ►Most common ► Analysis of portfolios of relationships ► Not so common ► Analysis of networks of relationships ► Less common

5. Measuring Effectiveness, Melbourne, Sept 20075 1. Analysis of individual relationships ► “Autonomy or Dependence: Case Studies of North-South NGO Partnerships” Brehm et al 2004 ► “This book explores the concept and practice of ‘partnership’ between non-government organisations (NGOs) in the North and South. Based on a rigorous four-year study, the book draws together the perspectives of a group of European NGOs and compares these with the experiences of a selection of their partners in Brazil, Cambodia and Tanzania” ► Oxfam GB in Partnership: A Global Strategic Evaluation, 2007

6. Measuring Effectiveness, Melbourne, Sept 20076 2. Analysis of portfolios of relationships ► Example from Bangladesh 1994 ► Result of a card sorting exercise ►CAA Country Programme Director ► Method at http://www.mande.co.uk/docs/hierarch.htm http://www.mande.co.uk/docs/hierarch.htm ►Identified a nested set of “Most Significant Differences” between 12 partners ►Strategy can then expressed as choices between types of partners ► later performance can be compared to strategy

7. Measuring Effectiveness, Melbourne, Sept 20077 A nested classification Other rankings

8. Measuring Effectiveness, Melbourne, Sept 20078 3. Analysing networks of relationships ► As well as looking at the portfolio of actors we are working with, we also need to consider the relationships between these actors, ► and their relationships with others ► As we increase the number of actors, the number of possible relationships between them increases exponentially i.e. A LOT ► In order to describe, or plan, what is happening with these relationships, we need to move from a hierarchical view to a network view

9. Measuring Effectiveness, Melbourne, Sept 20079 One view of 16 NGOs in Ghana In social network analysis, this is a “one-mode” network, showing … Actors x Actors Cell values = frequency of references to each other, in their Progress Reports between 2005-6 Cells can represent many other aspects of their relationships Matrices can hold data on thousands of relationships Up to 120 here But it is hard to read.

10. Measuring Effectiveness, Melbourne, Sept 200710 The same data, presented in network diagram form.. makes many features of the relationships more visible Weak links Strong links Reciprocated links Well connected actors Unconnected and marginalised actors Groups of actors

11. Measuring Effectiveness, Melbourne, Sept 200711 Of course the picture can get much more complex. Here we have all the other working relationships they reported for the 2005/6 period NGOs Shared links Unique links 270 actors in all

12. Measuring Effectiveness, Melbourne, Sept 200712 What should we be assessing? ►Actors have their own strategies. When implemented via relationships with others, these strategies connect up, to form a larger structure, which no single actor designed or controls a larger structurea larger structure ►Emergent structure = collective strategy ►But the big question is… ►How functional is that emergent structure & strategy? ►And how do we ask about this? ► Are there standard questions and different “right answers” ►According to specific contexts

13. Measuring Effectiveness, Melbourne, Sept 200713 Emergence? One view…  Complex structures  that arise from the interactions of many actors,  who are following local rules  Their properties not predictable from knowledge of the parts  Weak form: If you are one of the actors  Strong form: Even if you know what all the actors are doing (because they react to each others reactions)

14. Measuring Effectiveness, Melbourne, Sept 200714 Some network perspectives and questions  Matrix view  One mode: G-rap NGOs, IMHI indicators  Two mode: Molisa, Vietnam; Oxfams, Laos  Diagram view  One mode: Ghana workshop participants  Two mode: M4P Workshop, OI internal groups

15. Measuring Effectiveness, Melbourne, Sept 200715 Matrix view: Ministry of Labour…, Vietnam 1. List existing programs 2. List future goals 3. Rank expected impact of a program on each objective 4. View summary rows 5. Discuss acceptability of overall result 6. Then what to change (a two mode network)

16. Measuring Effectiveness, Melbourne, Sept 200716 2 nd matrix view: Oxfam Int. in Laos Question : What will change after review and re-iteration? The objectives, the programs and their contribution, or the desirability of the collective outcome

17. Measuring Effectiveness, Melbourne, Sept 200717 1 st diagram view: Oxfam in Laos Questions to ask 1.Shared links: actual communication, specialisation, most valued 2.Unique links Most valued, balance of shared vs unique 3.Missing links between these actors

18. Measuring Effectiveness, Melbourne, Sept 200718 Oxfam in Laos: A different view 1.Shared links. Shared preferences = communication about same objectives? And specialisation? 2.Unique links. Which are most valued? 3.Missing links. Is isolation of some objectives a problem, when there should be a causal connection? Strategic Plan objectives Affiliates Affiliates involvement with SP objective

19. Measuring Effectiveness, Melbourne, Sept 200719 Internal diagram view: M4P, Hanoi M4P project, Hanoi Nodes = workshops Links = movement of participants from one workshop to another Thicker link = more participants moving Node colour = type of workshop (by topic ) Unique links Shared links Missing links People’s participation in multiple workshops

20. Measuring Effectiveness, Melbourne, Sept 200720 Roles  Mapping Gather data Gather data Opportunistic sourcesOpportunistic sources Planned sourcesPlanned sources Collate, feedback, raise questions Collate, feedback, raise questions  Navigating Actors choose how they want to respond Actors choose how they want to respond The mapped territory then changes The mapped territory then changes

21. Measuring Effectiveness, Melbourne, Sept 200721 What's new and not new?  Not new:  Emergent strategy: that which arises from the collision of intentions, and circumstances  Structure as strategy: structure as means to an end  New  The capacity to represent and analyse the collective structures that are emerging, via new methods  Social Network Analysis, and software  Needed  Finding useful generic questions to ask network participants about the emergent structure, based on network analysis findings so far  More participatory tools, to engage the actors within the network