1. Evaluation in European Foundations: Trends and Perspectives Andrew Barnett UK Branch Director Calouste Gulbenkian Foundation

2. Outline 1.Two key trends 2.Our recent experience of designing an evaluation system 3.What is evaluation for?

3. Lifecycle of a programme 1 Scoping: research and consultation to identify most effective intervention in response to an issue/problem 2 Objectives and outcomes: developing a plan of activity to maximise beneficial impact including determining what success might look like. 3 Implementation: might include funding pilot projects. 4 Evaluation: assessing impact and discerning learning. 5 Dissemination: targeted communication of the learning to those who can make a difference, can change systems, scale or replicate successful initiatives. 6 Exit: concluding the programme

4. Trend 1: Focus on ‘effectiveness’ Traditional giving helps one person or organisation at a time by providing support for immediate needs. Strategic philanthropy focuses on systemic change and builds for the future. Centre for Effective Philanthropy

5. Foundation sector in Europe is growing dynamically 110,000 ‘public-benefit foundations’ in the EU 43% set up as recently as the early 1990s (many of these small and associated with ‘new wealth’) Foundations in Europe spend between €83 billion and €150 billion annually, over twice as much as the US foundation sector Direct full-time employment: between 750,000 to 1 million people in the EU

6. Some important players – but not really a movement Bertelsmann Foundation (Germany) Compagnia di San Paolo (Italy) King Baudouin Foundation (Belgium) Bernard van Leer Foundation (Netherlands) International network of strategy philanthropy (2001-2005) Publication: Rethinking Philanthropic Effectiveness (2005) Foundations are magpies. They rarely stick with one way of approaching evaluation. Gerry Salole, EFC

7. Trend 2: the quest to measure value Helping all organisations identify, measure and evaluate their organisational outcomes would be hugely valuable as a whole... Funders and commissioners have a vital role to play in incentivising good outcomes measurement – funders need to incorporate evaluation data into subsequent rounds of grant giving in order for organisations to see a return for their efforts, and commissioners need to put money aside in contracts specifically for the evaluation of projects. Measuring Social Value Demos 2010

8. Examples UK Players: New Economics Foundation New Philanthropy Capital NCVO Charities Evaluation Services Cabinet Office

9. Principles of SROI Involve stakeholders Understand what changes Value the things that matter. Only include what is material. Do not over claim. Be transparent. Verify the result

10. Our experience: drivers and lessons Need to maximise impact Need to tell a compelling story to partners and collaborators and to the wider sectors Need to extract learning from individual activities Desire to set an example and to lead: to be at the forefront of thinking and practice

11. Key stages in the journey Develop a strategy and operationalise the strategy Go live as soon as possible – ‘retrofit’ existing projects and apply system to new or early stage projects Use an external consultant to draw strands together and co-devise a system with the team – codify the process, make it explicit. Review and iterate. And keep on doing it.

12. Our strategy One overriding purpose which links to 3 main strategic aims which link to 3 objectives under each aim which reflect Time-limited programmes and activities 1 cross-cutting aim concerned with capacity

13. A vital planning and management tool: the evaluation cycle

14. From implicit to explicit

15. Main lessons learned Evaluation is the means not the end. It is part of your planning tool box. At the most back level, an evaluation system consists of making the implicit explicit. Everyone needs to own any evaluation system. A good evaluation system should not constrain but should provide greater freedom. Be realistic about measuring long term top-level impact The system should focus on the programme level and above

16. Live design by users

17. Our emerging ‘theory of change’ Scoping Coalition building Persuading Demonstrating Learning and improvement

18. Concluding questions What is evaluation for? What do you believe to be the main drivers? What’s your journey like? Questions for me.

19. For more information andrew.barnett@gulbenkian.org.uk www.gulbenkian.org.uk