1. Quality Seems to be the Hardest Word: How one UK Funder Uses Evaluation to Achieve Policy Change Dr Andrew Cooper
Opener:
The idea was to define the scope of the group and map out the big questions that members wanted to discuss
Anecdote:

3. What I am going to talk about
Brief overview of The Diana, Princess of Wales Memorial Fund (the Fund)
Our evaluation framework
Example 1: Cluster Munitions
Fund-wide evaluation themes
Example 2: Penal Reform
Challenges of evaluating policy change work
Example 3: A Funder Coalition for Children
Summary – achieving a quality evaluation
Questions for the Fund and others
Test notes 1

4. Brief overview of the Fund
Set up 1997 after the death of the Princess
Initial grant rounds and themes for ‘vulnerable groups’
From 2007 a new Strategic Plan with specific policy change objectives and a spend out timetable
Four Initiatives: Penal Reform, Cluster Munitions, Palliative Care, Refugees and Asylum Seekers
Around £100 million given away (around USD 160m)
‘Funder plus’ and collaboration focus
18 staff – with specific policy knowledge
Committed to research and evaluation
to share learning externally

5. Why do a ‘quality’ evaluation and what is it for us?
Learning
Accountability
Contribution
Attribution?
Impact
Terminology
Flexibility
Proportionate
Policy change

6. The Fund’s evaluation framework
3 levels: project, initiative, Fund-wide
External focus for learning
Theory of change
Formative evaluation
Flexible – policy change and advocacy context (e.g. elections)
Key question for us –
why evaluate?

8. Fund-wide evaluation themes
Theme 1: How can funders create changes in policy and practice?
Theme 2: Does ‘funder plus’ add value, or is it more trouble than it’s worth?
Theme 3: Does working in collaboration (with each other) make funders more effective?
Major evaluation to share learning and contribute to our legacy
Aimed at influencing funders and philanthropists
What has worked and what has failed in our approach
How can other funders and philanthropists achieve policy change

10. Challenges of evaluating policy change work
What is the difference between planning and evaluating?
What is the role of the funder in policy change and advocacy?
How can we respond to changes in the external context?
What is an ‘adequate’
theory of change?

12. Possible factors in achieving a quality evaluation
Why evaluate and how will the learning be used.
Policy change work is messy and tactics may change over time.
Quality does not always mean academic, ‘scientific’, or complex!

13. To summarise: what is a ‘gold standard’ high quality evaluation?
Clear purpose and objective(s), personalities, audience(s), proportionate.
‘Contribution’ to policy change, distinction between ‘process’ and ‘impact’, rigorous.
Flexible, useful, well written and presented, challenging, accessible!

14. Questions
How to use simple but effective models to evaluate policy change and advocacy work?
What are the differences between quality advocacy evaluation in the UK and in the US?
How can funders contribute to better quality advocacy evaluation?
Thank you!