1. 1 Unlocking Impact: The Five Key Questions Malen Davies RNIB Research Officer (Impact) 9 September 2014

2. Part One: Defining Success 2 What you do and what happens when you do it

3. Part One: Defining Success 3 What you do and what happens when you do it Lunch club model

4. Part One: Defining Success Five key questions 1. Why measure outcomes and impact? 2.What does success look like…exactly? 3. How are you going to make change happen? 4. Who needs to know you are being successful? 5. How are you going to know you're being successful? 4

5. Part One: Defining Success First Key Question: Why measure outcomes and impacts? 5

6. 6 Part One: Defining success Why measure outcomes and impact? Finding out progress against mission Learning how to improve Secure funding Inspiring staff and improving their work Raising profile Contributing to knowledge of ‘what works’

7. 7 Part One: Defining Success Second Key Question: What does success look like?

8. Out Part One: Defining Success come types Knowledge & Skill - "I know more" Emotional - "I feel different" Behavioural - "I act different" Opportunity - "I have a chance I didn't before" Economic - "I am financially better off" Health - "My health has improved" Outcome type

9. 9 Part One: Defining Success

10. 10 Part One: Defining Success Second Key Question: What does success look like…exactly? (What are your intended outcomes?)

11. 11 Part Two: Creating Change Third Key Question: How are you going to make change happen?

12. 12 Part Two: Creating Change Theory of change Inputs ActivitiesOutputs Outcomes Impacts

13. Impact Map Inputs Activities Outputs Outcomes Impacts What are the resources used to run activities and fulfil your objectives? What are the projects or processes that allow you to fulfil your objectives? What is produced as a direct result of these actions? Usually depicting completion of activity What benefit or change is accomplished as a direct result of the output? 13

14. 14 Part Two: Creating Change Theory of change: Counselling service Inputs ActivitiesOutputs Outcomes Impacts Counselling up to 10 clients each month 360 people a year provided with counselling Empower more people with sight loss 3 part time counsellors

15. 15 Part Two: Creating Change Theory of change: Counselling service Inputs ActivitiesOutputs Outcomes Impacts Counselling up to 10 clients each month 360 people a year provided with counselling Empower more people with sight loss 3 part time counsellors Challenging assumptions Are they qualified? Who is financing project? Is there need for this much support? External factors Is there other support delivered outside of counselling?

16. 16 Part Two: Creating Change

17. 17 Part Two: Creating Change Fourth Key Question: How are you going to make success happen? Building a theory of change group exercise

18. Part two: Creating Change Exercise: building a theory of change Plan a theory of change for a new telebefriending service for people with sight loss aged over 65 years old. 18

19. InputsActivities -Staff time. -Volunteer time - Grant funding -Telebefriending groups set up -Volunteers trained to become telebefriending facilitators - Develop training guides and resources for groups OutputsOutcomes / Impact -50 telebefriending groups set up - 20 volunteers trained - Information and resources provided to telebefriending groups -Improve sense of well-being for customers -Increased quality of life for customers - Volunteers improve knowledge and skills Theory of change Part two: Creating Change

20. Part Two: Creating Change Theory of Change: Help teams work together to achieve a shared understanding of a project and its aims. Bring the process of change to the forefront. Make projects more effective Quickly communicate a project’s aims Encourage teams to engage with the existing evidence base 20

21. 21 Part Three: Planning Measurement Fourth Key Question: Who needs to know you are being successful?

22. 22 Part Three: Planning Measurement Who wants to know and what do they want? Who needs to know what do they need?

23. 23 Part Three: Planning Measurement Fourth Key Question: Who needs to know you are being successful? (Identify your audience)

24. 24 Part Four: Measuring Success Fifth Key Question: How are you going to know you're being successful?

25. 25 Part Four: Measuring Success Impact Measurement Tools and Tips

26. 26 Part Four: Measuring Success Principles of Impact Measurement 1.Clarity - The reader can quickly and easily understand the intervention through a coherent narrative that connects charitable aims, plans, activities and results 2.Accessibility - Relevant information can be found by anyone who looks for it, in a range of formats suitable for different stakeholders 3.Transparency - Reporting is full, open and honest 4.Accountability - Reporting connects with stakeholders, partners and beneficiaries to tell them what they need to know; 5.Verifiability - Claims about impact are backed up appropriately, allowing others to review 6.Proportionality - The level and detail of reporting reflects the size of the intervention Charity Finance Group, ACEVO & NPC (2012)

27. Part Four: Measuring Success Technical and Established Methodologies Example: Social Return on Investment (free): http://www.thesroinetwork.org/ http://www.thesroinetwork.org/ 27

28. Part Four: Measuring Success Casework Measures Examples: Outcomes Star (cost): http://www.outcomesstar.org.uk/ http://www.outcomesstar.org.uk/ Rickter Scale (cost): http://www.rickterscale.com/http://www.rickterscale.com/ 28

29. Part Four: Measuring Success Validated Questionnaires Example: Warwick-Edinburgh Mental Well-Being Scale (free): http://www.healthscotland.com/scotlands- health/population/Measuring-positive-mental- health.aspx http://www.healthscotland.com/scotlands- health/population/Measuring-positive-mental- health.aspx 29

30. Part Four: Measuring Success Bespoke Tools Examples: Semi-Structured Outcome Questionnaires, Focus Groups, Diary Studies, Social Media. 30

31. 31 Part Four: Measuring Success Deciding Which Measure to Use 1. Audience 2. Resource 3. Access

32. 32 Part Four: Measuring Success Tips for success Be as clear as possible about the outcomes you're trying to measure - without this you will struggle measure them! Be flexible with you measurement methods - good evidence needs good research principles, but it can be done in flexible and 'natural' ways, through conversation and existing opportunities to engage with customers

33. 33 Part Four: Measuring Success Tips for success Explain to customers your rationale - evaluation need not be cold or abstract. Explain to your customers why you need to collect impact information, and why it's important for you, them and other customers

34. 34 Part Five: Communicate Success Communicate Your Evidence! Example: Annual Report Format Why we're doing this (Need) What we did (Activity) What difference it makes (Impact)

35. Unlocking Impact: The Five Key Questions 1. Why measure outcomes and impact? 2.What does success look like…exactly? 3. How are you going to make change happen? 4. Who needs to know you are being successful? 5. How are you going to know you're being successful? 35

36. Useful links New Philanthropy Capital: https://www.thinknpc.org/publications/the-principles-of- good-impact-reporting-2/ https://www.thinknpc.org/publications/the-principles-of- good-impact-reporting-2/ CENI: http://www.ceni.org/evaluation-impact- measurementhttp://www.ceni.org/evaluation-impact- measurement Impact HUB: http://inspiringimpact.org/listings/http://inspiringimpact.org/listings/ 36

37. Questions Malen Davies Research Officer (Impact) Malen.davies@rnib.org.uk 37