1. nef (new economics foundation)
Workshop: Value for Money (VfM) in international development using Social Return on Investment (SROI)
Natalie Nicholles
nef (new economics foundation)
19th October 2010

2. Housekeeping
Ground rules
Fire exits
Toilets
Workshop materials

3. Agenda 09:30 Big picture introduction to Value for Money and SROI
10:00
Crash course on SROI: understanding change
10:45
Tea/coffee break
11:00
Crash course on SROI: measuring change
11:30
Crash course on SROI: valuing change
12:30
Lunch
13:15
Crash course on SROI: attribution & partnership working
13:45
Crash course on SROI: forecasting long term benefits
14:45
15:00
Worked-through example & organisational decision making
15:45
Answering key questions
17:00
Close

4. Aims for the day
To present an overview of what Value for Money (VfM) means
To introduce ways of capturing Value for Money in international development through the Social Return on Investment approach
To share nef’s work on this subject to date
To facilitate discussion on how to further develop demonstrating value and impact of international development work

5. new economics foundation (nef)
Founded in 1986
Economics Think Tank working to promote:
Innovative solutions that challenge mainstream thinking on economic, environmental and social issues
nef consulting is the consultancy arm. It exists to promote and disseminate nef solutions
History of working with public sector on implementing value for money solutions

6. Problems with measurement
Financial measurement:
limited measure of value
2. We allocate resources only
to the things we can measure
3: Stakeholders are left out of
decision making

7. The challenge Measurement across the ‘triple bottom line’ The economy
The environment
People

8. Value for Money (VfM)
VfM is about making sure that spending achieves as much as possible
A way of making decisions about how to use limited resources
Sometimes mistaken for lowest unit cost
Without a measure of quality or effectiveness, risk false economies
E.g. Children in care

9. Treasury VfM Definition
“VfM is defined as the optimum combination of whole-of-life costs and quality (or fitness for purpose) of the good or service to meet the user’s requirement. VfM is not the choice of goods and services based on the lowest cost bid.”
HM Treasury (2006) Value for money guidance

10. Treasury VfM Definition
“Wider social and environmental costs and benefits for which there is no market price also need to be brought into any assessment. They will often be more difficult to assess but are often important and should not be ignored simply because they cannot easily be costed.”
HM Treasury Green Book
Effectiveness, not lowest cost
Wider costs and benefits
Whole-of-life costs

11. Value for Money Resources / Investment Often VfM is understood by
comparing unit costs
Resources /
Investment
Service & Wider
Outcomes
Money
Economic
People
Inputs
Outputs
Social
Environment
Environmental
Real VfM is achieved
by comparing outcomes
with investment

12. How does SROI fit in?
Framework for telling us how effectively money is spent – make VfM decisions
Adjusted form of cost-benefit analysis.
SROI = value of positive + negative outcomes investment (or cost)
Essentially a measure of the efficiency of achieving outcomes
It considers triple-bottom-line benefits and investments – economic, social & environmental

13. History of SROI Mid 1990s: REDF & Jed Emerson Early 2000s:
new economics
foundation
Mainstreaming:
nef consulting
& others

14. SROI today Third Sector: SROI Guide Local Government: commissioning
Think tanks/public policy
Central Government
Department of Health
National Audit Office: VfM
DFID: VfM
Pilot: HIV Aids Alliance
International interest
Europe, Australia, Canada, Asia

15. Crash Course on SROI: Understanding change

16. SROI Principles 1. Involve Stakeholders 2. Understand what changes
3. Value the things that matter
4. Only include what is material
5. Do not over claim
6. Be transparent
7. Verify the result

17. SROI Process Engage stakeholders to identify outcomes Data collection
Deadweight, attribution, displacement
Benefit period and drop off
Model and calculate
Valuation of non-traded outcomes
Report

18. Features that enhance VfM measurement
Focus on outcomes
Places quality and effectiveness at heart of analysis
Measures what matters, not what is easiest
Stakeholder-informed
Grapples with outcomes measurement
Values traded and non-traded outcomes
Failing to monetise non-traded outcomes effectively they gives these a value of zero
Rigour and transparency
Concerned with impact: deadweight, displacement, attribution, drop off

19. Develop a theory of change
Inputs
Activities
Outputs
Outcomes
Output: Tells you an activity has taken place and is usually quantitative (e.g. number of people trained)
Outcome: The change that occurs as a result of an activity (e.g. improved well-being of training participants)

20. Case study: African Community Development Foundation (ACDF)
 Background
ACDF started of as a U.K community based organisation, supporting
socially isolated Africans in various ways.
In 2009, ACDF sent 28 volunteers as a pilot in Kisumu, Kenya - known as Diaspora Development Associates (DDA).
ACDF specialise in team volunteering
Kenya has a preference for formal employment rather than self-employment but ormal employment hard to get
High level migration from Kisumu to Nairobi to secure jobs
Due to preference for formal employment, very limited public interest for business training in Kisumu so organisations do not provide it.
The Government of Kenya in 2008 launched an ‘enterprise fund’ to encourage people to start their own business but there was hardly any uptake of these funds in Kisumu. 20

21. Case study: African Community Development Foundation (ACDF)
Activity
ACDF delivered Train the Trainer courses to infrastructure organisations and to direct beneficiaries.
This comprised of DDAs with skills covering- business set up, business management, book-keeping, organisational development, fundraising etc. 21

22. Exercise 2: Impact Mapping
What changes do the stakeholders experience as a result of the project? I.e. what is the story of change?

23. X Impact map Stakeholders Inputs / Outputs Outcomes Activities
Train the trainer courses
on:
business set up, business
management, book-keeping
Kisumu’s unemployed resident X
Kisumu’s infrastructure organisations
ACDF’s volunteers (DDAs)
State 23

24. Outputs vs. outcomes Outputs do not always lead to the desired outcome
Key challenge in determining VfM is measuring outcomes
Outcomes measurement rarely takes place
Biggest obstacle in our work across sectors/organisations is the lack of outcomes data

25. Crash Course on SROI: Measuring & valuing change

26. Measuring outcomes Some outcomes are ‘hard to measure’
subjective, qualitative, diverse, far removed from the activity, long term
Systems and tools for capturing complex outcomes are improving
distance travelled
translating into quantitative format that enables aggregation
collecting complex information simply

27. Evidencing outcomes
Distance travelled

28. Example: National Accounts Model of well-being

29. Evidencing outcomes
Select ‘ways of knowing’ that an outcome (a change) has taken place = indicators
Stakeholder
Stakeholder (A): long term unemployed Kisumu resident
Outcome
Beneficiary (A) has sustainable, meaningful employment
Indicators
(A) sustains employment for at least 6 months
(A) reports levels of job satisfaction
(A) improved financial situation

30. Ways of measuring Stakeholder Outcome(s)
Indicators (Data Collection Method)
Stakeholder (B): Volunteers (DDAs)
Increased employability
(B) increase in salary
Increased well-being e.g. confidence, empowerment, aspiration
(B) reported improvements in well-being
Stakeholder (C): Kisumu State
Improved economic situation
Increase in tax take
Reduction in use of state services (if relevant)
Stakeholder (D): UK State
Improved economic situation (from volunteering)
Increased tax take from salary increase

31. Valuing
SROI analyses value to all material stakeholders, not just the one funding the activity.
Key question: “what does this (change) mean to you?”
Exercise
Imagine your employer gave you a day off. What will you do on your day off? What does this free time mean to you?
If you had to put a financial value on this, what would it be? I.e. what is this day off worth to you?

32. Valuing and Pricing Value means different things to different people.
How much is your house/flat worth?
Who sets the price of fish?
BUT
What does it mean to a Kisumu resident to have a job?
SROI uses financial proxies to estimate the social value of non-traded goods to different stakeholders.
Common currency

33. Subjectivity of value
Value is adjusted...not by any accurate measure, but by the higgling and bargaining of the market, according to that sort of rough equality which, though not exact, is sufficient for carrying on the business of common life.

34. is sufficient for helping
Subjectivity of value
Can I put a value
on anything?
Can I find a value
which, though not exact,
is sufficient for helping
me evaluate change?

35. Financial proxies Sometimes this is straightforward
mainly with outcomes to the State (e.g. value of health)
More difficult with non-traded outcomes
These do not have a ‘price’ and so require a proxy, or stand-in (e.g. emotional well-being)
Standard economic valuation techniques
Contingent Valuation
Willingness to pay (e.g. new park) or willingness to accept compensation (e.g. noise pollution)
Revealed Preference
Hedonic pricing (e.g. high risk job); Travel cost method (e.g. local service); Observed spending on related goods (e.g. leisure)

36. Monetisation Each outcome is then valued
Where no direct financial value is available, we use financial proxies to represent the social value created
Stakeholder
Outcome
Possible proxy value
Beneficiary (A): long term unemployed Kisumu resident
Beneficiary (A) has sustainable, meaningful employment
£ Value of increased earning potential OR
£ Cost of fees for a skills development training course
Stakeholder (B): Volunteers (DDAs)
Increased well-being e.g. confidence, empowerment, aspiration
Contingent valuation: what would (B) pay for to get similar level of well-being?
Revealed preference – what do we observe (B) does to get a similar level of well-being?

37. Valuation exercise Exercise:
In pairs, select 2 outcomes from the ACDF impact map
Think about options for financial proxies for the outcome.
Remember, we are putting a value on the outcome for that particular stakeholder!

38. Crash Course on SROI: Attribution & Partnership working

39. Understanding impact Deadweight: what would have happened anyway?
Attribution: how much is down to this project, and how much down to other factors?
Displacement: have we just moved an outcome to / from somewhere else?
Benefit period: how long does the outcome last, and does the effect ‘drop off’ over time?

40. Impact: Attribution
Attribution – how much credit can your organisation take for the outcomes?
Expressed as a percentage
Not an exact science
Methods:
Based on your understanding of the journey of change
Involve stakeholders – interviews or surveys
Consult with other organisations

41. Attribution stages No account of attribution levels
Effectively attribution = 100%
2. Informed estimate of attribution levels
Qualitative research allows SROI practitioner to divide attribution between contributors
3. Quantitative research
Quantitative stakeholder research gives average stakeholder’s view on contributors
Academic research
Academic research can help refine (although not replace) stakeholder research

42. Quantitative stakeholder research
Attribution
Time (years)

43. Partnership working

44. Projecting into the future
Benefit period – the length of time over which outcomes are expected to endure
e.g. benefits of an employment training programme may endure for some years after the course
Drop off – the rate at which benefits decrease over time
e.g. it is likely that benefit for training participants wears off as time goes on
Time value of money : discount rate
NB. Comparing value in different countries

45. Inputs Size of investment
If just looking at one project within organisation, need to work out its ‘share’ of inputs (Full Cost Recovery)
Where multiple funding streams, need to have identified the share of benefits when scoping if looking at return on just one of these investments
Include financial and non-financial inputs e.g. volunteer time

46. Example & Organisational decision-making

47. Hypothetical Example INGO in Nigeria
Outcome: capture the value of the influence on national policy-making i.e. changes for stakeholders that result from the national policy.
INGO in Nigeria
Part of National Task Group on Sanitation
Successfully completed community-led total sanitation pilot (CLTS)
Advocated within the Task Group for drafting of National Strategy for Scaling-up Rural Sanitation and Hygiene
Key lesson: INGO’s advocacy on the National Strategy is not the outcome – it is a means to an end.

48. INGO Impact Map Outcomes Value of improved health to the individual
Value of increased economic
activity to
the State

49. Value to the individual
No monitoring in place to track impact of the National Strategy
Forecasted using secondary research and stakeholder engagement
Outcome Incidence
Assumption 1: An additional 2% of the rural population is provided with improved access to sanitation.
Disease
Number of incidences avoided
Diarrhoea
29596
Schistosomiasis
19319
Ascariasis
85601
Hookworm
9111

50. Value to the individual
Impact
Deadweight (population reached) -0.25%
baseline trend improvements in sanitation for rural communities (UNICEF/WHO)
Attribution to National Strategy 75%
significant change but concurrent factors included: International Year of Sanitation 2008 which saw increases in funding and other new initiatives
Attribution to INGO 5%
number of other actors involved in Task Group e.g. UNICEF, DFID. Credit for role in policy formation is 35% but implementation involved other actors including local government and this reduced attribution to 5%.

51. Value to the individual
Proxies – value of health to individual is made up of:
Increased income: fewer days lost to illness (working age population)
Based on secondary research on average rural wage in Nigeria
Direct costs associated with illness (all ages)
Based on WHO research into transport & medical expenses
Value of increased well-being (all ages)
Non-traded outcome.
(Limited) stakeholder engagement to find out what investments rural Nigeria might make to improve well-being.
Cost of generator divided by 5 for value to individual.
All proxies converted in US$ using World Bank PPP

52. Value to the individual
Benefit Period
Health improvement dependent on continued access to sanitation
WaterAid research suggests 5 years benefit period
Policy assumed to be in effect for 3 years
Drop-off
WaterAid research suggests 16% drop-off over 3 years.
Year 1: 2%
Year 2: 4%
Year 3: 10%
Years 4 & 5: 20% per year (nef estimate)
Discount rate: 10% (DFID)

53. Value to the individual
Social Value
Total value of improved health to individuals as a result of policy $22,561,910.66
INGO share of value $1,128,095.53
Cost incurred by INGO during policy influencing $14,863.00
Return on investment
$1:$75.90
Caution: benchmarks needed to understand whether ratio represents VfM

54. Value to the individual
Sensitivity Analysis ($1:$75.90)
Assumptions in model systematically varied.
Decreasing outcome by 95% = $1:$11.81
Well-being proxy is a sensitivity
Doubling proxy = doubling ratio
Removing proxy = $1:$1.21.
Proxy informed by limited stakeholder engagement therefore worth investigating further
Reducing attribution to 2% reduced ratio to $1:$30.36

55. Decision-making: organisational
Model to commission/plan/ design services or projects
Recommendation to DFID
Adoption by Local Authorities

56. Organisational Decision-Making

57. Sustainable Commissioning
Commissioner & service user priorities
Community strategy & Corporate priorities 1.
Activity 2.
Output 3.
Service level outcomes 4.
Camden Community outcomes
social
economic
environment 5.
Value
Quantitative
Qualitative
Monetiseable
Where value accrues:
To Service
Camden wide
central government
National outcome frameworks

58. Answering key questions
As a sector, how do we take this forward as a sector in terms of demonstrating the value and impact of our work?
Is this is an approach that can be useful?
In what circumstances can it be useful?
What are the limitations of this approach?
Where are the opportunities to for this to be adopted?
Any other reflections/thoughts on the approach?

59. Summary: SROI as VfM measurement
Focus on outcomes
Places quality and effectiveness at heart of analysis as only outcomes tell you that change has occurred
Values traded and non-traded outcomes
Enables consideration of wider costs and benefits
Failing to monetise non-traded outcomes effectively they gives these a value of zero
Stakeholder informed
Ensures that you measure what matters to those experiencing the change – i.e. ‘quality of fit’ for service user
Rigour and transparency
Concerned with impact: deadweight, displacement, attribution, drop off 59

60. Summary
SROI is a tool for assessing VfM and for achieving continuous improvement
Proving and improving
Not an ‘all or nothing’ approach, complement existing M&E systems
Next steps – from Bond or nef
Pilot?