1. HACT Round Table Discussion
Where has the housing sector got to in measuring its social impact and what approaches are being taken?
David Mullins and Vanessa Wilkes, TSRC, University of Birmingham
Housing providers: evidencing their social impact
HACT Round Table Discussion
10:00am pm on 7 June 2012, London

2. Social Impact – A Continuing agenda for housing sector
Investing for Social Purpose in C19– Remember Peabody, Guinness? – implicit social enterprise model
Social Investors of the 1920s – COPEC & Miss Fenter’s CI (financial inclusion and youth diversionary) activities!
1960s Cathy Come Home – housing as social movement – homelessness and neighbourhood renewal
1997 Giddens/Blair Social investment state – HAs as Social Investment Agencies – ‘from housing plus-CI’
2002 In Business for Neighbourhoods – sector rebranding - CI symbol of independent social purpose
2000s Government promotion of social enterprise & SROI across third sector
Never forget the impact of secure affordable homes!!

3. Social Impact measurement in housing sector – research evidence
2008 First NHF Neighbourhood Audit – first picture of extent of CI (actually quite marginal to housing investment & management in most organisations but £435 million invested across sector, £272m from own resources)
2010 TSA Study – Community investment performance management toolkit for housing organisations - no golden bullet – wide range of tools - make or buy?
2011 Second NHF Neighbourhood Audit – important high level indication of change - but still mainly inputs & outputs
PhD Study – from inputs & outputs to outcomes and impacts - Understanding why and how housing associations measure the social impact of their community investment activities (supported by NHF and engaging with HACT)

4. TSA Study 2010
Anglo-Dutch trawl of approaches to measurement– 17 tools identified to plan, manage & measure CI activities
G15 Roundtables – scope and plans for measurement
8 case studies – 4 internal tools, 4 off-shelf
Approach influenced by scale, type & organisation of CI & level (individual, project, programme, corporate, sector)
Towards Impact - Considerable interest in moving from inputs/outputs to outcomes/impacts
No established practice - Adaptation and use of wide range of tools – choices often a condition of grant funding
NHF Audit – led to common scoping & classification activities across sector – but significant differences in range of activities and ambitions of different HAs
Launch of Community Impact Tracker as sector tool – would this standardise – enable benchmarking?

5. TSA Study 2010– Approaches & Gaps
‘need more than a good story now to fund CI’.
Projects – main focus, moving to harder quantitative
approach alongside case studies
Programmes – external accountability to funders, some strategies & theming – common reporting. Some move to standardisation & KPIs
Corporate Overview – weakly developed – CI not on balanced scorecards – BITC, SROI, Social Audit being explored by a few
Collaborative planning – weakly developed – organisational measures a barrier to collaboration? –credit claiming – going it alone
Area Based – not much progress- floor standards, neighbourhood profiles looking dated- difference between nationals & community based HAs
Ex-ante – Dutch focus on planning and goal setting – independent SEs?
Ex-post – English focus on monitoring – regulatory mindset?
Toolkit – no single tool meets all the aims – distance travelled tools for individual impacts – project management tools– corporate & sector indicators – collaborative planning tools (such as Outcomes Arena)

6. Outcomes arena – setting priorities together with partners

7. So what’s really changing?
‘Fences coming down’ – need for self-steering (more of a Dutch approach needed?)
CI mainstreamed
Economic crisis – need to harness the local £
HAs as SEs and as incubators of community and tenant based SEs
Welfare Reforms – urgency of financial inclusion work
ASB – recognition of CI investment in ‘diversionary activity’ (remember Miss Fenter) to include in cost benefit analysis of ASB responses (HouseMark)
from ‘CSR extra’ to ‘core business’

8. What Drives your Community Investment work?

9. What kind of CI – what kind of measurement
‘making sure people enjoy the projects’.
Society led – responsive & consultative (measures set with residents and communities)
Partnership led – LSPs & community commissioning (measures set with LA and community partners)
Strategy led – strategic themes set priorities = synergies with core business - (measures set corporately)
Market led – the commissioning game – Supply chains and all that)- often based on individuals rather than neighbourhoods
–(measures set by contracts)
“CI washes its face”
“If its not in the contract we don’t do it”

10. Hact Survey Methodology
- 34 Respondents
- Self selecting organisations and interviewees
- Telephone interview in November 2011
The stage of measurement activity
Size of housing association
Not started any formal measurement and looking around for tools
Fairly new to measuring and waiting to see what results the current tools give them
Currently measuring but aware that need to make the tools / indicators better
Have established measurement systems and are able to see the benefits
Medium 1
Medium-Large 3
Large 4
10,000-29,999 2 5
30,000-49,999
50,000+
7 respondents
4 respondents
10 respondents
13 respondents
Uses PHd as well as hact survey
NHF sizebands – ranged from 3,400 – 56,000 in stock
Fieldwork – Nov 2011
Self selecting sample

11. Growing importance / drivers
“Prove we are making a difference”
External
Internal
Wider third sector interest
Shift from ‘monitoring’ to ‘impact’
High profile networks
SROI
Inspiring Impact
Think tanks
Economic climate
Funders
“Keeping up” with the sector
Accountability
Tenants
Boards
Validate social as well as economic value
Growing importance and integration of community investment
Increased desire to understand neighbourhoods …. and see if making a difference
Increasing emphasis within orgs to identify effectiveness
Ec climate – changed CI approach in a lot of orgs – become more focused and inwardly driven rather than responding to individual requests
“If it’s not measured, it’s not done”

12. Approaches to Impact Measurement
Wide and varied approaches
… in tools and methodologies
Externally developed tools include:
Advice Pro; Balance Scorecard; Business In The Community; CITs; CP Tracker; CR Tracker; Lamplight; Social accounting; SROI; Views (formerly SPRS)
Paper based systems
SROI across 4 countries

13. “What does good look like”
Common Issues
Whilst doing:
Resources
Skills
Understanding complex methodologies or tools
Development of outcomes measures
Development of financial proxies
Confidence (or lack of) in reporting results
Whilst thinking about it:
No perfect off the shelf answer
Different tools for different types of projects
Too much choice .versus.
no knowledge of the options
Waiting for the golden bullet
Drawing on external resources, consultants, networks
Inter HA discussion “A common problem”
Outcomes – Which they felt accurately captured what they are trying to achieve
Overlapping

14. The Purpose of Impact Measurement
Why do it?
Caveats
Accountability
Self evaluation
Using the data
for learning
in bidding
Layers of measurement
Impact of some or all activities?
Impact as whole organisation?
Importance of marginal work
Funders demands
Use of data
What is useful?
Is it always appropriate and useful?
To housing associations
To them
63% of all respondents measured all CI activities
Marginal (often pastoral) work often harder to measure but very important
How do we think of Impact Measurement – as being ‘done’ to tenants – one way process or by looking inwardly also?

15. Lesson Learnt Steep learning curve Ambiguity in …
Build on what achieved, expand breadth and depth
Ambiguity in …
Methodology
Proxy values (e.g. SROI database)
Assessment does not give a definite answer
But … opens up debate
Tension between ‘doing’ and measuring
Expectation of partners involvement
Manage expectations
Promotes a culture change
A shared problem

16. Are you intending to change your measurement tool in the next 12 months?
Response
Reason
External tool users
Internal tool users No
Happy with current tool
Waiting to see the success of the one we are currently using 8 1
Yes
Looking around for alternative tool(s)
Further develop the current tool(s) 2 7
Don’t know 3
TOTAL 14 12
Total Respondents: 26 hact research
- Need to move towards measuring outcomes (rather than outputs) and social impact
Need to keep up to date with new tools and methodologies
Need to investigate the functionality of our current tools

17. Moving forward ?? Overall Strategy Resources
“Chaotic progress”
Moving forward ??
Overall Strategy
Demonstrating the economic as well as the social contribution of RSLs (not just community investment)
More Group structures adopting a joined-up approach
Resources
The need for appropriate time and resources
Analytical skills
Specialisation of roles

18. Moving forward cont.. Methodologies An area which needs improving
Recognition that the complexity of some approaches may not fit all organisations / social enterprises
Development of joint indicators
Greater use of proxy indicators
More methodological project planning /theory of change
Arena for ‘challenges’ within projects to be addressed
- The indicators would need to be appropriate for the smaller, as well as the larger, organisations;
= There was concern about the resources an organisation would need to put into such a process but there was a willingness to be a ‘sounding board’;
= The question of whether organisations wanted standardised measures was raised, something organisations need to consider; and
= Some concern was raised about duplication of effort given that there is already a regional grouping considering this issue.
Add some questions for the day
Implication s of type and nature of CI for measurement
What support does the sector need to take this work forward
lots of different starting points
areas on which useful to collaborate
Social and economic value within groups

19. Questions
Value of standardisation within housing sector v common approaches cross-sector?
Does measurement inhibit or enable collaboration?
How do aims of CI and delivery models (society led, partnership led, strategy led, market led) affect approaches to measurement?
Is social impact of HAs just about CI or about whole business impact? (where are the measures of social impact of secure affordable housing?)

20. More Questions
How do motivations affect type of impact measurement (external v internal drivers)
Should we wait for ‘golden bullet’ to solve problems at a sector level?
How can progress become less chaotic?
What support resources do different types of HAs need?
What can we best do together?

21. Thanks
For further info on TSA study, HACT survey and PhD please contact us at TSRC: