1. Economic Impact Toolkits for Archives, Museums and Libraries ALMA UK 1 st December 2010 Oliver Allies & Jamie Buttrick

2. Introduction  Our Brief  To analyse economic impact methodologies for the sector  Assess the pros and cons and thinking behind each method  Recommend methods for the creation of toolkits, suitable for application across a range of organisations  Recommend an outline process for piloting these

3. Our Approach  Extensive desk based review of economic impact research (reviewed c.60 research documents)  Consultation with (19) key stakeholders from each representative sector to discuss feasibility of data capture  Progress session – (today)  Workshop session with ALMA Working Group 1 st December

4. Economic Impact  Definition “Examines the economic effect of an activity on a specific geographical area”  Origin  Sporadic usage in 1980s (although said to have emerged in 1960s – John Galbraith)  More commonplace in the 1990s linked to appraisal and evaluation in the public sector (HM Treasury Green Book first published in 1991)  Widespread usage over the last 10 years linked to the need to identify “additionality” and return on investment (English Partnerships Additionality Guide 2001, Impact Evaluation Framework, BERR 2008)

5. Economic Impact Approaches  Many different types of impact assessment. Three techniques are most commonplace for the assessment of economic impact:  Multipliers – calculating expenditure impacts and the “multiplier effects” of these  Contingent Valuation – (willingness to pay/willingness to accept)  Return on Investment – broadly combining the two approaches above to generate a “ratio”

8. Economic Impact Approaches Multiplier  Seeks to capture and map expenditure related impacts from an organisation over a given area/areas  Procurement spend  Spend on employees  Visitor related spend  Utilises Keynesian Multipliers to assess the “indirect” and “induced” effect of this spend

9. Economic Impact Approaches - Multiplier

10.  Pros  “Relatively” straightforward and resource efficient  “Benchmarks” available and considered appropriate  Most widely used and recognised impact approach  Cons  Relatively narrow in focus – tendency to overlook social impacts  Most suited to those activities that offer an economic return  Additionality and attribution aspects can cause confusion – particularly visitor spend

11. Economic Impact Approaches  Contingent Valuation  More widely associated with an assessment of the public’s “willingness to pay” (WTP) or “willingness to accept” (WTA)  WTP - Surveys of users and non-users of services to assess what value they would pay for those services (if they were free)  WTA – the level of compensation an individual (user and non) is willing to accept for the loss of a service/good  Popular in the United States and used by the British Library

12. Economic Impact Approaches – Contingent Valuation  Pros  A means to capture value placed on non-marketed goods and services  An effective way of capturing “intrinsic value”  Could utilise benchmarks (derived from pilot or targeted analysis) for aggregation  Cons  Demands extensive survey consultation carefully worded to avoid confusion  Is a challenging concept for the public to grasp in relation to both libraries and archives in particular  The challenge of capturing non-user value

13. Economic Impact Approaches  Return on Investment  Offers a combination of contingent valuation (CV) and multiplier techniques  Seeks to provide a ratio of return per pound invested (e.g. £2.60 return for every £1 invested)  Is increasingly utilised to assess government spend (particularly RDA investment)  Has driven the desire to capture economic values for all activity delivered…emergence of Social Return on Investment (SROI)

14. Economic Impact Approaches – Return on Investment  Pros  Enables the greatest breadth of activity to be captured  Is equipped (where possible) to provide values associated with social activities (through the adoption of SROI techniques)  A focus on user value would overcome some confusion  Cons  Can demand extensive resources  Due to diversity of socially related activity it would not be possible to capture social returns through a toolkit approach  Demands training and knowledge of SROI

15. Assessing the Impact of Socially Oriented Activities  Danger of overlooking diversity of activities delivered within the sector  Capturing social value increasingly popular over the last 5-10 years  Emerged within this sector e.g. Generic Learning Outcomes and Generic Social Outcomes  Challenge of capturing this data in an effective, robust and consistent manner… e.g. 22 different approaches identified (NEF 2005)  Social Audit and Accounting and Social Return on Investment (SROI) most prominent

16. Stakeholder Feedback  Broadly, greater understanding and experience of impact assessment in museums sector than in libraries/archives  In general information related to the multiplier approach was not deemed too onerous  General recognition of the need for impact assessment and willingness to participate  Consensus of the need to know what to collect, when and why

17. Stakeholder Feedback  Require clarity on what value they will gain from the exercise  Resource (time and personnel) main barrier to participation  Alignment to frameworks or measurements or performance measurements will boost participation  Represented a broad and extremely diverse spectrum of social activity

18. Implications of Findings  Clarity on what are the goals  Advocacy?  Benchmarking performance?  Clarity on what are the returns  Fine balance of an organisation or individual’s return on investment from participating in this research  Consistency on what messages are provided

19. Implications of Findings – Toolkit Potential?  Is the development of a “toolkit” and its roll-out a feasible prospect?  Does a one-size fits all approach seem possible?  What type of “toolkits” are already being used?

20. Toolkit Review  Few toolkits exist with a specific economic impact remit  Toolkits as guides  AIM Toolkit (DC Research)  Contingent Valuation Toolkit (JURA)  Toolkits as “tools”  Scottish Enterprise Additionality toolkit  For effective aggregation a combined guide and toolkit is required  Needs to be simple and straightforward but robust

21. Toolkit Options  Format  Electronic favoured by stakeholders (albeit with scope to provide hard copy)  Could enable aggregation of impact yet provide individual organisations with data (a “hook”)  Ownership and support challenges

22. Toolkits – Initial Proposals  Impact Approach – Museums/Archives  Multiplier methodology for assessment of Museums and Archives is fit for purpose  Options of Approach (resource implications)  Low - Hard copy guide  Medium - Provide a electronic/hardcopy guide and toolkit  High - Provide online resource for download/upload of returns

23. Toolkits – Initial Proposals – Museums and Archives  “Medium” resource approach (ERS preferred approach) as:  Facilitates organisational impact  Scope for aggregation  Utilise existing data where possible to minimise resource investment (PSQG and Gift Aid for example)  Potential agreement for required primary research  Piloting  Suggest testing internally prior to initial roll-out  Clarity of messages and promotion  Managed roll-out and testing with “early adopters”

24. Toolkits – Initial Proposals  Impact Approach – Libraries  Multiplier – whilst possible, in danger of underplaying scale of impact  Within the United States ROI increasingly utilised – based on UK research (Economic Value of Public Libraries in the UK – Morris et al 2002)  ROI approach in US now standardised and available as an online calculator - http://www.chelmsfordlibrary.org/library_info/calculato r.html http://www.chelmsfordlibrary.org/library_info/calculato r.html  ROI approach in the UK would incorporate elements of the Museum/Archive tool (procurement and employment) alongside user value of services – derived from a survey)

25. Toolkits – Initial Proposals – Libraries  Proposed Option  ROI – user values combined with procurement and employment expenditure  Approach  Design survey through ALMA UK to capture range of library services that demand a user-value  Survey of xxxx library users to obtain perceived value of services  Establish Calculator tool to supplement multiplier tool to establish return on investment.

26. Questions? Oliver Allies 07725 672068 oallies@ers.org.uk www.ers.org.uk