Impact assessment framework Natasha Innocent Senior Policy Adviser Learning and Skills MLA

Background How libraries support learning Strong roots into communities Expertise in working in partnership with a range of other partners Critical role with digital participation Support the wider benefits of learning Provide choice and flexibility Strong links to Ageing Society

What does the framework do? Helps libraries describe the impact of their learning activities on individuals and communities A common language to talk to policy makers and funders about the role of libraries in supporting learning outcomes A methodology to support libraries to plan, evaluate and develop learning activity

Inspiring Learning This framework is based on Inspiring Learning commissioned by MLA and launched in 2003 Inspiring learning is based on a broad and inclusive definition of learning Launched 5 generic learning outcomes 3 more recent generic social outcomes www.inspiringlearningforall.gov.uk

Generic learning outcomes Knowledge and understanding Increase in skills Attitudes and values Enjoyment, inspiration, creativity Activity, behaviour and progression Learning activities in libraries include any combination of the above

decide on data collection methods present and use your findings develop your story select SMART indictors decide on data collection methods analyse your results present and use your findings

Develop your story NEEDS why you are developing this learning activity and who it is for? INPUTS AND ACTIVITIES what resources and activities do you need to develop to deliver this service OUTPUTS AND OUTCOMES what you expect you will achieve

Selecting SMART indicators Indicators are used to measure, simplify and articulate your story Indicators can be descriptive, performance or efficiency based Any good indicator should be SMART – simple to implement, measurable, action-focused, relevant and time-bound

ENTITLE output indicators Number of people who have participated in library activities designed to improve their ICT skills Number of people who have participated in library activities designed to increase their interest in reading and/or improve their reading abilities

ENTITLE outcome indicators These are specific changes in the attitude, behaviours, knowledge, skills and enjoyment of users as a result of library services ENTITLE has agreed that the key generic outcomes relating to library ICT and reading activity are knowledge and understanding, skills and attitudes

How to collect your data Review the data you already collect to see to see if you already collect evidence of generic learning outcome indicators This will help you to decide on whether to adapt a data collection method you already use or develop a new method of data collection

Data you may already collect Data source Collected when Does it give evidence of learning? Comments book Daily Yes Annual library survey Yearly Maybe Evaluation forms Post project Letters from users

Quantitative research Requires a large number of respondents to ensure any sample you take is representative and averages meaningful. Usually collected by questionnaires with multiple choice questions or by structured interviews with a series of closed or multiple choice questions

Qualitative research Useful for understanding something in depth Involves talking at greater length to a smaller group of people Qualitative information is gathered by face to face or telephone interviews, focus group discussions, comments cards or books, open ended questions on questionnaires

Ethics It is important that library staff consider ethical or legal issues when undertaking research. Laws vary from country to country but there are shared principles on Consent Interviewing children Confidentiality Data Protection

Analyse your results Analysis of the data you collect will reveal if the ‘story’ you mapped out at the beginning actually happened. That means you can test your hypothesis The approach to analyse will vary according to the type of tool you used and the type of data you collected – whether it is quantitative or qualitative

Things to think about when analysing data In interpreting results you need to decide if 64% is a high or low result The meaning and significance of the results – you may need to group the data into categories depending on who the programme is targeted at Template 2 provides examples of how to link outputs and outcomes to an indicator, data collection methods and results

Present and use your findings Highlight your key messages in an executive summary Describe the purpose of the learning activity that you have evaluated Describe your chosen methodology Think about how you could effectively present your findings Conclusions – relate to your original story

Going further Make your evidence more robust by Examining the long-term impact of the activity by re-surveying people to check if there has been any further progress Share your findings with other libraries locally and nationally to establish a benchmark Compare the results for users and non-users and identify if any significant differences

Advocacy Make the most of the evidence you collect Advocacy is speaking out to win influence, gain support, recognition, partners and funding Combine your outcomes, research, policy and statistical evidence to create a powerful argument for the role your library plays in supporting lifelong learning.

For further information contact Natasha Innocent Senior Policy Officer: Learning and Skills MLA Council natasha.innocent@mla.gov.uk