1. Introduction to the refreshed Value & Impact Toolkit
Dominic Gilroy
June 2016
Hi I’m {Dominic Gilroy}
This presentation aims to give a brief introduction to the new Value and Impact Toolkit covering how it was put together and some of the ways it can be used to assist NHS LKS in demonstrating the impact and value of their services.

2. Quality & Impact: Clare Edwards
Metrics
Impact
LQAF
Research
Optimising Funding and RoI
The refreshed impact toolkit has been put together by the Value and Impact Task and Finish Group – one of the work streams of the Quality and Impact Working Group of the Knowledge for Healthcare programme
@NHS_HealthEdEng #HEELKS

3. Value & Impact Task and Finish Group
Jenny Turner – East Sussex Healthcare NHS Trust (Chair)
Stephen Ayre – George Eliot Hospital NHS Trust
Alison Brettle – Acting Director PG Research, University of Salford
Douglas Knock – King’s College Hospital NHS Foundation Trust
Rebecca Mitchelmore – Isle of Wight NHS Trust
Susan Smith – Mid Cheshire Hospitals NHS Foundation Trust
Sophie Pattison – Royal Free London NHS Foundation Trust/UCL
Dominic Gilroy – Health Education England – Yorkshire & Humber
…………………………plus an enthusiastic and responsive Reference Group
Led by Jenny Turner of East Sussex Healthcare NHS Trust, the Task and Finish Group had good geographical coverage and included:
Stephen Ayre who had experience undertaking a widespread project involving questionnaires targeted at recipients of information skills training
Alison Brettle who has recently undertaken research in the North West to capture impact evidence relating to Clinical Librarians

4. T&F Brief and Definition
refreshing and renewing the existing impact toolkit
ensure suitability for use in identifying non-clinical impacts
Definition of Impact
“Difference or change in an individual or group resulting from the contact with library services”
ISO 16439:2014 Information and documentation — Methods and procedures for assessing the impact of libraries
The brief of the Task and Finish Group was to refresh the existing Impact Toolkit sited on the libraryservices website – making a particular effort to ensure that the resulting toolkit was suited to non-clinical as well as clinical impacts
We took our definition of Impact from the ISO 16439: 2014 standard which addresses methods and procedures for assessing the impact of libraries

5. Planning Scoping literature search
Analysis of fully compliant LQAF submissions for 1.3c
SurveyMonkey analysis of use of current impact tools by NHS LKS
Thorough and meticulous mapping of the outcomes of the above
As good evidence-based practitioners we started our work with a literature search to see what other pieces of work had been undertaken that were relevant to our task. A summary of useful papers was provided.
An analysis of LQAF submissions for 1.3c (the impact criteria) was undertaken examining only those submissions with full compliance in this specific criteria. In addition there was a SurveyMonkey questionnaire sent out to gather NHS LKS staff perspectives on Impact both in terms of LQAF submissions and in general.
Through these two methods we were able to gather evidence in terms of what was currently being used successfully for impact measurement and to provide an insight into current levels of usage of the toolkit.

6. Outputs
Revised Generic Questionnaire and Interview/Case Study documents
Mapping Toolkit – also providing access to other useful resources
Recommendations for future work
We identified two specific outputs from our work:
First the review of the questionnaire based on meticulous study of the evidence we had gathered to reduce the size of the questionnaire to as few elements as possible and ensuring it could be used in non-clinical as well as clinical scenarios
Alongside this we reviewed the Interview and Case Study templates to ensure they met current KfH priorities
The second main output was a piece of work we commissioned Alison Brettle from Salford University to deliver for us. This was based on her own expertise as well as some of the evidence generated by the Task and Finish Group. It was recognised that many people, in both their LQAF submissions and in their responses to our questionnaires were confusing Impact measures with other measures such as User Satisfaction. Therefore the toolkit aims to help NHS LKS to identify the type of evidence they need to gather depending on what they wish to use it for and the type of stakeholders they wish to use the evidence with.
In undertaking the work we also recognised that there were additional questions to be answered – themselves possibly generating additional work. These were captured in a recommendations report and will be mentioned briefly at the end of this presentation.

7. Value and Impact Toolkit
KfH Impact Tools
What and for whom?
Definitions
Methods
Tools
References
KfH Impact Tools
Generic Questionnaire
Impact Interview Schedule
Impact Case Study Templates
So to the toolkit itself is / will be accessible via a tab on the Knowledge for Healthcare Blog
As we realise that some LKS will be using the toolkit to go straight to the revised version of the questionnaire, interview schedule, and case studies found in the former version of the toolkit on the libraryservices website – we have made these easy to find by giving them a menu tab
To identify these as the main outputs of the T&F group we have labelled the “KfH Impact Tools”

8. Here is the generic questionnaire page with a link to the core generic questions
There is also guidance as to how the questionnaire might be used
These questions – while relatively few – represent a high proportion of the work of the Value & Impact T&F group.
The questions couldn't be developed without exploring the whole landscape of value and impact in the context of healthcare libraries.
They are based on all the research undertaken during the year and, while appearing misleadingly simple, have been pared down to what the T&F group and our reference group believe to be the bare minimum options to capture most likely responses to the survey in terms of services used and purposes for which services and resources are put.
The core questionnaire is quick to complete, and easy to administer by busy managers. It will capture quantitative data for key messages demonstrating  value to managers, library members, funders,  and the public. (X% of responders thought that use of the library resources or services contributed to reduced risk or improved safety)
While we encourage LKS to add questions to the survey to reflect local need - we do ask that these core questions be retained unchanged in any local surveys undertaken. This will allow national collation of data in the future and ensure validity
You will notice the LQAF icon with a tick on this page. The new toolkit has a range of tools which can be used to measure a range of values – such as customer satisfaction – as well as Impact. To avoid confusion as to what is acceptable for use as evidence for compliance with LQAF criteria 1.3c we have labelled the pages containing useful Impact tools with this logo.

9. The Interview Schedules and Case Study Templates have been adapted from the former versions to reflect Knowledge for Healthcare priorities and to ensure maximum chance of capturing high-level advocacy material to inform national conversations as well as local requirements

10. What they are trying to measure (it might not always be impact)
Value and Impact Toolkit
KfH Impact Tools
What and for whom?
Definitions
Methods
Tools
References
The “What and for whom” section of the toolkit provides a mapping function which helps LKS to identify:
What they are trying to measure (it might not always be impact)
Who they are measuring for – what is the audience or stakeholder
Three examples are provided which point LKS staff to the most appropriate tools for their needs.

11. Two of the mapping examples are reproduced here to give a taster of this resource
You can see that each gives a scenario from three different perspectives
A list of appropriate tools is provided at the foot of the table with links to resources provided within the toolkit where appropriate

12. The methods and tools sections provide a list of resources which can be used to measure Library and Knowledge Services
They offer a direct route in to the tools for anyone already knowing what tool or method they wish to use
The mapping tool “What and for whom?” also links to these resources

13. The tools section includes links (through “impact tools”) to other questionnaires which have been used – including the one used in the Clinical Librarian Impact Study led by Alison Brettle

14. A page has also been created to guide LKS who wish a quick reference to impact materials which are acceptable as evidence on compliance with criteria 1.3c
This – in addition to the “LQAF” icon – should assist LKS in differentiating between impact and other tools.

15. What next for the Value and Impact Task and Finish Group?
Role out and marketing
Meetings, events, s, blogs etc
Showcasing at HLG and other event
Collation
Mechanisms for providing Generic Questionnaire nationally
National collection of questionnaire results and case studies
Streamlined mechanisms linked to LQAF
Updating
Identifying a mechanism for keeping the toolkit up-to-date and evidence based
A page has also been created to guide LKS who wish a quick reference to impact materials which are acceptable as evidence on compliance with criteria 1.3c
This – in addition to the “LQAF” icon – should assist LKS in differentiating between impact and other tools.

16. Questions and comments
Value and Impact T&F Chair:
Jenny Turner