1. Developing a self evaluation and improvement model
Being the best we can
Developing a self evaluation and improvement model
Scottish Library and Information Council
Scottish Library and Information Council

2. Background How we got to the Public Library Quality Improvement Matrix
Legislative context
Standards and benchmarks based on inputs and outputs
Changing paradigm - local government re-organisation, People’s Network, SNP administration
Increased scrutiny-Best Value, Efficient Government,
Changing national policies on Culture and shared services
Currently working with 28 authorities – 9 reports published, plus 9 pilot reports
Europe, UK and devolved
Passive standards and quantitative benchmarking, e.g. COSLA, Statutory Performance Indicators and CIPFA
Lifelong learning change in attitude to skills development and building Scotland’s workforce skills base
HMIEducation – Schools and Community Learning and Development
Policy influence; Cultural Commission, draft Culture Bill, shared services,
Scottish Library and Information Council
Scottish Library and Information Council

3. Public Library Quality Improvement Matrix
Developed 2004
European Framework of Quality Management and schools inspection
7 Quality Indicators
4 service QIs and 3 management QIs
Toolkit to support roll out
Benchmarks and characteristics
more at
Scottish Library and Information Council

4. So how did self evaluation work in Scotland in 2008?
Moray QI 2 Community and personal participation and QI 6 Organisation and use of resources and space
East Dunbartonshire QI 1 Access to information and QI 5 Ethos and values
Edinburgh – all 7 QIs
Dundee QI 3 Meeting readers’ needs and QI 5 Ethos and values
Shetland – all 7 QIs
Fife – QI 3 Meeting readers’ needs
Falkirk QI 1 Access to information and QI 6 Organisation and use of resources and space
North Ayrshire QI 2 Community and personal participation and QI 3 Meeting readers’ needs
Scottish Library and Information Council
Scottish Library and Information Council 4

5. So what were the outcomes of self evaluation in Scotland in 2008?
10 Very good
12 Good
3 Satisfactory
Level 5s are predominantly in QI 2 Community and personal participation. The level 3s are in Organisation and use of resources and space and Leadership.
Scottish Library and Information Council
Scottish Library and Information Council 5

6. Some examples of strengths in 2008
Moray Council
Accredited courses. Use of Individual Learning Accounts to fund two posts which improved learning opportunities offered by the library.
East Dunbartonshire
Range of information services available to residents in a small area. Staff are well-trained, knowledgeable and keen to share their enthusiasm for subjects and online sources with the public
Edinburgh
Award winning Libraries4YOU initiative, working with young teenagers to combat anti-social behaviour problems
Dundee
Staff commitment to needs of vulnerable groups: provision for minority groups, Customer Care Standards, literacies learners and engagement with drug and alcohol recovery groups
Scottish Library and Information Council
Scottish Library and Information Council 6

7. Scottish Government Public Libraries Improvement Fund
The Scottish Government Public Libraries Improvement Fund is a £500,000 fund for a 5 year period to encourage service development in authorities who already investing and achieving progress in their public library service delivery.
Authorities who undertake the self-evaluation process and are assessed externally verified as satisfactory or better will be able to submit proposals based on their action plans to the Improvement Fund.
The Public Library Quality Improvement Matrix can be used as a performance management tool within local authorities.
Scottish Library and Information Council
Scottish Library and Information Council

8. Excellence frameworks
Japanese established the Deming Prize in 1951.
In 1987 Australia established the Australian Quality Award.
In 1988 US established Malcolm Baldrige National Quality Award.
1991 European Foundation for Quality Management (EFQM) established the European Model for TQM.
All models are reviewed regularly and have developed a focus on overall business excellence rather than quality.
Deming Prize focused on TQM
Baldrige was US Secretary of Commerce from 1981 to Award originally recognised business excellence in private sector. In 1999 extended to healthcare & education. Baldrige was keen supported of quality to improve business.
The EFQM offered much more business focus than Deming or Baldrige awards – included explicit reference to business results

9. Australian Business Excellence Framework

10. Being the best we can Developed by the Library Network in Victoria
Provides tools for local and state government to meet service obligations and objectives
Focus on planning and continuous improvement
Demonstrates the continuing relevance, contribution and impact of library services
Based on the Scottish Library and Information Council’s Public Library Quality Improvement Matrix
Contextualized to reflect Australian policy, the Australian Business Excellence Framework and Best Value
The framework has been developed by the Library Network to provide tools to enable local and state government to meet their service obligations and objective. This framework will focus on service delivery planning and continuous improvement. This will demonstrate the continuing relevance, contribution and impact of library services.
The Scottish Library and Information Council’s Public Library Quality Improvement Matrix was developed in 2004 and has already had some considerable success in Scotland. Almost all of the Scottish council library services are using this tool for both self evaluation and performance management.
The Being the Best We Can Framework has been contextualized to reflect Australian policy, the Australian Business Excellence Framework and Best Value.
Public library services have a long history of collecting statistical information on how they provide services. We now what is put in and what comes out. However, measuring what is put into services does not guarantee delivery of a good public library service.

11. What is self-evaluation?
3 basic steps
How are we doing?
How do we know?
What are we going to do now?
The Scots call this
What?
So what?
Now what?
How well are we doing? –an assessment of current practice
 Services need to assess the impact of their work with individuals and communities to know that the services are Being the Best they can.
How do we know? – gathering evidence to support assessment
 Evidence is proof that tells you what you’re doing is having a direct and active benefit , leading to your users valuing the library service.
What are we going to do now? – improvement planning and on-going monitoring
 Once you know where the service is good it can be celebrated and efforts and resources can be focused tackling areas for improvement

12. What benefits are there?
Enables us to take stock of current performance
Measures services in terms of progress towards objectives
Allows us to identify and understand our key strengths
Allows us to identify areas for improvement
Supports us as we prioritise areas for action
The improvement plan is the key outcome for staff and users.
Areas for improvement may also include building up keys strengths – perhaps by rolling out good practice in one area across provision.
Going through the process also helps team working.
As resources are always limited, we should make manageable lists for improvement and that means we have to prioritise.

13. Outputs of self evaluation
Improvement plan which helps us use resources and finances effectively
Reporting to stakeholders, so improved understanding of services
Benchmarks for ourselves and others
Training resources
Case studies of good practice
Underpinning a culture of quality
As well as the improvement plan there are other outputs which have value for staff.
This is an on-going process and not a single event which should aspire to create and maintain a culture of quality.

14. Why is self evaluation effective?
More a process than an event
Needs to be owned not imposed
Must be inclusive not exclusive
Must not depend on measurability
Built in not bolted on
Size does NOT matter
Art not science
Needs to be embedded in daily practice – if there’s been a successful event, put the press cutting in the evidence file afterwards.
It needs to include all staff and requires ‘buy-in’ and active participation to have its fullest impact.
It’s not a bean counters approach so some of the things that you can prove with statistics can help build the picture of a quality library service.
Size does not matter – big services with big new buildings are not automatically advantaged in this process. It is what you do with what you have and how you meet the needs of your customers that counts.
And it is an art, so a certain amount of professional judgment needs to be exercised.

15. Nine Step Process Step 1 Orientation Step 2 Training for staff
Step 3 Deciding which Key Result Areas to review
Step 4 Preparation of the evidence and identification of key strengths and areas for improvement with completion of template in toolkit for review
Step 1 Orientation
is likely to be carried out in with the support of the Library Network in Victoria. Takes place Week 1
Step 2 Training:
Self assessment
5 Key Result Areas
What is evidence and how I find it
How to identify strengths and areas for improvement
Writing improvements plans
is largely internal and is likely to be carried out in with the support of the Library Network in Victoria. Starts Week 2
Step 3 Deciding which Key Result Areas to review
is internal. From Week 2
 Step 4 Preparation of the evidence and identification of key strengths and areas for improvement with completion of template in toolkit for review
is internal. From Week 3

16. Nine Step Process 2 Step 5 Award star score
Step 6 Visit and assessment with external peer reviewers
Step 7 Feedback from external review team and a agreement is reached on the star award
Step 8 Report
Step 9 Improvement Plan
Step 5 Award star score
is internal. Before the external reviewers arrive in Week 10
Step 6 Visit and assessment with external peer reviewers
involves internal team and is likely to be carried out in with the support of the Library Network in Victoria. Takes place in Week 10
Step 7 Feedback
involves internal team and external review team
feedback to Library Management Team when the key findings are discussed and a agreement is reached on the star award
Takes place in Week 10
Step 8 Report
Completion of report using template. By Week 14
Step 9 Improvement Plan
Is internal. By Week 16

17. Key Result Areas and users
Providing information
Building skills
Developing social capital
Adults
Children
Special Needs
Groups
Businesses
Library services need to consider the service areas in the context of different types of users.

18. What are we going to evaluate?
Key Result Areas for Library Services
Overall evaluation
1. Providing gateways to information
2. Building individual skills, capability and wellbeing
3. Developing social capital
4. Demonstrating leadership and values
5. Designing, managing and improving systems and processes
The Being the Best We Can Framework is based on business excellence frameworks. These comprise of three main elements:
KRAs 1, 2 and 3 fit into the impact elements
KRA 4 is the leadership element
KRA 5 is the process element

19. KRA 1 Providing gateways to information
1. Sufficiency, range and suitability of resources
Why is this important?
Library services should offer consistent and equitable access to print and digital collections, internet and to a range of other resources for their communities. Provision of a range of resources, fit for purpose and specific need allows individuals to make informed choices. A successful library should have a high percentage of use by its local population.
Key question to ask
To what extent are the information needs of all met?
Things to think about
What range of print and digital collections does the library offer across a range of user groups?
How is the selection of information resources made?
How well are the information services used by the population?
What are the most frequently used services?
The first part contains and over-arching statement introducing the Key Result Area.
The grid layout:
Key Result Area 1 Providing gateways to information: The grid layout
the theme at the top
a statement explaining why this matters
one or more key questions to challenge service providers
a sub set of questions which are intended to drill down into the Key Result Area and to aid services to reflect as they start to self evaluate.
Signposts to good practice have been drawn from existing practice.
Scottish Library and Information Council

20. Gathering the evidence
Consulting users, non-users and staff
Observing activities
Considering measurable outcomes
Examining documentation and resources

21. Evaluating the evidence
Best done as a team
Does evidence show a service strength or an area for improvement
Is everyday a strength?
Some evidence is neither but shows  
effective and necessary processes
Template in the toolkit
This is best done as a team.
Organise the evidence and decide whether the particular piece of evidence supports the case that there is service strength or is an area for improvement. It is important to consider whether common practice is a strength, so it is helpful to think about what sets it above the ordinary. This might be because a particular project has won awards, has a particular strong partnership enabling extended reach into communities or has been developed in a unique or innovative way.
Not all evidence will fall into one category – for example a project can be a strength but also an area for improvement because it is already good and needs additional development to make it very good or extend its scope.
Some evidence will support effective and necessary processes and systems but not fall in either category.
There is a template in the toolkit and references can be added to the grids. There is no need for lengthy explanations.

22. 5 Star Awards
 Outstanding delivery of core and a range of enhanced services
Excellent
Sector leading and worthy of wider dissemination
 Major strengths
High standard of provision
Any weaknesses do not impact on users’ experience
The framework applies a five star rating system to indicate the quality of library service performance.
This option is chosen after consideration of the use and impact of a similar self-evaluation scheme in Scotland and the rating systems for the Australian Business Excellence Framework and Best Value. Crucially, it is a system well understood by all.

23.  Important strengths that have a positive impact
5 Star Awards 2
 Important strengths that have a positive impact
Weaknesses do not have substantial adverse effect
Weaknesses are recognised and action is being taken to improve
 Satisfactory provision of core services
Provision with a few strengths and examples of good practice
Weaknesses have some adverse effects on some users
Continued

24.  Limited range of services or weaknesses in a core service
5 Star Awards 3
 Limited range of services or weaknesses in a core service
Weaknesses have a significant impact on the quality of the users’ experiences
Planned action required for improvement
Contiued

25. Illustrative models
These provide an example of what star awards mean in practice. Between the descriptions set out in the five star rating system and these illustration, services are asked to make their own star award. As more services go through the process the Library Network will be able to collect information and benchmarks which will also be helpful.

26. External review
It is always useful to have a comparative view. Peers will be acting as external reviewers of the self-evaluation process and outcomes.
The Library Network will help co-ordinate peer review and there is support documentation for them.

27. Being the Best We Can – Pilot
Initial training, pilot library April
Self-assessment, pilot library May-June
Verification & report July
Review & modify process August
Training, second round pilots September
Second round self-assessments October March

28. Thoughts from working group
The rigour of the process
It’s a journey – it’s constantly informed
Self evaluation which leads to a cycle of improvement
Be creative and consistent where you find the evidence
It’s about telling your own story
It’s all staff engagement
It’s a learning and knowledge experience
It builds the evidence base for advocacy – locally, state-wide and nationally!