Developing a self evaluation and improvement model Being the best we can Developing a self evaluation and improvement model Scottish Library and Information Council Scottish Library and Information Council

Background How we got to the Public Library Quality Improvement Matrix Legislative context Standards and benchmarks based on inputs and outputs Changing paradigm - local government re-organisation, People’s Network, SNP administration Increased scrutiny-Best Value, Efficient Government, Changing national policies on Culture and shared services Currently working with 28 authorities – 9 reports published, plus 9 pilot reports Europe, UK and devolved Passive standards and quantitative benchmarking, e.g. COSLA, Statutory Performance Indicators and CIPFA Lifelong learning change in attitude to skills development and building Scotland’s workforce skills base HMIEducation – Schools and Community Learning and Development Policy influence; Cultural Commission, draft Culture Bill, shared services, Scottish Library and Information Council Scottish Library and Information Council

Public Library Quality Improvement Matrix Developed 2004 European Framework of Quality Management and schools inspection 7 Quality Indicators 4 service QIs and 3 management QIs Toolkit to support roll out Benchmarks and characteristics more at www.slainte.org.uk Scottish Library and Information Council

So how did self evaluation work in Scotland in 2008? Moray QI 2 Community and personal participation and QI 6 Organisation and use of resources and space East Dunbartonshire QI 1 Access to information and QI 5 Ethos and values Edinburgh – all 7 QIs Dundee QI 3 Meeting readers’ needs and QI 5 Ethos and values Shetland – all 7 QIs Fife – QI 3 Meeting readers’ needs Falkirk QI 1 Access to information and QI 6 Organisation and use of resources and space North Ayrshire QI 2 Community and personal participation and QI 3 Meeting readers’ needs Scottish Library and Information Council Scottish Library and Information Council 4

So what were the outcomes of self evaluation in Scotland in 2008? 10 Very good 12 Good 3 Satisfactory Level 5s are predominantly in QI 2 Community and personal participation. The level 3s are in Organisation and use of resources and space and Leadership. Scottish Library and Information Council Scottish Library and Information Council 5

Some examples of strengths in 2008 Moray Council Accredited courses. Use of Individual Learning Accounts to fund two posts which improved learning opportunities offered by the library. East Dunbartonshire Range of information services available to residents in a small area. Staff are well-trained, knowledgeable and keen to share their enthusiasm for subjects and online sources with the public Edinburgh Award winning Libraries4YOU initiative, working with young teenagers to combat anti-social behaviour problems Dundee Staff commitment to needs of vulnerable groups: provision for minority groups, Customer Care Standards, literacies learners and engagement with drug and alcohol recovery groups Scottish Library and Information Council Scottish Library and Information Council 6

Scottish Government Public Libraries Improvement Fund The Scottish Government Public Libraries Improvement Fund is a £500,000 fund for a 5 year period to encourage service development in authorities who already investing and achieving progress in their public library service delivery. Authorities who undertake the self-evaluation process and are assessed externally verified as satisfactory or better will be able to submit proposals based on their action plans to the Improvement Fund. The Public Library Quality Improvement Matrix can be used as a performance management tool within local authorities. Scottish Library and Information Council Scottish Library and Information Council

Excellence frameworks Japanese established the Deming Prize in 1951. In 1987 Australia established the Australian Quality Award. In 1988 US established Malcolm Baldrige National Quality Award. 1991 European Foundation for Quality Management (EFQM) established the European Model for TQM. All models are reviewed regularly and have developed a focus on overall business excellence rather than quality. Deming Prize focused on TQM Baldrige was US Secretary of Commerce from 1981 to 1987. Award originally recognised business excellence in private sector. In 1999 extended to healthcare & education. Baldrige was keen supported of quality to improve business. The EFQM offered much more business focus than Deming or Baldrige awards – included explicit reference to business results

Australian Business Excellence Framework

Being the best we can Developed by the Library Network in Victoria Provides tools for local and state government to meet service obligations and objectives Focus on planning and continuous improvement Demonstrates the continuing relevance, contribution and impact of library services Based on the Scottish Library and Information Council’s Public Library Quality Improvement Matrix Contextualized to reflect Australian policy, the Australian Business Excellence Framework and Best Value The framework has been developed by the Library Network to provide tools to enable local and state government to meet their service obligations and objective. This framework will focus on service delivery planning and continuous improvement. This will demonstrate the continuing relevance, contribution and impact of library services. The Scottish Library and Information Council’s Public Library Quality Improvement Matrix was developed in 2004 and has already had some considerable success in Scotland. Almost all of the Scottish council library services are using this tool for both self evaluation and performance management. The Being the Best We Can Framework has been contextualized to reflect Australian policy, the Australian Business Excellence Framework and Best Value. Public library services have a long history of collecting statistical information on how they provide services. We now what is put in and what comes out. However, measuring what is put into services does not guarantee delivery of a good public library service.

What is self-evaluation? 3 basic steps How are we doing? How do we know? What are we going to do now? The Scots call this What? So what? Now what? How well are we doing? –an assessment of current practice  Services need to assess the impact of their work with individuals and communities to know that the services are Being the Best they can. How do we know? – gathering evidence to support assessment  Evidence is proof that tells you what you’re doing is having a direct and active benefit , leading to your users valuing the library service. What are we going to do now? – improvement planning and on-going monitoring  Once you know where the service is good it can be celebrated and efforts and resources can be focused tackling areas for improvement

What benefits are there? Enables us to take stock of current performance Measures services in terms of progress towards objectives Allows us to identify and understand our key strengths Allows us to identify areas for improvement Supports us as we prioritise areas for action The improvement plan is the key outcome for staff and users. Areas for improvement may also include building up keys strengths – perhaps by rolling out good practice in one area across provision. Going through the process also helps team working. As resources are always limited, we should make manageable lists for improvement and that means we have to prioritise.

Outputs of self evaluation Improvement plan which helps us use resources and finances effectively Reporting to stakeholders, so improved understanding of services Benchmarks for ourselves and others Training resources Case studies of good practice Underpinning a culture of quality As well as the improvement plan there are other outputs which have value for staff. This is an on-going process and not a single event which should aspire to create and maintain a culture of quality.

Why is self evaluation effective? More a process than an event Needs to be owned not imposed Must be inclusive not exclusive Must not depend on measurability Built in not bolted on Size does NOT matter Art not science Needs to be embedded in daily practice – if there’s been a successful event, put the press cutting in the evidence file afterwards. It needs to include all staff and requires ‘buy-in’ and active participation to have its fullest impact. It’s not a bean counters approach so some of the things that you can prove with statistics can help build the picture of a quality library service. Size does not matter – big services with big new buildings are not automatically advantaged in this process. It is what you do with what you have and how you meet the needs of your customers that counts. And it is an art, so a certain amount of professional judgment needs to be exercised.

Nine Step Process Step 1 Orientation Step 2 Training for staff Step 3 Deciding which Key Result Areas to review Step 4 Preparation of the evidence and identification of key strengths and areas for improvement with completion of template in toolkit for review Step 1 Orientation is likely to be carried out in with the support of the Library Network in Victoria. Takes place Week 1 Step 2 Training: Self assessment 5 Key Result Areas What is evidence and how I find it How to identify strengths and areas for improvement Writing improvements plans is largely internal and is likely to be carried out in with the support of the Library Network in Victoria. Starts Week 2 Step 3 Deciding which Key Result Areas to review is internal. From Week 2  Step 4 Preparation of the evidence and identification of key strengths and areas for improvement with completion of template in toolkit for review is internal. From Week 3

Nine Step Process 2 Step 5 Award star score Step 6 Visit and assessment with external peer reviewers Step 7 Feedback from external review team and a agreement is reached on the star award Step 8 Report Step 9 Improvement Plan Step 5 Award star score is internal. Before the external reviewers arrive in Week 10 Step 6 Visit and assessment with external peer reviewers involves internal team and is likely to be carried out in with the support of the Library Network in Victoria. Takes place in Week 10 Step 7 Feedback involves internal team and external review team feedback to Library Management Team when the key findings are discussed and a agreement is reached on the star award Takes place in Week 10 Step 8 Report Completion of report using template. By Week 14 Step 9 Improvement Plan Is internal. By Week 16

Key Result Areas and users Providing information Building skills Developing social capital Adults Children Special Needs Groups Businesses Library services need to consider the service areas in the context of different types of users.

What are we going to evaluate? Key Result Areas for Library Services Overall evaluation 1. Providing gateways to information 2. Building individual skills, capability and wellbeing 3. Developing social capital 4. Demonstrating leadership and values 5. Designing, managing and improving systems and processes The Being the Best We Can Framework is based on business excellence frameworks. These comprise of three main elements: KRAs 1, 2 and 3 fit into the impact elements KRA 4 is the leadership element KRA 5 is the process element

KRA 1 Providing gateways to information 1. Sufficiency, range and suitability of resources Why is this important? Library services should offer consistent and equitable access to print and digital collections, internet and to a range of other resources for their communities. Provision of a range of resources, fit for purpose and specific need allows individuals to make informed choices. A successful library should have a high percentage of use by its local population. Key question to ask To what extent are the information needs of all met? Things to think about What range of print and digital collections does the library offer across a range of user groups? How is the selection of information resources made? How well are the information services used by the population? What are the most frequently used services? The first part contains and over-arching statement introducing the Key Result Area. The grid layout: Key Result Area 1 Providing gateways to information: The grid layout the theme at the top a statement explaining why this matters one or more key questions to challenge service providers a sub set of questions which are intended to drill down into the Key Result Area and to aid services to reflect as they start to self evaluate. Signposts to good practice have been drawn from existing practice. Scottish Library and Information Council

Gathering the evidence Consulting users, non-users and staff Observing activities Considering measurable outcomes Examining documentation and resources

Evaluating the evidence Best done as a team Does evidence show a service strength or an area for improvement Is everyday a strength? Some evidence is neither but shows   effective and necessary processes Template in the toolkit This is best done as a team. Organise the evidence and decide whether the particular piece of evidence supports the case that there is service strength or is an area for improvement. It is important to consider whether common practice is a strength, so it is helpful to think about what sets it above the ordinary. This might be because a particular project has won awards, has a particular strong partnership enabling extended reach into communities or has been developed in a unique or innovative way.   Not all evidence will fall into one category – for example a project can be a strength but also an area for improvement because it is already good and needs additional development to make it very good or extend its scope. Some evidence will support effective and necessary processes and systems but not fall in either category. There is a template in the toolkit and references can be added to the grids. There is no need for lengthy explanations.

5 Star Awards  Outstanding delivery of core and a range of enhanced services Excellent Sector leading and worthy of wider dissemination  Major strengths High standard of provision Any weaknesses do not impact on users’ experience The framework applies a five star rating system to indicate the quality of library service performance. This option is chosen after consideration of the use and impact of a similar self-evaluation scheme in Scotland and the rating systems for the Australian Business Excellence Framework and Best Value. Crucially, it is a system well understood by all.  

 Important strengths that have a positive impact 5 Star Awards 2  Important strengths that have a positive impact Weaknesses do not have substantial adverse effect Weaknesses are recognised and action is being taken to improve  Satisfactory provision of core services Provision with a few strengths and examples of good practice Weaknesses have some adverse effects on some users Continued  

 Limited range of services or weaknesses in a core service 5 Star Awards 3  Limited range of services or weaknesses in a core service Weaknesses have a significant impact on the quality of the users’ experiences Planned action required for improvement Contiued  

Illustrative models These provide an example of what star awards mean in practice. Between the descriptions set out in the five star rating system and these illustration, services are asked to make their own star award. As more services go through the process the Library Network will be able to collect information and benchmarks which will also be helpful.

External review It is always useful to have a comparative view. Peers will be acting as external reviewers of the self-evaluation process and outcomes. The Library Network will help co-ordinate peer review and there is support documentation for them.

Being the Best We Can – Pilot Initial training, pilot library April Self-assessment, pilot library May-June Verification & report July Review & modify process August Training, second round pilots September Second round self-assessments October - March

Thoughts from working group The rigour of the process It’s a journey – it’s constantly informed Self evaluation which leads to a cycle of improvement Be creative and consistent where you find the evidence It’s about telling your own story It’s all staff engagement It’s a learning and knowledge experience It builds the evidence base for advocacy – locally, state-wide and nationally!