From statistics to quality measures: evaluation of libraries Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and performance evaluation Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and performance evaluation Zagreb 2011

Zagreb 2011 Methods of quality assessment in libraries Performance indicators measure the effectiveness and cost- efficiency of library services: quantitative, objective User satisfaction surveys measure the perceived quality, the users‘ impression of library services: qualitative, subjective Impact (outcome) assessment tries to show the benefits, the value for individual users and society

ISO Technical Information and committee 46: documentation Subcommittee 8:Quality, statistics and performance evaluation WG 2: International library statistics ISO 2789 (2006) International library statistics,4th edition Revision ongoing since March 2010 WG 4: Performance indicators for libraries ISO (2008) Library performance indicators, 2 nd edition Revision ongoing since December 2010 WG 7: Quality measures for national libraries ISO TR (2009) Performance indicators for national libraries ISO standards for quantitiy and quality in libraries Zagreb 2011

ISO Technical Information and committee 46: documentation Subcommittee 8:Quality, statistics and performance evaluation WG 8: Statistical data for library buildings ISO TR (in progress) Qualitative conditions and basic statistics for library buildings Voting planned for this summer ISO TR (in progress) Statistics and quality issues for web archiving Started December 2009 WG 9: Statistics and quality issues for web archiving WG 10: Methods and procedures for assessing the impact of libraries ISO Methods and procedures for assessing the impact of libraries Started December 2010 ISO standards for quantitiy and quality in libraries Zagreb 2011

To know what the library does: Statistics Zagreb 2011

ISO 2789 International library statistics 1. Edition Edition Edition Edition 2006 Revision for the 5th edition since March 2010 Library statistics are short-lived today Zagreb 2011

ISO 2789 standardizes terminology and definitions methods of data collection (e.g. sampling) methods of analyzing the data ISO 2789 tries to include data for ISO 2789 tries to include data for all types of librariesall types of libraries all library resourcesall library resources all current library servicesall current library services Zagreb 2011

Zagreb 2011 ISO 2789 Traditional data Collection-based statistics (volumes, journal subscriptions, additions, withdrawals, etc.) Lending-based statistics (loans, interlibrary loans, reservations, copies, etc.) User-based statistics (number of registered or active users, types of users, library visits, user training, etc.) Facilities-based statistics (space, equipment, opening hours) Staff-based statistics (number and types of staff, staff training) Expenditure-based statistics (income; expenditure on resources, services or staff) These data are still needed in spite of new services

Zagreb 2011 ISO 2789 Data for electronic services Electronic services Electronic services online catalogues, library website, online reference, Internet access via the library, e-collections Electronic collection Electronic collection databases, e-serials, digital documents (incl. ebooks); can be acquired by purchase, license, gift, legal deposit Electronic usage Electronic usage searches, accesses, rejected accesses, downloads (all similar to COUNTER); virtual visits (to the library’s website) Costs of electronic collections and services

Zagreb 2011 ISO 2789: Revision New topics More details about Preservation issues Digitisation of analogue materials Tasks of National Libraries Legal deposit National bibliography Cooperation issues Number and types of partnerships Cooperative projects From ISO TR „Performance measures for national libraries“

Zagreb 2011 ISO 2789: Revision New topics Open access issues documents in institutional repositories use data of such documents Promotion of library services reports about the library in the media Library research publications or conference papers by library staff

Zagreb 2011 ISO 2789: Revision Services for mobile devices (e.g. for smartphones or PDAs) Number of services designed or formatted for mobile devices Number of accesses to a service via mobile devices

Zagreb 2011 ISO 2789: Revision Interactive services 1.Participation of the library in interactive Internet services (social platforms) e. g. Twitter, YouTube, Facebook What should be counted? Number of: social network services on which the library is using an account or has registered as member content units that the library has created for the social network service accesses (visits) to the library’s account downloads of documents that the library has uploaded on a social network subscribers (e.g. fans/followers)to the library’s account posts or new entries Each social network should be counted separately

Zagreb 2011 ISO 2789: Revision Interactive services 2. Interactive services hosted by the library (e. g. blogs, wikis, podcasts) What should be counted? Number of such services number of accesses number of entries (posts, comments, messages) received number of subscribers to the service, if adequate number of downloads, if adequate Each service to be counted separately

Zagreb 2011 ISO 2789: revision Services for special groups Services for target populations with special needs, e. g. persons with physical or health impairment persons with economic disadvantages non-native speakers persons with educational disadvantages Such services can be: special collections special events long-term services (e.g. special reference or job service) Children, young people and seniors without additional needs are not included

To know whether it is good: Quality measures Zagreb 2011

Zagreb 2011 Collections of performance indicators for libraries ISO 11620, 2nd ed. (2008), Information and documentation – Library performance indicators (45 indicators) Revision ongoing ISO TR (2008), Information and documentation – Performance indicators for national libraries (30 indicators) Poll, R. and te Boekhorst, P., 2nd ed. (2007), Measuring quality, performance measurement in libraries, Saur, München (IFLA Publications 127) (40 indicators)

Zagreb 2011 Quality in libraries user-orientation accessibility (building, collection, services) accuracy and reliability speed and currency competence and helpfulness of staff effectiveness and efficiency of processes

Zagreb 2011 What performance indicators measure Adequacy of resources - user places per capita - staff per capita Adequacy of the collections - collection use rate - availability of required titles - downloads per capita The quality of processes - shelving accuracy - speed of interlibrary lending The attractiveness of library services - library visits per capita - attandances at events per capita Cost-efficieny - Cost per database session - Cost per loan

Zagreb 2011 ISO 11620: Proposed changes Indicators to remove (outdated by IT development, ca. 9 – 10) - workstation use per capita (wireless LAN!) Indicators to be taken over from the TR for national libraries, e. g. - number of documents digitized per year per 1000 titles in the collection - percentage of the collection in stable condition New topics for indicators - same issues as for ISO additional: staff competence, services for researchers Please add!

Zagreb 2011 Performance indicators in groups of libraries: BIX – The Library Index a benchmarking system for public and academic libraries since 1999 (public libraries) resp (academic libraries) participation voluntary, but the indicators a must organized by: Bertelsmann Foundation (start); German Library Association (DBV) Participants 2010: 270 libraries  177 public libraries  93 academic libraries (13 from Austria)

Zagreb 2011 BIX – The Library Index 2 separate indices (public and academic libraries) 17/18 indicators each 4 dimensions = Balanced Scorecard (Kaplan/Morton) resources customer focus (use) efficiency development (potentials)

BIX indicators for public libraries Resources Media in the collection per capita User area in m 2 per capita Library staff per capita Workstation hours available per capita Internet services Events per capita Customer Focus (Use) Visits per capita Virtual visits per capita Loans per capita Collection turnover rate Opening hours per year per capita Efficiency Acquisitions budget per loan Employee hours per opening hour Visits per opening hour Total expenditure per visit Development Renewal rate (additions to stock) Training as percentage of all employee hours Investment budget per capita

BIX indicators for academic libraries Resources User area in m 2 per capita Library staff per capita Acquisitions expenditure per capita Percentage of acquisitions expenditure for e-media Opening hours per week Customer Focus (Use) Visits per capita Virtual visits per capita Training attendances per capita Immediate availability (immediate loans as percentage of total loans, incl. reservations and ILL) Efficiency Library expenditure per user Ratio of acquisitions expenditure to staff costs Efficiency of processes (Exemplarily: Media processing) Efficiency of processes (Exemplarily: Lending and ILL) Development Days of staff training per person Library budget as percentage of university budget Percentage of library means from third-party funds and special funds Percentage of library staff providing electronic services

Zagreb 2011 From statistics to impact assessment Statistics = input/output data Size of the library‘s population Attendances at user training sessions Performance indicators = combinations of data, describe quality Attendances at user training sessions per capita Impact „indicator“ User shows higher skills after a training session (to be measured e. g. by pre-test and post-test) Has the library‘s teaching had a positive effect? What the library offers and how it is used Has the library attracted a great part of the population?

Quality matters more than quantity