Assessing the impact of libraries Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and performance evaluation Roswitha Poll Chair of ISO TC 46 SC 8: Quality – statistics and performance evaluation Zagreb 2011

Zagreb 2011 Libraries are expensive Collections, equipment, buildings, and especially qualified staff Libraries are expensive Collections, equipment, buildings, and especially qualified staff Libraries compete for funding with cultural institutions like theatres, orchestras, museums, etc. recreational facilities like swimming baths, sports grounds, etc. Libraries compete for funding with cultural institutions like theatres, orchestras, museums, etc. recreational facilities like swimming baths, sports grounds, etc. Can we prove that we produce value for money?

Zagreb 2011 Goals for public good social inclusion equal opportunities education life-long learning free access to information cultural awareness local culture and identity Goals for public good social inclusion equal opportunities education life-long learning free access to information cultural awareness local culture and identity Equal chances and individual well-being can help to ensure social peace and stimulate learning and development. Equal chances and individual well-being can help to ensure social peace and stimulate learning and development.

Zagreb 2011 What public libraries support literacy information literacy IT skills learning life-long learning professional education health information skills for job-seeking skills in the labour market cultural enrichment contacts, communication What public libraries support literacy information literacy IT skills learning life-long learning professional education health information skills for job-seeking skills in the labour market cultural enrichment contacts, communication Better education and information, equal chances

Zagreb 2011 The academic library’s impact time saved in study and research by information provision, reference and ILL services a space for learning and research information literacy IT skills information about grants, scholarships and student jobs cultural enrichment by events (e. g. exhibitions) for the public contacts, communication The academic library’s impact time saved in study and research by information provision, reference and ILL services a space for learning and research information literacy IT skills information about grants, scholarships and student jobs cultural enrichment by events (e. g. exhibitions) for the public contacts, communication

Zagreb 2011 The library as public place a quiet, secure space a place for learning and research a learning space for children a place for reading newspapers, books … a place for cultural and other events a place for mailing and web browsing a place for meeting and communication The library as public place a quiet, secure space a place for learning and research a learning space for children a place for reading newspapers, books … a place for cultural and other events a place for mailing and web browsing a place for meeting and communication The library imparts a sense of belonging to people who could not pay or would not dare The library imparts a sense of belonging to people who could not pay or would not dare to go to other institutions to go to other institutions

Zagreb 2011 Methods for assessing impact 1. Usage statistics New or enlarged user groups (e. g. unemployed persons, young adults) Influence of new services, direct contacts … Rising use of e-services (decline of traditional use) Influence of training, active promotion Less reference questions as to use of e-resources and e-services Familiarity caused by training and information Methods for assessing impact 1. Usage statistics New or enlarged user groups (e. g. unemployed persons, young adults) Influence of new services, direct contacts … Rising use of e-services (decline of traditional use) Influence of training, active promotion Less reference questions as to use of e-resources and e-services Familiarity caused by training and information The data should be supported by other methods, e. g. user surveys The data should be supported by other methods, e. g. user surveys

Zagreb 2011 Methods for asessing impact 2. Qualitative (soft) methods surveys (in-house, telephone, mail, online) street surveys interviews focus groups, discussion groups self-assessment of users The methods produce much „anecdotal evidence“, but the results must be made quantifiable to show patterns of impact Methods for asessing impact 2. Qualitative (soft) methods surveys (in-house, telephone, mail, online) street surveys interviews focus groups, discussion groups self-assessment of users The methods produce much „anecdotal evidence“, but the results must be made quantifiable to show patterns of impact

Zagreb 2011 Methods for assessing impact 3. Quantitative (“objective”) methods 1. observation (incl. data mining) 2. correlation between library use and the user’s success 3. tests for ascertaining an increase of skills and competences (e. g. before and after user training) Methods 1 and 2 do not always yield statistically valid results. Method 2 can be affected by data protection regulations. Methods for assessing impact 3. Quantitative (“objective”) methods 1. observation (incl. data mining) 2. correlation between library use and the user’s success 3. tests for ascertaining an increase of skills and competences (e. g. before and after user training) Methods 1 and 2 do not always yield statistically valid results. Method 2 can be affected by data protection regulations.

Zagreb 2011 Methods for assessing impact 4. Economic value of library services (ROI) Calculation of time costs (replacement value of a user’s time, identified by surveys) - time invested - time saved Contigent valuation - willingness-to-pay - willingness to accept Methods for assessing impact 4. Economic value of library services (ROI) Calculation of time costs (replacement value of a user’s time, identified by surveys) - time invested - time saved Contigent valuation - willingness-to-pay - willingness to accept What would users pay for the maintenance of a service? Which sum would users accept as equivalent for the deletion of a service?

Zagreb 2011 Problems of showing the impact of libraries Impact can be very specific in a community or university, difficult to compare There are many other influences on persons; the library’s part is difficult to identify Direct causal relationships are rare But it can be shown that the library contributes to positive developments Problems of showing the impact of libraries Impact can be very specific in a community or university, difficult to compare There are many other influences on persons; the library’s part is difficult to identify Direct causal relationships are rare But it can be shown that the library contributes to positive developments

Zagreb 2011 Project global statistics: “Impact survey” Drafted 8/2009 separate surveys for academic and public libraries suitable for all countries as short as possible, not more than 15 minutes to complete easy to understand A simple survey with wide applicability Project global statistics: “Impact survey” Drafted 8/2009 separate surveys for academic and public libraries suitable for all countries as short as possible, not more than 15 minutes to complete easy to understand A simple survey with wide applicability

Zagreb 2011 Questions Questions Why do you not visit the library?  long distance, no convenient transportation  opening hours not convenient  does not have material that I want  materials that I want always on loan  staff too busy to help me  too much library jargon  not enough seats/computers  too noisy  I have no time  I get the necessary information elsewhere  other (please explain) Shows why the library has no positive impact

Zagreb 2011 How often do you visit the library? 1. the library building (including branch libraries or mobile libraries) 2. the library’s online services (electronic journals, online catalogue etc.) via the Internet How often do you visit the library? 1. the library building (including branch libraries or mobile libraries) 2. the library’s online services (electronic journals, online catalogue etc.) via the Internet What do you do when you visit the library? 15 options (public libraries), 13 options (academic libraries) Which library services do you use online? 8 options For both questions name your 3 most frequent activities What do you do when you visit the library? 15 options (public libraries), 13 options (academic libraries) Which library services do you use online? 8 options For both questions name your 3 most frequent activities

Zagreb 2011 Have you benefited from visiting the library or using its online services?  Have you developed new skills?  Have you obtained new ideas, new interests?  Have you got helpful information for - school and learning?  - job seeking?  - health and well-being?  - business and commerce?  - about your community?  Has the library helped you to save time?  Have you enjoyed the visits; have you felt comfortable?  Have you experienced the library as safe and quiet place?  Have you made contacts with other people? Have you benefited from visiting the library or using its online services?  Have you developed new skills?  Have you obtained new ideas, new interests?  Have you got helpful information for - school and learning?  - job seeking?  - health and well-being?  - business and commerce?  - about your community?  Has the library helped you to save time?  Have you enjoyed the visits; have you felt comfortable?  Have you experienced the library as safe and quiet place?  Have you made contacts with other people? Public libraries

Zagreb 2011 Have you benefited from visiting the library or using its online services?  Have you developed new skills, e.g. in information seeking?  Have you obtained new ideas, new interests?  Has the library supported you in studying?  Has the library supported you in your research?  Has the library helped you to save time?  Has the library helped you to be better in examinations?  Has the library helped you to be better in your job?  Have you experienced the library as safe and quiet place for study and reading?  Have you made contacts with other people? Have you benefited from visiting the library or using its online services?  Have you developed new skills, e.g. in information seeking?  Have you obtained new ideas, new interests?  Has the library supported you in studying?  Has the library supported you in your research?  Has the library helped you to save time?  Has the library helped you to be better in examinations?  Has the library helped you to be better in your job?  Have you experienced the library as safe and quiet place for study and reading?  Have you made contacts with other people? Academic libraries

Zagreb 2011 If this library did not exist, could you have got the same information and help elsewhere? From another library? Via the Internet? From teaching staff? From colleagues, friends etc.? In the media (newspapers, radio, television)? Perhaps, but it would have taken more time and effort No, I do not think I would have got the same information From another library? Via the Internet? From teaching staff? From colleagues, friends etc.? In the media (newspapers, radio, television)? Perhaps, but it would have taken more time and effort No, I do not think I would have got the same information All libraries Time and effort saved can mean costs saved (very interesting for funding institutions)

Comfortable user places will influence individual well-being