1. An analysis of the performance of research libraries in Poland – project description Elżbieta Górska The Polish Librarians’ Association Leistungsmessung und Benchmarking in Bibliotheken Internationale Konferenz, Bratislava 23.9.2008

2. Library statistics and library standard in Poland there is no institution to develop tools and methods of evaluating the work of libraries. lack of uniform criteria with which to assess and compare libraries performance. The Central Statistical Office investigates the work of libraries only fragmentarily, CSO sends out questionnaires to libraries every two years – lack of financial aspects of the performance of libraries, – lack of their infrastructure and such areas as educational activity and publishing – the data gathered in such a way is insufficient for comparative studies. The State Accreditation Commission, which evaluates the level of education in given subjects, monitors the performance of libraries merely on a general level, Rankings of educational establishments cause much annoyance and dissatisfaction among librarians.

3. Research Libraries in Poland 1.The National Library, 2.The academic libraries, 3.The libraries run by the Polish Academy of Sciences, 4.The libraries of branch R&D units 5.Selected public libraries 6.Other libraries listed by the Minister of Culture.

4. Task Group for Standardization appointed in 2001, initially as an informal group Since 2004 based on “Agreement on Realising the ‘Analysis of the Performance of Research Libraries in Poland’ Project,” The Group currently consists of 10 representatives of research libraries, representing academic, public and special libraries. The project coordinator is the Library of the University of Poznań, and it is partly financed by the Ministry of Science and Higher Education. the Group’s work has been a subject of interest by and support from the Conference Library Directors of Academic Schools in Poland

5. The general objectives of the project to formulate methods of evaluating the performance of research libraries in Poland, to obtain data for a computer database in order to define indicators and standards for research libraries in Poland, to compare the results of analyses with the indicators and standards governing the performance of research libraries abroad, to prepare and publish annual reports.

6. International standards and recommendations used ISO 11620:1998 AD1:2003 Information and documentation. Library performance indicators=PN-ISO 11620:Informacja dokumentacja – Wskaźniki funkcjonalności bibliotek ISO/TR 20983:2003 Information and documentation. Performance indicators for electronic library services PN-EN ISO 2789:2005 Informacja i dokumentacja – Międzynarodowa statystyka biblioteczna Poll R.,te Boekhorst P.Measuring Quality.International Guidelines for Performance Measurement in Academic Libraries. Polish ed. Wrocław 2004

7. Questionnaire (1) We ask about the environment, processes and services in the following spheres: staff, collection, budget, infrastructure, circulation, reference services, educational activity, publications and databases, library cooperation, events,staff professional activities, selected performance indicators

8. Questionnaire(2) 48 different type questions – statistical data e.g. size of collection, number of users – questions about services YES/NO answers e.g. electronic ordering of books 86 indicators – 19 provided by libraries – 67 evaluated automatically – list of indicators http://ssk2.bu.amu.edu.pl/standaryzacja/AFBN_wskazniki.pdf

9. Performance indicators – why so many? the need for a comprehensive analysis of the performance of libraries, the need to include in the research all the aspects covered by the questionnaires, plans to produce standards governing the evaluation of libraries in successive years, on the basis of the indicators currently analysed, a “triple” calculation of selected indicators – there is no uniform definition of a statistical student.

10. The performance indicators provided by libraries – examples the percentage share of specific expenditures in the overall budget, the percentage share of specific revenues in the overall budget, expenditures by libraries per student/user, expenditures on the purchase of collection/books per student/user, book processing speed, the percentage of books produced with the aid of computers, the percentage of catalogue records taken from outside resources.

11. The software libraries can complete, modify and send out questionnaires on-line, selected performance indicators can be calculated automatically on the basis of the information on the questionnaire. the correct completion of the questionnaire can be verified automatically, the Task Group for Standarization may conduct comparative research, each library can analyse the data and the performance indicators for a given year or for several years, and compare them with the mean values of libraries of the same type.

12. A fragment of the questionnaire form Explanations Numerical data Yes/No question

13. Most important performance indicators computed automatically ratio of total library staff to the university staff ratio of users registered to potential library users number of items/books added per student/user number of students/users per library staff member total library area per one student/user number of students/users per one study place in reading rooms number of loans per registered user number of loans per library staff member ratio of interlibrary loans to the total amount of loans.

14. Internet application website – information about project, a set of instructions, a questionnaire, useful links, survey results module for libraries – work with questionnaire – analysis of one’s own library data – compare own library’s data and performance indicators with “average type” the same library type an administrator’s module – registration of libraries, contacts with librarians the database to incorporate and register data from questionnaire a module for the Task Group for Standardization – statistical analyses on data and performance indicators

15. About project Set of instructions Questionnaires (pdf) Survey results Access to questionnaire (password) List of indicators

16. Analysis for Group or registered library Choice of indicator or data for group

17. Analysis results Own library result Other libraries’ results indicator

18. Analysis by the Task Group results Results by library types Indicator

19. Participation of libraries in the survey by number of questionnaires certified in 2002– 2006 Libraries year 20022003200420052006 State universities 68888 State technical universities 810 12 Other state universities (economic, medical ) 14654 Non state universities 1331415 Public libraries 12233 others –2232 17293144

20. Participation of libraries in the survey Registered libraries(August 2008) – 94 includes: 87 academic libraries: 55 state and 32 non state 4 public libraries 3 others Certified questionnaires by year 2002 – 17 (including 17 academic libraries) 2003 – 29 ( 25 academic) 2004 – 31(27 academic) 2005 – 44 (38 academic) 2006 – 44 (39 academic)

21. Difficulties with completing the questionnaire an absence of the statistical data needed to complete the questionnaire, an absence of comparative data on the use of electronic sources (differences in the statistics generated by providers), divergent organisational structures and models of financing educational establishments, difficulties with certifying the questionnaires – errors requiring correction, a misunderstanding of data, misinterpretation of questions.

22. The collective analysis of the data available to the public on the project website indicators from the questionnaires (calculated by the applicants) indicators calculated by the programme automatically, the percentage share of budget expenditures, data broken down into library categories, results: arithmetical average, median value, maximum and minimum values.

23. Analysis by the Group – By library types,financing,years

24. Database statistics and survey results available to the public

25. The results of analysis by type of indicators general, financial, indicators regarding collections, indicators regarding services and their use indicators regarding staff, organisational structure and vocational activity, remote catalogue services, own publications, cooperation between libraries, events.

26. The results of the analysis by the Task Group – example

27. Use of the results of the surveys registered libraries which submit questionnaires, Task Group for Standardization, librarianship practicians and theoreticians, institutions which assess libraries.

28. Plans the production of a domestic set of performance indicators in compliance with ISO 11620, a survey into user satisfaction on the basis of a standard questionnaire, the formulation of a nationwide set of standards and a definition of indicators for each of them.

29. Thank you