1. Implementing LibQUAL+ ® Massachusetts LSTA Orientation MCCLPHEI Consortium Boston, MA November 1, 2006 Martha Kyrillidou Association of Research Libraries old.libqual.org

2. Overview Research Foundation and Current Uses of LibQUAL+ ® Preparing to Implement the Survey The Survey Process: Initial Steps Mapping Disciplines Approving and Running Your Survey Understanding Your Results: An Overview General Discussion and Q&A

3. old.libqual.org Total Circulation

4. old.libqual.org Reference Transactions

5. old.libqual.org Assessment “The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

6. old.libqual.org Multiple Methods of Listening to Customers Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture *A SERVQUAL-type instrument is most suitable for these methods Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

7. old.libqual.org The LibQUAL+ ® Premise PERCEPTIONS SERVICE “….only customers judge quality; all other judgments are essentially irrelevant” Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

8. old.libqual.org 13 Libraries English LibQUAL+™ Version 4000 Respondents QUAL QUAN QUAL QUAN QUAL PURPOSE DATA ANALYSIS PRODUCT/RESULT Describe library environment; build theory of library service quality from user perspective Test LibQUAL+™ instrument Refine theory of service quality Refine LibQUAL+™ instrument Test LibQUAL+™ instrument Refine theory Unstructured interviews at 8 ARL institutions Web-delivered survey Unstructured interviews at Health Sciences and the Smithsonian libraries E-mail to survey administrators Web-delivered survey Focus groups Content analysis: (cards & Atlas TI) Reliability/validity analyses: Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Reliability/validity analyses including Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Vignette Re-tooling Iterative Emergent 2000 2004 315 Libraries English, Dutch, Swedish, German LibQUAL+™ Versions 160,000 anticipated respondents LibQUAL+ ® Project Case studies 1 Valid LibQUAL+™ protocol Scalable process Enhanced understanding of user-centered views of service quality in the library environment 2 Cultural perspective 3 Refined survey delivery process and theory of service quality 4 Refined LibQUAL+™ instrument 5 Local contextual understanding of LibQUAL+™ survey responses 6

9. old.libqual.org Dimensions 2000200120022003-2006 41 items56 items25 items22 items Affect of Service Service Affect Library as Place Reliability Personal Control Information Control Provision of Physical Collections Self-Reliance Information Access Access to Information

10. old.libqual.org Survey Structure – Page 2 (Detail View)

11. old.libqual.org Rapid Growth in Other Areas Languages –American English –British English –French –Dutch –Swedish –Norwegian –Finnish –Danish Consortia –Each may create 5 local questions to add to their survey Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –European Business –Hospital –Public –State Countries –Australia, Canada, Denmark, Egypt, Finland, France, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S.

12. old.libqual.org LibQUAL+ ® Participants

13. old.libqual.org LibQUAL+ ® Surveys by Type Years 200020012002200320042005 Academic law 12510 Academic military 6 1 College or university1341111244150191 Community college 1629314 European business 5 Family history library 1 Health sciences library 135231312 Hospital 101 New York Public library 1 Public 411 Smithsonian 1 1 State 1 1 University/TAFE 2 Electronic 1 Natural Resources 2 FFRDC 5

14. old.libqual.org LibQUAL+ ® Languages

15. old.libqual.org LibQUAL+ ® Surveys by Language Years 200020012002200320042005 American English1342164285176207 British English 202231 Continental French 1 Dutch 1 Dutch English 11 French Canadian 214 Swedish 52 Swedish English (A.E.) 21 Afrikaans 4

16. old.libqual.org LibQUAL+ ® Surveys by Consortia 200020012002200320042005 AAHSL 135211412 AJCU 20 Alabama Academic (NAAL) 1018 CES 6 City University of New York 19 CUC 8 EBSLG 6 FFRDC 5 Hospital/MLA 7 MERLN 6 NELLCO 8 NY3Rs 76 2 Oberlin 12913 OhioLINK 5745114 SCONUL 201716 State Universities of Florida 62 University of Wisconsin System 14 VALE 12

17. old.libqual.org Participating Libraries by Country Country200020012002200320042005 Australia 16 Canada13481015 Denmark 1 Egypt 1 France 1 Ireland 11 Netherlands 11 Sweden 34 Switzerland 2 U.A.E. 1 UK 201716 South Africa 12

18. old.libqual.org Surveys by Session 2004-2005 YearSession 1Session 2 20042022 200519956

19. old.libqual.org World LibQUAL+ ® Survey

20. old.libqual.org Preparing for the Survey Implementation

21. old.libqual.org Why is Your Library Participating in LibQUAL+ ® ? Institutional goals: –what do you want to get out of the survey? Survey requirements: –people, supplies, technology

22. old.libqual.org Get Permission from your Institutional Review Board Local group that approves human subject research May also be called –Committee for Human Subject Research –Committee for the Protection of Human Subjects Not all institutions need to obtain permission

23. old.libqual.org Get Permission from your Institutional Review Board If permission is required: –Seek well in advance of survey –Supply a copy of the survey, if requested –Inform them that results will be shared among participants No need to inform LibQUAL+ ® of decision

24. old.libqual.org Determine Whom to Survey Random sample or Entire population

25. old.libqual.org If You Sample… Recommendations: –At least 1,200 random email addresses for each user group –Separate sample groups for undergraduates, graduates, faculty, and staff –Think about survey fatigue if you want to repeat the survey regularly Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

26. old.libqual.org If You Survey the Entire Population…. Recommendations: –Think about survey fatigue if you want to repeat the survey regularly Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

27. old.libqual.org Obtain Email Addresses Typical sources include: –Campus computing office –Campus administrative records office –Library patron database

28. old.libqual.org Promoting your Survey –Place ads in campus newspaper –Write article for library newsletter –Post flyers around campus –Present at faculty meetings & student orientations –Create a survey Web site and feature on library’s home page –Take the survey to where users are: dining halls, study rooms, dorms

29. old.libqual.org Become Familiar with LibQUAL+ ® Resources Management Center http://www.libqual.org/Manage/index.cfm LibQUAL+ ® Procedures Manual http://www.libqual.org/Manual/index.cfm –Updated Version – September 2006 –Much more detail Discussion list LIBQUAL-L@listserv.tamu.edu

30. old.libqual.org The LibQUAL+ ® Management Center

31. old.libqual.org The Survey Process: Initial Steps

32. old.libqual.org Mapping Disciplines

33. old.libqual.org Mapping Disciplines Results notebooks summarize findings by user group and provide a chart for, both, standard and custom disciplines Standard disciplines Customized disciplines –Recommend no more than 10 disciplines, if possible

34. old.libqual.org The Standard Disciplines Agriculture/Environmental Studies Architecture Business Communications/ Journalism Education Engineering/Computer Science General Studies Health Sciences Humanities Law Military/Naval Science Other Performing & Fine Arts Science/Math Social Sciences/ Psychology Undecided

35. old.libqual.org Customized Disciplines Use your local terminology to map to the standard disciplines Cautions: –Need to provide representativeness data for each discipline –Too many choices present challenges to users

36. old.libqual.org Standard Disciplines: A PA Academic Library

37. old.libqual.org Customized Disciplines: A PA Academic Library

38. old.libqual.org Getting Practice with Mapping Let’s practice mapping your disciplines

39. old.libqual.org Approving and Running Your Survey

40. old.libqual.org Understanding Your Results

41. old.libqual.org Results Notebooks Sections for Overall, Undergraduates, Graduates, Faculty, Staff, Library Staff include: –Demographic Summary –Core Questions Summary –Dimensions Summary –Local Questions –General Satisfaction Questions –Information Literacy Outcomes Questions –Library Use Summary Appendix describing changes in the dimensions and the questions included in each dimension.

42. old.libqual.org Results Results Notebooks (PDF) http://old.libqual.org/Manage/Results/index.cfm –Group Notebook (PDF) –Unified Notebook (PDF) –User Comments –Excel Data File SPSS data files (if requested at registration) will be e-mailed to the primary contact two to three months after the session has closed.

43. old.libqual.org Core Items and Dimensions 22 core items (i.e., questions) Three dimensions: Affect of Service – 9 questions Information Control – 8 questions Library as Place – 5 questions

44. old.libqual.org Understanding a Radar Chart

45. old.libqual.org Key to Radar Charts

46. old.libqual.org What Do the Colors Mean? Green And Blue

47. old.libqual.org What Do the Colors Mean? Red

48. old.libqual.org What Do the Colors Mean? Little Yellow

49. old.libqual.org What Do the Colors Mean? A lot of Yellow

50. old.libqual.org Dimension Summary

51. old.libqual.org Library Use Question (Unified)

52. old.libqual.org 2006 LibQUAL+ ® Highlights: Overall

53. old.libqual.org Qualitative Analysis: User Comments About one-half of users include comments on their surveys User Comments available on the LibQUAL+ ® Web site –Download comments in Excel or text file Skim the comments Conduct ATLAS.ti analysis

54. old.libqual.org General Discussion and Q&A

55. old.libqual.org Summary and Closure

56. old.libqual.org LibQUAL+ ® Resources LibQUAL+ ® Web site: http://old.libqual.org Publications: http://old.libqual.org/publications Events and Training: http://old.libqual.org/events Gap Theory/Radargraph Introduction: http://old.libqual.org/Information/Tools/libqualpresentation.cfm LibQUAL+ ® Procedures Manual: http://old.libqual.org/Publications/index.cfm

57. old.libqual.org The LibQUAL+ ® Team MaShana Davis Technical Communications Liaison mashana@arl.org Richard Groves Customer Relations Coordinator richard@arl.org Martha Kyrillidou Director, Statistics and Service Quality Programs martha@arl.org