1. LibQUAL+ ® 2007: An Introduction American Library Association (ALA) Midwinter Meeting Seattle, WA January 22, 2007 MaShana Davis and Martha Kyrillidou Association of Research Libraries old.libqual.org

2. Overview Research Foundation and Current Uses of LibQUAL+ ® Preparing to Implement the Survey The Survey Process: Initial Steps Mapping Disciplines Approving and Running Your Survey General Discussion and Q&A

3. old.libqual.org LibQUAL+ ® Background and Overview

4. old.libqual.org Reference Transactions

5. old.libqual.org Assessment “The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable—i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two.” Sarah Pritchard, Library Trends, 1996

6. old.libqual.org Multiple Methods of Listening to Customers Transactional surveys* Mystery shopping New, declining, and lost-customer surveys Focus group interviews Customer advisory panels Service reviews Customer complaint, comment, and inquiry capture Total market surveys* Employee field reporting Employee surveys Service operating data capture *A SERVQUAL-type instrument is most suitable for these methods Note. A. Parasuraman. The SERVQUAL Model: Its Evolution And Current Status. (2000). Paper presented at ARL Symposium on Measuring Service Quality, Washington, D.C.

7. old.libqual.org The LibQUAL+ ® Premise PERCEPTIONS SERVICE “….only customers judge quality; all other judgments are essentially irrelevant” Note. Zeithaml, Parasuraman, Berry. (1999). Delivering quality service. NY: The Free Press.

8. old.libqual.org 13 Libraries English LibQUAL+™ Version 4000 Respondents QUAL QUAN QUAL QUAN QUAL PURPOSE DATA ANALYSIS PRODUCT/RESULT Describe library environment; build theory of library service quality from user perspective Test LibQUAL+™ instrument Refine theory of service quality Refine LibQUAL+™ instrument Test LibQUAL+™ instrument Refine theory Unstructured interviews at 8 ARL institutions Web-delivered survey Unstructured interviews at Health Sciences and the Smithsonian libraries E-mail to survey administrators Web-delivered survey Focus groups Content analysis: (cards & Atlas TI) Reliability/validity analyses: Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Reliability/validity analyses including Cronbachs Alpha, factor analysis, SEM, descriptive statistics Content analysis Vignette Re-tooling Iterative Emergent 2000 2004 315 Libraries English, Dutch, Swedish, German LibQUAL+™ Versions 160,000 anticipated respondents LibQUAL+ ® Project Case studies 1 Valid LibQUAL+™ protocol Scalable process Enhanced understanding of user-centered views of service quality in the library environment 2 Cultural perspective 3 Refined survey delivery process and theory of service quality 4 Refined LibQUAL+™ instrument 5 Local contextual understanding of LibQUAL+™ survey responses 6

9. old.libqual.org Dimensions 2000200120022003-2007 41 items56 items25 items22 items Affect of Service Service Affect Library as Place Reliability Personal Control Information Control Provision of Physical Collections Self-Reliance Information Access Access to Information

10. old.libqual.org Survey Structure (Detail View)

11. old.libqual.org Rapid Growth in Other Areas Languages –Afrikaans –American English –British English –Chinese Traditional (2007) –Danish –Dutch –Finnish –French –German –Norwegian –Swedish Consortia –Each may create 5 local questions to add to their survey Countries –Australia, Canada, Denmark, Egypt, Finland, France, Hong Kong, New Zealand, the Netherlands, Norway, South Africa, Sweden, Switzerland, UAE, U.K., U.S. Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –Electronic –European Business –Family History –FFRDC –High School (2007) –Hospital –National Health Service England –Natural Resources –New York Public –Public –Smithsonian –State –University/TAFE

12. old.libqual.org LibQUAL+ ® Participants

13. old.libqual.org LibQUAL+ ® Participants

14. old.libqual.org LibQUAL+ ® Surveys by Type Years 2000200120022003200420052006 Academic law 125106 Academic military 6 1 College or university1341111244150201219 Community college 162931527 Electronic 1 European business 5 16 Family history library 1 2 FFRDC 51 Health sciences library 1352313 9 Hospital 1011 National Health Service Eng. 10 Natural Resources 4 New York Public library 1 Public 4113 Smithsonian 1 1 State 1 13 University/TAFE 21

15. old.libqual.org LibQUAL+ ® Languages

16. old.libqual.org LibQUAL+ ® Surveys by Language 2000200120022003200420052006 Afrikaans 41 American English1342164285176207236 British English 20223150 Danish 1 Dutch 1 2 Dutch English 11 2 Finnish 1 French Canadian 214 French Continental 1 1 German 1 Norwegian 1 Swedish 521 Swedish British English 1 Swedish English (A.E.) 21

17. old.libqual.org LibQUAL+ ® Surveys by Consortia Years 2000200120022003200420052006 AAHSL 1352114127 AJCU-Academic 20 21 AJCU-Law 1 Alabama Academic (NAAL) 10182 CES 6 Church Libraries and Archives 7 CUNY Libraries 19 CONSULS 5 CUC Group 8 EBSLG 6 17 Research Centers (FFRDC) Libraries 5 Georgia Consortium 19 Harrisburg Area Community Colleges 5 Keystone Library Network 15 Hospital/MLA 7

18. old.libqual.org LibQUAL+ ® Surveys by Consortia (cont’d) Years 2000200120022003200420052006 Mass-LSTA 5 MERLN 6 National Health Service England 10 North Carolina Community Colleges 15 NELLCO 8 NY3Rs 76 21 Oberlin Group 129132 OhioLINK 57451142 SCONUL 20171620 State Universities of Florida 621 University of Wisconsin System 14 VALE 121

19. old.libqual.org Participating Libraries by Country Country2000200120022003200420052006 Australia 162 Canada1348101511 Denmark 1 2 Egypt 1 2 Finland 2 France 1 2 Ireland 112 Netherlands 11 5 New Zealand 1 Norway 2 South Africa 128 Sweden 344 Switzerland 2 2 U.A.E. 1 U.K. 20171633

20. old.libqual.org Surveys by Session: 2004-2006 YearSession 1Session 2 20042022 200519956 200620593

21. old.libqual.org World LibQUAL+ ® Survey

22. old.libqual.org Preparing for the Survey Implementation

23. old.libqual.org Why is Your Library Participating in LibQUAL+ ® ? Institutional goals: –what do you want to get out of the survey? Survey requirements: –people, supplies, technology

24. old.libqual.org Get Permission from your Institutional Review Board Local group that approves human subject research May also be called –Committee for Human Subject Research –Committee for the Protection of Human Subjects Not all institutions need to obtain permission

25. old.libqual.org Get Permission from your Institutional Review Board If permission is required: –Seek well in advance of survey –Supply a copy of the survey, if requested –Inform them that results will be shared among participants No need to inform LibQUAL+ ® of decision

26. old.libqual.org Determine Whom to Survey Random sample or Entire population

27. old.libqual.org If You Sample… Recommendations: –At least 1,200 random email addresses for each user group –Separate sample groups for undergraduates, graduates, faculty, and staff –Think about survey fatigue if you want to repeat the survey regularly Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

28. old.libqual.org If You Survey the Entire Population…. Recommendations: –Think about survey fatigue if you want to repeat the survey regularly Keep notes on your methodology as you will be asked to describe it in the Post Hoc Questionnaire

29. old.libqual.org Obtain Email Addresses Typical sources include: –Campus computing office –Campus administrative records office –Library patron database

30. old.libqual.org Marketing your Survey –Place ads in campus newspaper –Write article for library newsletter –Post flyers around campus –Present at faculty meetings & student orientations –Create a survey Web site and feature on library’s home page –Take the survey to where users are: dining halls, study rooms, dorms

31. old.libqual.org Become Familiar with LibQUAL+ ® Resources Management Center http://www.libqual.org/Manage/index.cfm LibQUAL+ ® Procedures Manual http://www.libqual.org/Manual/index.cfm –Updated Version – September 2006 –Much more detail Discussion list LIBQUAL-L@listserv.tamu.edu

32. old.libqual.org The LibQUAL+ ® Management Center

33. old.libqual.org The Survey Process: Initial Steps

34. old.libqual.org Survey Process: Initial Process The Beginning of the Survey Process

35. old.libqual.org Survey Process: Initial Process Preferences –Language (REQUIRED) –Survey Start & End Dates –Local Incentive (REQUIRED) –SPSS Data File Delivery –Support E-mail Address (REQUIRED) –Upload Institution’s Logo (REQUIRED) Customization –Local Questions –Disciplines –Branch Libraries

36. old.libqual.org Survey Process: Preferences

37. old.libqual.org Survey Process: Customization Local Questions

38. old.libqual.org Survey Process: Customization Branch Libraries The library do you use most often.

39. old.libqual.org Mapping Disciplines

40. old.libqual.org Mapping Disciplines Results notebooks summarize findings by user group and provide a chart for, both, standard and custom disciplines Standard disciplines Customized disciplines –Recommend no more than 16 disciplines, if possible

41. old.libqual.org The Standard Disciplines Agriculture/Environmental Studies Architecture Business Communications/ Journalism Education Engineering/Computer Science General Studies Health Sciences Humanities Law Military/Naval Science Other Performing & Fine Arts Science/Math Social Sciences/ Psychology Undecided

42. old.libqual.org Customized Disciplines Use your local terminology to map to the standard disciplines Cautions: –Need to provide representativeness data for each discipline –Too many choices present challenges to users

43. old.libqual.org Standard Disciplines: An Academic Library

44. old.libqual.org Customized Disciplines: An Academic Library

45. old.libqual.org Approving and Running Your Survey

46. old.libqual.org Previewing and Approving Your Survey Preview Complete at least one full run of your preview survey Test in different settings, using different platforms and Web browsers Get library staff involved in testing Approved Can no longer make changes Live survey URL will appear at the top of the “Manage Your Survey” page marked as Survey URL

47. old.libqual.org Reviewing Comments About one-half of users include comments on their survey Download via the Monitor Survey Progress page (Excel file or text file) http://old.libqual.org/Manage/Results/Comments/index.cfm

48. old.libqual.org Sending Reminders 3-5 reminders to sample populations Thank respondents who have completed the survey Boost up marketing efforts around campus –More fliers, table tents, ads in campus newspaper –Get professors and other staff involved

49. old.libqual.org Completing Your Representativeness Questionnaire Determines how your institutional profile compares to your survey data Requires the following information: –# of individuals per user group –# of individuals within each discipline –# of males and females –Library Statistics Volumes added during the year – Gross/Total (including e-books) Total number of current serials received (including electronic serials) Total library expenditures (U.S. dollars) Personnel – professional staff, full-time equivalent (FTE) Personnel – support staff, full-time equivalent (FTE) Must be complete before closing survey

50. old.libqual.org Completing Your Representativeness Questionnaire Representativeness Completed Representativeness NOT Completed

51. old.libqual.org Closing Your Survey We recommend a survey run of at three weeks Once you close your survey: –Retrieve list of incentive winners –Fill out Post Hoc and Evaluation Questionnaires

52. old.libqual.org Completing Post Hoc and Evaluation Questionnaires Post Hoc Questionnaire Information about your survey –Sample size –# of e-mails sent –#of invalid e-mail addresses –Incentives offered –Marketing techniques –Etc. Evaluation Questionnaire Feedback about your LibQUAL+ ® experience All survey liaisons and assistants are encouraged to complete this questionnaire

53. old.libqual.org General Discussion and Q&A

54. old.libqual.org Summary and Closure

55. old.libqual.org LibQUAL+ ® Resources LibQUAL+ ® Web site: http://old.libqual.org Publications: http://old.libqual.org/publications Events and Training: http://old.libqual.org/events Gap Theory/Radar Graph Introduction: http://old.libqual.org/Information/Tools/libqualpresentation.cfm LibQUAL+ ® Procedures Manual: http://old.libqual.org/Publications/index.cfm

56. old.libqual.org The LibQUAL+ ® Team MaShana Davis Technical Communications Liaison mashana@arl.org Yolanda Glass Administrative Assistant yolanda@arl.org Richard Groves Customer Relations Coordinator richard@arl.org Martha Kyrillidou Director, Statistics and Service Quality Programs martha@arl.org