1. LibQUAL+™ Origins, Design, Interpretation La Calidad en las Bibliotecas Conferencia Palma de Mallorca 13-14 January 2005 Fred Heath Vice Provost and Director, University of Texas Libraries

2. Why Assessment? “In an age of accountability, there is a pressing need for an effective…process to evaluate and compare research libraries.” u 124 Association of Research Libraries (ARL) alone, over $3.2 billion dollars were expended in 2000/2001 u 500 LibQUAL+ participants in LibQUAL+ Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.5.

3. The Challenge of Assessment in Libraries 1.Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. 2.Help funding agencies understand success of their investments 3.No demonstrable relationship between expenditures and service quality—spending money is not enough…. 4.Lack of metrics describing outcomes: how can we measure success from the user’s point of view 5.Need to redesign library services to better meet changing patterns of use

4. Libraries Remain a Credible Resource in 21 st Century Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. 98% agree with statement, “My … library contains information from credible and known sources.”

5. Changing Behaviors Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. Recent Survey: Only 15.7% agreed with the statement “The Internet has not changed the way I use the library.”

6. Library Use Summary LibQUAL+ Spring 2003 Survey – UT Austin Aggregate

7. “Googleization” Users and funders alike can begin to wonder about the relevance of libraries

8. “…everyone in class tried to get those articles on line and some people didn’t even bother to to to the stacks when they couldn’t Google them.” Graduate Student NYT Online 6/21/04 (Katie Hafner, “Old search engine in the the library tries to fit into a Google world”)

9. Facilities Usage: University of Texas Entrance Statistics - UT Austin Libraries 1991-2003

10. Printed Book Circulation: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7.

11. Research Behavior: Personal Control When searching for print journals for research: Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. Only 13.9% ask a librarian for assistance Only 13.9% ask a librarian for assistance Only 3.2% consider consulting a librarian a preferred way of identifying information Only 3.2% consider consulting a librarian a preferred way of identifying information

12. Reference Decrease: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics 2000-01. Washington, D.C.: ARL, p.7. Web-savvy users wish to be able to negotiate the information labyrinth on their own terms the information labyrinth on their own terms

13. Searches for Online Journals: University of Texas UT Austin Libraries 2002-2004 Monthly

14. Web Usage: University of Texas Total File Requests - UT Austin Libraries 2000-2003

15. Enter LibQUAL+: A response to 1.The necessity of assessment 2.Rapid shifts in information-seeking behavior 3.The reallocation of resources from traditional services into technology- enabled inquiry

16. LibQUAL+ ™ Goals 1.Improve mechanisms and protocols for evaluating libraries 2.Develop web-based tools for assessing library service quality 3.Identify best practices in providing library service 4.Support libraries seeking to understand changes in user behavior 5.Assist libraries seeking to re-position library services in the new environment

17. LibQUAL+ ™ Outcomes 1.Securing information that contributes meaningfully to planning and improvement efforts at a local level 2.Providing analytical frameworks that institutional staff can apply without extensive training or assistance 3.Helping decision-makers understand success of investments 4.Finding useful inter-institutional comparisons

19. 76 Interviews Conducted 1.York University (Canada) 2.University of Arizona 3.Arizona State 4.University of Connecticut 5.University of Houston 6.University of Kansas 7.University of Minnesota 8.University of Pennsylvania 9.University of Washington 10.Smithsonian Institution 11.Northwestern Medical

20. LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred Atlas Ti

23. LibQUAL+ ™ Participants

25. Validity Correlations Serv_AffInfo_ConLibPlaceTOTALper Serv_Aff1.0000.7113.5913.9061 Info_Con.71131.0000.6495.9029 LibPlace.5913.64951.0000.8053 TOTALper.9061.9029.80531.0000 ESAT_TOT.7286.6761.5521.7587 EOUT_TOT.5315.6155.4917.6250

26. alpha By Language By Language ServiceInfo.Lib as Group n AffectControlPlaceTOTAL American (all)59,318.95.91.88.96 British (all) 6,773.93.87.81.94 French (all) 172.95.90.89.95

28. Survey Structure – Page 2

29. “And a Box” Why the Box is so Important –About 40% of participants provide open- ended comments, and these are linked to demographics and quantitative data. –Users elaborate the details of their concerns. –Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

30. 1.Languages –American English –British English –French –Dutch –Swedish 2.Consortia –Each may create 5 local questions to add to their survey 3.Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –European Business –Hospital –Public –State 4. Countries Canada, the Netherlands, South Africa, Sweden, France, Australia, New Zealand, Malaysia, U.K., U.S. Rapid Growth in Other Areas

31. Understanding LibQUAL+ Results Measures the distance between minimally acceptable and desired service quality ratingsMeasures the distance between minimally acceptable and desired service quality ratings Perception ratings ideally fall within the Zone of TolerancePerception ratings ideally fall within the Zone of Tolerance

32. LibQUAL+ Survey Tool Conducted at UT Austin in 2001, 2002 and 2003, 2004Conducted at UT Austin in 2001, 2002 and 2003, 2004 Web-based survey sent to 1200 faculty, 1200 graduate students and 1800 undergraduatesWeb-based survey sent to 1200 faculty, 1200 graduate students and 1800 undergraduates Participants selected randomly from Participants selected randomly from University email databases University email databases 22 questions measuring users’ perceptions of library service quality22 questions measuring users’ perceptions of library service quality

33. LibQUAL+™ 2004 Summary Colleges or Universities –Faculty - American English (n = 11,755) Question view Dimension view

34. Key to Radar Charts

35. Key to Bar Charts

36. LibQUAL+™ 2004 Summary Colleges or Universities – Faculty - American English (n = 11,755) Negative gap Positive gap

37. Institutional Norms for Perceived Means on 22 Core Questions Note: Thompson, B. LibQUAL+  Spring 2002 Selected Norms, (2002).

38. Access to Information by Status LibQUAL+ Spring 2003 Survey – UT Austin Faculty at Texas less approving of collection quality than students

39. Library as Place by Status LibQUAL+ Spring 2003 Survey – UT Austin Students rate “Library as Place” more disapprovingly than Faculty (size of gap)

40. Four Dimensions – Social Science & Psychology LibQUAL+ Spring 2003 Survey – UT Austin Our Psychology faculty do not rate their access to needed collections approvingly

41. Trends: Access to Information by Status Look for steadily improving trajectories

42. LibQUAL™ Interactive Institutional Statistics Master List Your peer list of institutions

43. The very act of administering LibQUAL+™ is beneficial

45. LibQUAL+ ™ Resources 1.LibQUAL+™ Website: http://www.libqual.org http://www.libqual.org 2.Publications: http://www.libqual.org/publications http://www.libqual.org/publications 3.Events and Training: http://www.libqual.org/events http://www.libqual.org/events 4.LibQUAL+™ Bibliography: http://www.coe.tamu.edu/~bthompson/servqbib http://www.coe.tamu.edu/~bthompson/servqbib 5.LibQUAL+™ Procedures Manual: http://www.libqual.org/Information/Manual/index.cfm http://www.libqual.org/Information/Manual/index.cfm

46. LibQUAL+ ™ Contact Information 1.Martha Kyrillidou  Senior Program for Office of Statistics and Measurement  martha@arl.org martha@arl.org 2.Consuella Askew  LibQUAL+™ Program Specialist  consuella@arl.org consuella@arl.org 3.Amy Hoseth  LibQUAL+™ Project Assistant  amyh@arl.org amyh@arl.org 4.Jonathan D. Sousa  Technical Applications Development Manager  jonathan@arl.org jonathan@arl.org

47. This presentation available at: http://webspace.utexas.edu/fh355/www

48. Core Questions Summary LibQUAL+ Spring 2003 Survey – ARL Faculty

49. UT Austin vs. Peers LibQUAL+ Spring 2003 Survey – UT Austin