LibQUAL+™ Origins, Design, Interpretation La Calidad en las Bibliotecas Conferencia Palma de Mallorca January 2005 Fred Heath Vice Provost and Director, University of Texas Libraries

Why Assessment? “In an age of accountability, there is a pressing need for an effective…process to evaluate and compare research libraries.” u 124 Association of Research Libraries (ARL) alone, over $3.2 billion dollars were expended in 2000/2001 u 500 LibQUAL+ participants in LibQUAL+ Note. M. Kyrillidou and M. Young. (2002). ARL Statistics Washington, D.C.: ARL, p.5.

The Challenge of Assessment in Libraries 1.Traditional statistics emphasize inputs, expenditures, acquisitions, holdings, etc. 2.Help funding agencies understand success of their investments 3.No demonstrable relationship between expenditures and service quality—spending money is not enough…. 4.Lack of metrics describing outcomes: how can we measure success from the user’s point of view 5.Need to redesign library services to better meet changing patterns of use

Libraries Remain a Credible Resource in 21 st Century Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. 98% agree with statement, “My … library contains information from credible and known sources.”

Changing Behaviors Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. Recent Survey: Only 15.7% agreed with the statement “The Internet has not changed the way I use the library.”

Library Use Summary LibQUAL+ Spring 2003 Survey – UT Austin Aggregate

“Googleization” Users and funders alike can begin to wonder about the relevance of libraries

“…everyone in class tried to get those articles on line and some people didn’t even bother to to to the stacks when they couldn’t Google them.” Graduate Student NYT Online 6/21/04 (Katie Hafner, “Old search engine in the the library tries to fit into a Google world”)

Facilities Usage: University of Texas Entrance Statistics - UT Austin Libraries

Printed Book Circulation: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics Washington, D.C.: ARL, p.7.

Research Behavior: Personal Control When searching for print journals for research: Note. Digital Library Federation and Council on Library and Information Resources. (2002). Dimensions and Use of the Scholarly Information Environment. Only 13.9% ask a librarian for assistance Only 13.9% ask a librarian for assistance Only 3.2% consider consulting a librarian a preferred way of identifying information Only 3.2% consider consulting a librarian a preferred way of identifying information

Reference Decrease: All ARL Libraries Note. M. Kyrillidou and M. Young. (2002). ARL Statistics Washington, D.C.: ARL, p.7. Web-savvy users wish to be able to negotiate the information labyrinth on their own terms the information labyrinth on their own terms

Searches for Online Journals: University of Texas UT Austin Libraries Monthly

Web Usage: University of Texas Total File Requests - UT Austin Libraries

Enter LibQUAL+: A response to 1.The necessity of assessment 2.Rapid shifts in information-seeking behavior 3.The reallocation of resources from traditional services into technology- enabled inquiry

LibQUAL+ ™ Goals 1.Improve mechanisms and protocols for evaluating libraries 2.Develop web-based tools for assessing library service quality 3.Identify best practices in providing library service 4.Support libraries seeking to understand changes in user behavior 5.Assist libraries seeking to re-position library services in the new environment

LibQUAL+ ™ Outcomes 1.Securing information that contributes meaningfully to planning and improvement efforts at a local level 2.Providing analytical frameworks that institutional staff can apply without extensive training or assistance 3.Helping decision-makers understand success of investments 4.Finding useful inter-institutional comparisons

76 Interviews Conducted 1.York University (Canada) 2.University of Arizona 3.Arizona State 4.University of Connecticut 5.University of Houston 6.University of Kansas 7.University of Minnesota 8.University of Pennsylvania 9.University of Washington 10.Smithsonian Institution 11.Northwestern Medical

LoadedPT:P1:01xxxxxxxxxxxxxx.txt,S:\Admin\Colleen\ServQual Interviews\TEXT Only\01xxxxxxxxx.txt (redirected: c:\zz\atlasti\fred Atlas Ti

LibQUAL+ ™ Participants

Validity Correlations Serv_AffInfo_ConLibPlaceTOTALper Serv_Aff Info_Con LibPlace TOTALper ESAT_TOT EOUT_TOT

alpha By Language By Language ServiceInfo.Lib as Group n AffectControlPlaceTOTAL American (all)59, British (all) 6, French (all)

Survey Structure – Page 2

“And a Box” Why the Box is so Important –About 40% of participants provide open- ended comments, and these are linked to demographics and quantitative data. –Users elaborate the details of their concerns. –Users feel the need to be constructive in their criticisms, and offer specific suggestions for action.

1.Languages –American English –British English –French –Dutch –Swedish 2.Consortia –Each may create 5 local questions to add to their survey 3.Types of Institutions –Academic Health Sciences –Academic Law –Academic Military –College or University –Community College –European Business –Hospital –Public –State 4. Countries Canada, the Netherlands, South Africa, Sweden, France, Australia, New Zealand, Malaysia, U.K., U.S. Rapid Growth in Other Areas

Understanding LibQUAL+ Results Measures the distance between minimally acceptable and desired service quality ratingsMeasures the distance between minimally acceptable and desired service quality ratings Perception ratings ideally fall within the Zone of TolerancePerception ratings ideally fall within the Zone of Tolerance

LibQUAL+ Survey Tool Conducted at UT Austin in 2001, 2002 and 2003, 2004Conducted at UT Austin in 2001, 2002 and 2003, 2004 Web-based survey sent to 1200 faculty, 1200 graduate students and 1800 undergraduatesWeb-based survey sent to 1200 faculty, 1200 graduate students and 1800 undergraduates Participants selected randomly from Participants selected randomly from University databases University databases 22 questions measuring users’ perceptions of library service quality22 questions measuring users’ perceptions of library service quality

LibQUAL+™ 2004 Summary Colleges or Universities –Faculty - American English (n = 11,755) Question view Dimension view

Key to Radar Charts

Key to Bar Charts

LibQUAL+™ 2004 Summary Colleges or Universities – Faculty - American English (n = 11,755) Negative gap Positive gap

Institutional Norms for Perceived Means on 22 Core Questions Note: Thompson, B. LibQUAL+  Spring 2002 Selected Norms, (2002).

Access to Information by Status LibQUAL+ Spring 2003 Survey – UT Austin Faculty at Texas less approving of collection quality than students

Library as Place by Status LibQUAL+ Spring 2003 Survey – UT Austin Students rate “Library as Place” more disapprovingly than Faculty (size of gap)

Four Dimensions – Social Science & Psychology LibQUAL+ Spring 2003 Survey – UT Austin Our Psychology faculty do not rate their access to needed collections approvingly

Trends: Access to Information by Status Look for steadily improving trajectories

LibQUAL™ Interactive Institutional Statistics Master List Your peer list of institutions

The very act of administering LibQUAL+™ is beneficial

LibQUAL+ ™ Resources 1.LibQUAL+™ Website: Publications: Events and Training: LibQUAL+™ Bibliography: LibQUAL+™ Procedures Manual:

LibQUAL+ ™ Contact Information 1.Martha Kyrillidou  Senior Program for Office of Statistics and Measurement  2.Consuella Askew  LibQUAL+™ Program Specialist  3.Amy Hoseth  LibQUAL+™ Project Assistant  4.Jonathan D. Sousa  Technical Applications Development Manager 

This presentation available at:

Core Questions Summary LibQUAL+ Spring 2003 Survey – ARL Faculty

UT Austin vs. Peers LibQUAL+ Spring 2003 Survey – UT Austin