1. Service Quality Assessment in a Digital Library Environment
Service Quality Assessment in a Digital Library Environment
4th International JISC/CNI Conference
Edinburgh, Scotland
June 27, 2002
Joseph Boykin, Fred Heath, Duane Webster

2. Overview of Discussions
New Models for Understanding and Describing Library Success
ARL’s New Measures Initiative
LibQUAL+ Project Development
Experience with LibQUAL+
Conclusions and Next Steps

3. The Association of Research Libraries
Mission: Shaping and influencing forces affecting the
future of research libraries in the process of
scholarly communication.
Members: 123 major research libraries in North America.
Ratios: 4% of the higher education institutions
providing 40% of the information resources.
Users: 3 million students and faculty served.
Expenditures: $2.35 billion annually, $727 million for
acquisitions of which 9% is invested in access
to electronic resources.
ASSOCIATION OF RESEARCH LIBRARIES

4. The Problem of Assessment in Research Libraries
ARL Membership Criteria Index variables emphasize inputs, primarily expenditures
To rise in the ARL Index it is only necessary to spend more
No demonstrable relationship between expenditures and service quality
The lack of metrics describing performance

6. ARL New Measures Initiative
Collaboration among member leaders with strong interest in this area
Specific projects developed with different models for exploration
Intent to make resulting tools and methodologies available to full membership and wider community

7. ARL New Measures Projects
Demonstration project for service effectiveness measures (LibQUAL+)
Project to define usage measures for electronic information resources (includes institutional outcomes)
Investigation of role libraries play in support of the research process
Investigation of role libraries can play in campus learning outcomes activities
Identification of cost-drivers and development of cost-benefit studies

8. Assessment
“The difficulty lies in trying to find a single model or set of simple indicators that can be used by different institutions, and that will compare something across large groups that is by definition only locally applicable –i.e., how well a library meets the needs of its institution. Librarians have either made do with oversimplified national data or have undertaken customized local evaluations of effectiveness, but there has not been devised an effective way to link the two” Sarah Pritchard, Library Trends, 1996

9. LibQUAL+™ Description
LibQUAL+TM is a research and development project undertaken to define and measure library service quality across institutions and to create useful quality-assessment tools for local planning.

10. The Purpose of Our Research
To fill a knowledge void in modeling the dimensions of library service quality from a user perspective
Based upon the model, to develop a web-delivered, effective total market survey instrument equivalent for service quality assessment in academic libraries
Using the derived instrument to recommend a process for an ongoing program of comparative outcome measurement for academic libraries

11. LibQUAL+™ Project Goals
Development of web-based tools for assessing library service quality
Development of mechanisms and protocols for evaluating libraries
Identification of best practices in providing library service
Establishment of a library service quality assessment program at ARL

12. Project Resources
LibQUAL+TM is an ARL/Texas A&M University joint effort. The project is supported in part by a 3-year grant from the U.S. Department of Education’s Fund for the Improvement of Post-Secondary Education (FIPSE).

13. LibQUAL+TM Team ARL Duane Webster Martha Kyrillidou Kaylyn Hipps
Julia Blixrud
Jonathan Sousa
Consuella Waller
TAMU
Fred Heath
Colleen Cook
Bruce Thompson
Yvonna Lincoln
Trey Thompson
Julie Guidry

14. LibQUAL+TM Project History
Sept. 1999
Jan2000
July 2000
Sept. 2000
Oct. 2000
Jan. 2001
June 2001
9/99 - ARL launches “New Measures Initiative” which includes the study of service effectiveness known as SERVQUAL spearheaded by Texas A&M University.
1/00 - Initial 12 institutions begin the SERVQUAL study led by the Texas A&M team.
7/00 - LibQUAL+ as a distinct library-based assessment tool is presented.
9/00 - ARL and Texas A&M awarded a FIPSE grant to fund further development of the LibQUAL+ project.
10/00 - The ARL symposium, “New Culture of Assessment in Academic Libraries Measuring Service Quality” attracts a group of 170 people.
1/01 - Representatives from 43 research and university libraries participating in the Spring 2001
Implementation meet in Washington, DC during ALA midwinter.
6/01 - National Science Foundation awards grant to ARL and Texas A&M to adapt LibQUAL+
for NSDL

15. LibQUAL+™ Contribution
Grounded questions yield data of sufficient granularity to be of local use
Normative data across cohort group at first and higher order levels
Surfaces “Best Practices”
Web approach makes little demand of local resources while compiling robust dataset

16. Relationships: perceptions, service quality and satisfaction
….only customers judge quality;
all other judgments are essentially
irrelevant”
Zeithaml, Parasuraman, Berry. (1999).
Delivering quality service. NY: The Free Press.

17. LibQUAL+TM Participants
Year 3
Year 2
170 Participants
43 Participants
Year 1
12 Participants
Spring 2000
Spring 2001
Spring 2002
For More Information about Participants:
Visit the LibQUAL+ web site.

18. SURVEY
INSTRUMENT

19. Dimensions of Library
Service Quality

20. LibQUAL+ Core Questions Y1
_____________________________________________________________________________
Factor_ ______ _
No I II III IV Item Core
Willingness to help users
Giving users individual attention
Employees deal with users caring fashion
Employees who are consistently courteous
Employees have knowledge answer questions
Employees understand needs of users
Readiness to respond to users' questions
Employees who instill confidence in users
Dependability handling service problems
A haven for quiet and solitude
A meditative place
A contemplative environment
Space that facilitates quiet study
A place for reflection and creativity
* website enabling me locate info on my own
* elec resources accessible home or office
* access tools allow me find on my own
Modern equip me easily access info I need
* info easily accessible for independent use
Convenient access to library collections
Comprehensive print collections
Complete runs of journal titles
Interdisciplinary library needs addressed
Timely document delivery/interlibrary loan
Convenient business hours
________________________________________________________________

21. Affect of Service Emerged as the dominant factor early in our work
Absorbed several of the original SERVQUAL questions measuring Responsiveness, Assurance and Empathy
In the current analysis also includes Reliability
All in all: the Human Dimension of Service Quality

22. Library as Place
Transcends the SERVQUAL dimension of Tangibles to include the idea of the library as the campus center of intellectual activity
As long as physical facilities are adequate, library as place may not be an issue

23. Personal Control How users want to interact with the modern library
Personal control of the information universe in general and web navigation in particular

24. Access to Information
Ubiquity of access: information delivered in the format, location and time of choice
Comprehensive collections

25. Project Deliverables Print and web-based results include:
Aggregate Summaries
Demographics by Library
Item Summaries
Dimension Summaries
A copy of the survey instrument
Dimensions measured for survey implementation

34. Two Interpretation Frameworks
Score Norms
Zone of Tolerance

35. Score Norms
Norm Conversion Tables facilitate the interpretation of observed scores using norms created for a large and representative sample.
LibQUAL+TM norms have been created at both the individual and institutional level

36. C

37. C
Zone of Tolerance
The area between minimally acceptable and desired service quality ratings
Perception ratings ideally fall within the Zone of Tolerance

38. ZONE OF TOLERANCE: DIMENSIONS

40. LibQUAL+™ Fundamental Contributions to the Measurement of Effective Delivery of Library Services
Determine the degree to which information derived from local data can be generalized, providing much needed “best practices” information
Demonstrate the efficacy of large-scale administration of user-centered assessment transparently across the web
Shift the focus of assessment from mechanical expenditure-driven metrics to user-centered measures of quality
Re-ground gap theory for the library sector, especially academic libraries

41. LibQual and 2001 Results

46. NSF Grant Assess service quality in digital libraries 3 year period
Adopt LibQUAL+ instrument for use in the Science, Math, Engineering and Technology Education Digital Library community (NSDL)

47. NDSL LibQUAL+ Goals
Define dimensions of digital library service quality from the users’ perspectives
Develop tool for measuring user perceptions and expectations of digital library service quality across NSDL digital library contexts
Identify digital library best practices that permit generalizations across operations and development platforms

48. NDSL LibQUAL+ Activities
qualitative interviews to contribute to identifying dimensions of digital library service quality
Test and refine dimensions of digital library service quality and self-sufficiency through development of total market survey
Implement survey across variety of organizational and digital library implementations

49. LibQUAL+TM Related Documents
LibQUAL+TM Web Site
LibQUAL+TM Bibliography
Survey Participants Procedures Manual

50. LibQUAL+ Related Documents
LibQUAL+ Web Site
LibQUAL+ Bibliography
woof