LibQUAL+™ GUGM Reference Section May 19, 2005 Presentation by Brian Mathews, Georgia Tech Caroline Killens, UGA How to Listen to your Customers Using LibQUAL+™

Definition of Assessment  In an academic environment assessment can be defined as “an ongoing process aimed at understanding and improving student learning. It involves:  making our expectations explicit and public;  setting appropriate criteria and high standards for learning quality;  systematically gathering, analyzing, and interpreting evidence to determine how well performance matches those expectations and standards; and  using the resulting information to document, explain, and improve performance. When it is embedded effectively within larger institutional systems, assessment can help us focus our collective attention, examine our assumptions, and create a shared academic culture dedicated to assuring and improving the quality of higher education (Thomas A. Angelo, AAHE Bulletin, November 1995, p.7).

The Importance of Assessment  To evaluate your service.  To understand what patrons “really” want.  To guide the decision-making process.  To support administrative initiatives.

What is LibQUAL?  A survey sponsored by ARL, as part of the New Measures initiative, to augment the collection count and fiscal input measures of ARL that comprise the ARL Index and ARL Statistics. The survey is comprised of “22 questions + a box”  Is an assessment tool that lets customers judge quality  Produces outcome measures on service quality and satisfaction of users. Is a way of “listening” to users  Provides institutional data and reports that enables you to assess whether your library services are meeting user expectations  Provides libraries with comparable assessment information from peer institutions

Why Use LibQUAL+™?  Great value with minimal effort.  To gain an objective understanding of your patron’s unique needs and perceptions.  Provides comparative data with other libraries.  Can be utilized with campus library reviews, SACS reviews, accreditation reports, etc.

How To Prepare for LibQUAL+™  Complete LibQUAL+™ questionnaire on-line to activate your account. Determine a good source for your campus statistics!  Determine the best dates to conduct your survey.  Determine if you need permission from your campus IR Board.  Decide who to include in the survey and how it will be conducted. Everyone on campus, random selection, etc.; If using a random sample, how will you generate your sample? Circulation files, Human Resources files, Registrar’ files, etc. , website, mail paper copy  Determine how much advertisement you want to conduct prior to the survey Campus newspapers, website, posters, alert departments, etc.  Decide if you will use incentives. iPods, no fines, money, sponsored gifts, etc.  Prepare letter to be sent to participants. (sample provided by LibQUAL+™)

What Happens While the Survey Takes Place  You can track the number of surveys completed for your institution online in real time.  You can begin reviewing comments immediately.  Send out reminders if it looks like things are slowing down.  Delete undeliverable addresses from your files so your reminders will go only to active addresses.  Have someone dedicated to answer questions by phone or –  RELAX! and let the survey work for you.

What Do You Get From LibQUAL+™?  LibQUAL+™ report within a month of your survey’s closing  SPSS and Excel files with all of the raw data  Interactive files at the LibQUAL website  List of the comments made on your survey  Access to reports from all other participating libraries

How The Survey Works Participants are asked to score each question in 3 sub-categories, based on a scale of 1-9: The sub-categories for each question:  Minimum = “what they need” minimum acceptable level of service required by users users  Perceived = “what they have” - the level of service users think/believe the library is providing (perceived)  Desired = “what they want” - the level of service users want to have (desired) When it comes to….. My minimum service level My perceived service level My desired service low high low high low high When it comes to….. My minimum service level My perceived service level My desired service low high low high low high 1)Employees who instill confidence users )Employees who instill confidence users

Zone of Tolerance

GAPS The scores from the 3 sub-categories can be utilized to show the following gaps: 1. ADEQUACY GAP : how well the library is meeting the users’ “minimum” requirements/needs. Formula: Perceived score minus the minimum score 2. SUPERIORITY GAP : how far the library is from meeting or exceeding the users’ desired level of service Formula: Perceived score minus the desired score

One way of viewing GAPS (shows minimum perceived & desired scores on 1 graph)

Another view of Gaps UGA Adequacy Gaps by Question

Comparing gaps over time GT Graduates Adequacy 2003 & 2004

RADAR GRAPHS Radar graphs are a good way to show results of all questions in one chart. This is a good visual way to compare yourself overall with other libraries. RED – Scores are below your users minimum expectations BLUE – shows the gap between your minimum and perceived Yellow – shows the gap between your perceived and desired GREEN – shows scores above the desired expectations

LibQUAL+™ Comparisons: ARL Colleges & Universities and UGA Core Questions Summary ARL UGA

Benchmarking  An external focus on internal activities, functions, or operations in order to achieve continuous improvement.  LibQUAL+ enables you to compare your library with peer institutions.  LibQUAL+ enables you to identify areas of excellence and areas for potential improvement.

GT Benchmarking Examples Top Engineering Schools Undergraduates, Perceptions Printed Materials Top Engineering Schools Undergraduates, Adequacy Knowledge to Answer User Questions

The survey offers a “comments box” soliciting open- ended user views that provide a more detailed picture of how users view our resources and services 466 UGA participants utilized this box to give 799 specific comments. 231 GT participants utilized this box to give 400 specific comments. COMMENTS

Comments on Online Tools : Summary  Many comments expressed confusion or dissatisfaction with doing research online, but were not specific enough to apply to one tool such as GIL, GALILEO, EJL, etc.  Areas of difficulty/frustration expressed most frequently by survey participants: -the difficulty of finding journals online -too many disparate search engines available -online tools not user-friendly -searches that result in few or no hits.  Dissatisfaction with online tools seems to decrease with experience in searching and/or doing research, though all user groups had a high proportion of constructive comments.

Comments on Online Tools:  “ Most frustrating problem for GALILEO, etc., is the amount of drilling down to reach needed databases.” – Graduate Student, Education  “…sometimes GALILEO can be very confusing, particularly with all of the different search engines … is there any way to put it all into ONE search?” – Undergraduate Student, Business  “The only major complaint I have is that searches can be very in depth and complicated without assistance from a library employee at least until you have a lot of practice.” – Undergraduate Student, Journalism  “GIL is a huge problem for me. Half the time books don’t show up on general or advanced searches even though there are probably over 40 books available.” – Undergraduate Student, Social Science  “The online catalog search systems seem as though they were written by undergraduate computer science classes. You need a professionally written search system that is designed the way scientists and engineers use information. What good is a library if one can't find the information?” – Faculty, Science

Memorable Comments: Memorable Comments:  “The Student Learning Center is fantabulous. It’s better than a cell phone – how did I live without it?!?!?! – Undergraduate, Physical Sciences  “[Suggestion:] A good cafeteria – empty stomach is also a devil’s workshop.” – Graduate Student, Ag. & Env. Sciences  “Please have a drive-up renewal/fine window. That would be fabulous!” - Graduate Student, Humanities   “Allow us to check out laptops like books.” – Undergraduate, Engineering   “I think the employees of the library are doing a great job. They have always been very courteous and always able to answer my questions.” Undergraduate, Engineering   “I feel that the employees could be more courteous and willing to help.” Undergraduate, Engineering

What did GT and UGA hear from their users?  We are not even meeting the minimum expectations for faculty and/or graduate students in many of the information control questions which includes electronic access, electronic information & print materials. (Is this because the needs are very specialized and/or diverse? our collections are inadequate? or because users are unable to identify and locate what we do have?)  All users groups, especially undergraduates, have difficulties using our online tools. There is particular confusion for undergraduates over the terminology of GIL, GALILEO & EJL  Users find it difficult to locate materials on the shelves, especially those that GIL indicates are “not checked out.” There is also some confusion on the arrangement of materials in the Library.

DO IT AGAIN? Yes or No? Why?

LibQUAL+ Assessment Websites  Here you will find information about all aspects of the survey  This is the assessment site at UGA  This is the LibQUAL+ results and analysis site at GT Contacts: Brian Mathews : Caroline Killens :