Assessing Library Web Portals: Usability and Beyond Yu-Hui Chen University at Albany, State University of New York ENY/ACRL 2012 Conference Mohawk Valley Community College, Utica, New York May 21, 2012

Web Site Evaluation Methods Evaluation with user participation ◦ Think aloud ◦ Card sorting ◦ Prototyping (paper/online) ◦ Eye tracking ◦ Focus group discussions ◦ Field study ◦ Log analysis ◦ Web Survey Evaluation without user participation ◦ Cognitive walkthrough ◦ Heuristic evaluation

Think Aloud Users are asked to complete specific tasks As users are attempting to complete each task, they verbally report their thoughts and feelings of their actions Observers watch, listen, and takes notes

Card Sorting Open card sorting ◦ Give users labels representing the content of the Web site ◦ Users review these labels and then group them into categories. ◦ Users assign category names to these groups Closed card sorting ◦ Provide category names for users ◦ Users sort the labels into categories

Prototyping (Paper/Online) Provide users with descriptions and purposes of an intended Web site Have users brainstorm the design Have users draw the design Test the design

Eye Tracking Setting up a lab Training users in using the equipment Giving users tasks Review the reports

Cognitive Walkthrough The evaluators design specific task scenarios The user’s goals and purpose for each task are defined and tasks are broken down into relatively small pieces The evaluators role play the part of the user working with the site, noting problems, path, and barriers, essentially reviewing the ease of learning the site

Heuristic Evaluation Have a small group of evaluators (2-5) review the site using usability heuristics (e.g., Nielsen), standards (e.g., ISO), or guidelines (e.g., US Dept. of Health and Human Resources) ◦ Inspect the task flow ◦ Inspect details of individual elements Evaluators review the site independently Reconvene and discuss findings

Information Systems Success Model DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), (p. 24)

Measures of Information Quality Accuracy Currency Sufficiency Reliability Relevance Format options

Measures of System Quality Accessibility Ease of use Flexibility Response time Reliability

Measures of Service Quality Assurance Empathy Responsiveness Reliability

Measures of Use Frequency of use Extent of use Motivation to use

Measures of User Satisfaction System quality satisfaction Information quality satisfaction Service quality satisfaction Overall satisfaction

Measures of Net Benefits User productivity User performance

Assessment Approach Quantitative Qualitative

Bibliography DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), International Standards Organization (1994). Ergonomic requirements for office work with visual display terminals. Part 11: Guidance on usability (ISO DIS ). London: International Standards Organization. Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press. Popp, M. P. (2001). Testing library Web sites: ARL libraries weigh in. Proceedings of the ACRL Tenth National Conference, United States Department of Health and Human Services. (2006). Research-based Web design & usability guidelines. Washington, DC : U.S. Government Printing Office. Other entertaining resources: Chen, Y., Germain, C. A., & Yang, H. (2009). An exploration into the practices of library Web usability in ARL academic libraries. Journal of the American Society for Information Science and Technology, 60(5),