1. Assessing Library Web Portals: Usability and Beyond Yu-Hui Chen University at Albany, State University of New York ENY/ACRL 2012 Conference Mohawk Valley Community College, Utica, New York May 21, 2012

2. Web Site Evaluation Methods Evaluation with user participation ◦ Think aloud ◦ Card sorting ◦ Prototyping (paper/online) ◦ Eye tracking ◦ Focus group discussions ◦ Field study ◦ Log analysis ◦ Web Survey Evaluation without user participation ◦ Cognitive walkthrough ◦ Heuristic evaluation

3. Think Aloud Users are asked to complete specific tasks As users are attempting to complete each task, they verbally report their thoughts and feelings of their actions Observers watch, listen, and takes notes

4. Card Sorting Open card sorting ◦ Give users labels representing the content of the Web site ◦ Users review these labels and then group them into categories. ◦ Users assign category names to these groups Closed card sorting ◦ Provide category names for users ◦ Users sort the labels into categories

5. Prototyping (Paper/Online) Provide users with descriptions and purposes of an intended Web site Have users brainstorm the design Have users draw the design Test the design

6. Eye Tracking Setting up a lab Training users in using the equipment Giving users tasks Review the reports

7. Cognitive Walkthrough The evaluators design specific task scenarios The user’s goals and purpose for each task are defined and tasks are broken down into relatively small pieces The evaluators role play the part of the user working with the site, noting problems, path, and barriers, essentially reviewing the ease of learning the site

8. Heuristic Evaluation Have a small group of evaluators (2-5) review the site using usability heuristics (e.g., Nielsen), standards (e.g., ISO), or guidelines (e.g., US Dept. of Health and Human Resources) ◦ Inspect the task flow ◦ Inspect details of individual elements Evaluators review the site independently Reconvene and discuss findings

9. Information Systems Success Model DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. (p. 24)

10. Measures of Information Quality Accuracy Currency Sufficiency Reliability Relevance Format options

11. Measures of System Quality Accessibility Ease of use Flexibility Response time Reliability

12. Measures of Service Quality Assurance Empathy Responsiveness Reliability

13. Measures of Use Frequency of use Extent of use Motivation to use

14. Measures of User Satisfaction System quality satisfaction Information quality satisfaction Service quality satisfaction Overall satisfaction

15. Measures of Net Benefits User productivity User performance

16. Assessment Approach Quantitative Qualitative

17. Bibliography DeLone, W. H., & McLean, E. R. (2003). The DeLone and McLean model of information systems success: A ten-year update. Journal of Management Information Systems, 19(4), 9-30. International Standards Organization (1994). Ergonomic requirements for office work with visual display terminals. Part 11: Guidance on usability (ISO DIS 9241-11). London: International Standards Organization. Nielsen, J. (1993). Usability engineering. Boston, MA: Academic Press. Popp, M. P. (2001). Testing library Web sites: ARL libraries weigh in. Proceedings of the ACRL Tenth National Conference, 277-281. United States Department of Health and Human Services. (2006). Research-based Web design & usability guidelines. Washington, DC : U.S. Government Printing Office. Other entertaining resources: Chen, Y., Germain, C. A., & Yang, H. (2009). An exploration into the practices of library Web usability in ARL academic libraries. Journal of the American Society for Information Science and Technology, 60(5), 953-968.