1. 1 The Benefits of Using Eye Tracking in Usability Testing Jennifer C. Romano Usability Laboratory Statistical Research Division U.S. Census Bureau

2. 2 Overview What is usability? Introduction to eye tracking Using eye tracking in usability testing

3. 3 What is usability? Defined as “the effectiveness, efficiency, and satisfaction with which a specified set of users can achieve a specified set of tasks in a particular environment (ISO/TR 16982:2002)”

4. 4 What is usability testing? Usability testing is concerned with assessing how usable a product is. –Can users interact with the product in an effective, efficient, and satisfying way? –Evaluation at any stage of development –Iterative testing

5. 5 Why do usability testing? Design often does not take the users’ mental models into account. Users have their own cognitive limitations and capabilities. Test the product on the end user.

6. 6 Usability Evaluation Benefits Satisfied users Improved data quality Less measurement error Reduced training time

7. 7 Usability Testing Measures Effectiveness –Accuracy Efficiency –Time to complete a task/survey Satisfaction –Self-rated satisfaction

8. 8 Introduction to Eye Tracking Photo from: www.smith.umd.edu/behaviorlab/eyetracker.aspxwww.smith.umd.edu/behaviorlab/eyetracker.aspx

9. 9 Introduction to Eye Tracking Captures where people look as they navigate through a Web site Shows an individual’s pathway Accumulates data and shows common pathways

10. 10 Eye-Tracking Measures Fixations –Time to first look at an area of interest (AOI) –How often people look at AOI Hot spots –Areas that get most attention Gaze plots –Pattern of search

11. 11 Benefits of Eye Tracking Identify areas that attract users’ attention or areas that are difficult to understand Identify how long it takes people to look at something Identify where people look –Where people look –Where people never look

12. 12 Questions Eye Tracking Can Answer Why does it take users a long time to complete the task/survey? Why do users have low accuracy? Why aren’t the users satisfied? Are people looking at…something? Terminology or…something else?

13. 13 Usability Study

14. 14 Task 1 Task: You want to use data from two different data sources, but are unsure if they are comparable. Find out if this information exists on the site. Solution: Get Data or User Guide

15. 15 Task 1: Usability Findings –Users seemed unclear on where to find the information. Five participants (56%) selected Get Data from the center of the screen, and three (33%) selected User Guide from the top navigation. –Of the five participants who selected Get Data, two successfully completed the task. Of the three participants who selected User Guide, only one successfully completed the task. –Accuracy: 33% –Efficiency (correct only): 3m12s

16. 16

17. 17 Task 1: Hot Spot Findings

18. 18 Task 1: Hot Spot Findings

19. 19 Task 1: Hot Spot Findings

20. 20 Task 2 Task: You are unsure of the difference between ‘Places’ and ‘Metropolitan Areas.’ Where would you find this information on this site? Solution: Definition  Geography

21. 21 Task 2: Usability Findings Most participants initially scrolled through the definitions list, expecting to find the meanings of ‘Metropolitan Areas’ and ‘Places’ (which were not in the listing but rather located on the Geography page). Two participants never used the Geography link and instead navigated to the Site Map to find the target information.

22. 22 Task 2: Usability Findings Users said that they expected the format to be the same as the format of the Definitions page. Accuracy: 78% Efficiency (correct only): 4m32s

23. 23

24. 24 Task 2: Gaze Plot Findings

25. 25 Task 2: Gaze Plot Findings

26. 26 Fixation Findings Areas of Interest (AOIs) ParticipantTop navLeft navCenterRight nav 1 49s5s3s7s 2 2s3s2s4s 3 45s34s6s20s 4 7m48s, task 327s2s27s 5 2m2s, task 23s2s1m20s 6 18s, task 215s, task 22s16s, task 2 7 2m2s20s2s2m23s 8 49s1m50s8s9s Mean1m43s24s3s34s

27. 27 Conclusion Obtain measures that are not obtainable from traditional usability measures. –What people (do not) look at –How often they look there –Time into the study to look there –Quantitative data supports usability findings

28. 28 Conclusion Without obtaining this type of data, it is impossible to determine where people look on the screen and how often they look there. It’s simply not possible to get data on where people looked, how quickly they looked, and how often they looked using only conventional experimental methods.

29. 29 Conclusion Future usability testing should aim to include eye tracking as a method of collecting data.

30. 30 Thank you! Jennifer C. Romano Usability Laboratory Statistical Research Division U.S. Census Bureau (301)763-3577 Email: Jennifer.romano@census.gov LinkedIn: http://www.linkedin.com/in/jenniferromano http://www.linkedin.com/in/jenniferromano