1. Chapter 8 Usability Studies and Usability Testing Human Performance Engineering Robert W. Bailey, Ph.D. Third Edition

2. Usability Studies= help to make good initial decisions and may be collected any time in the design process. Usability Tests = used to evaluate design decisions after they are implemented.

3. The earlier in the system development process that data is collected, the more valuable the information will be. Cost justifying data collection decisions Using someone else’s data Sampling 1.Random Sampling 2.Subjective Sampling

4. Usability Studies -Observations -Questionnaires -Interviews -Face to Face -Telephone

5. Usability Testing: Should evaluate human performance and user preference.

6. Usability Engineering = combining prototyping and usability testing Set acceptable usability levels (goals) Conduct usability testing Make required changes Repeat until goals are met

7. Study: stimulus –response compatibility  Group 1= pressed a left key for a left stimulus and a right key for a right stimulus (direct mapping)  Group 2= pressed a left key for a right stimulus and a right key for a left stimulus (indirect mapping) After 2400 trials, group 1 still showed an advantage in both speed and accuracy over group 2.

8. Measuring Usability What is the relationship between Performance and Preference? 71% perform well and like a system or perform poorly and dislike a system. 29% perform well and dislike a system or perform poorly and like a system. (Nielsen & Levy, 1994) *** Preferences of system designers do not always lead to the best performance-related design decisions.

9. Table 8-1 Preferences and Performance Scores for Alternative User Interfaces DesignerAverage Preferences (%)per list (seconds) ______________________________________________________________________ Typed entry5 percent 9.9 One-level menu18 percent 11.1 Two-level menus40 percent 12.2 Multiple menus37 percent 14.7 ______________________________________________________________________ A follow-up study reported similar results (Bailey, 1993). A study was conducted to determine the best screen-based control for a new application. Forty-six computer professionals practiced using four different methods and then selected the one that they thought would elicit the fastest performance. The same people then participated in a performance test. The results are shown in Table 8-2. Again, a relatively large percentage of the designers preferences did not elicit the fastest performance.

10. Table 8-2 Preferences and Performance Scores for Alternative Widgets Percent PreferringAverage Time to Each WidgetPerform (seconds) ____________________________________________________ ___________ Open selection lists76 percent58 Drop-down selection lists13 percent90 Radio buttons 7 percent53 Typed entry 4 percent72 ____________________________________________________ ___________

11. Givon & Goldman, 1987: Designer: Is window A different from window B? User: I can’t tell. Designer: Which do you prefer? User: Window B

12. Types of Usability Tests: Formal = using theories to make predictions. Inspection= walkthroughs, questionnaires Performance = well-controlled simulations of people performing activities using system prototypes. Operational = methods used after a product is delivered.

13. Predicting Human Performance * Is the Principle purpose of usability testing. * Fidelity= variable that largely determines the predictive power of a usability test. *The fidelity of test conditions is directly related to cost and inversely related to the ease with which meaningful changes can be made to the system.

14. System Performance Accuracy Objective Cost-effective performance User acceptance