Usability Evaluation

Objectives for today Go over upcoming deliverables Learn about usability testing (testing with users) BTW, we haven’t had a quiz for a while.

Deliverables for the next week User test plans (based on this week. Due Monday) –Individual drafts … combined final plan Coded Prototype (due on Wednesday) –Plan to demonstrate the application –Be prepared to discuss whether and how you addressed usability problems from your paper prototype

Evaluation with Users Big investment – big potential –Remember, you can identify and solve lots of problems using cognitive walkthroughs and heuristic evaluation before you make the investment Many issues/techniques But they have a common foundation

Testing with Users Unlike some of the “userless” tests there aren’t a lot of fancy protocols we will learn. Most of the challenges around preparing for a user-based test involve PREPARING for a user-based test.

A Test Plan Checklist, 1 Goal of the test? Specific questions you want to answer? Who will be the experimenter? Who are the users going to be? How many users are needed? What kind of instruction will the users be given? What tasks will you ask the users to perform? What criteria will be used to determine the end of each task?

A Test Plan Checklist, 2 What aids will be made available to users? To what extent will the experimenter be allowed to help the users? What data is going to be collected and how will it be analyzed? What is the criterion for judging the interface a success?

A Test Plan Checklist, 3 Where and when will the evaluation be done? How long will the evaluation take? What computer support? What software? Initial state of the system? Are there any system/network load requirements?

The Role of the Experimenter Having the right experimenter makes a difference –Selecting an appropriate methodology and one the experimenter is familiar with significantly influences the quality of results Knowledge of system implementation can come in handy Participating in a usability study can have a profound impact on a designer –Even very simple, informal studies

The Role of the Experimenter Ensures that room, computer, etc are all ready During testing: –Should not interfere! –If a user is bogged down can give a small hint –If the user is hopelessly off track, can fake an equipment problem

Ethical treatment of subjects Your responsibility to protect subjects –distress, embarrassment –remind them that you are not testing them Informed, voluntary consent –understand that they can quit at any time –explain test in lay terms Privacy: anonymity, use of image/voice

Which Users? As close to real users as possible –if real users are scarce, try surrogates

How Many Users? Huge individual differences between users! Up to a factor of 10 A single point determines an infinite number of lines Some data is better than none

Which Tasks? Keep close to the real tasks –may need to shorten some for time reasons –may need to provide users with background information

When in the Process? Remember: early is better Formative vs. summative evaluation –During design  design modifications –After design  evaluation of “finished” product, comparison to baseline, rigorous statistics