1. 540-310 Human Factors in Information Seeking and Use
Wooseob Jeong

2. Notice Midterm Exam on April 7th. Usability Test I & II = 20%
Question Pool is available at
Only 10%
Usability Test I & II = 20%
Read Nielson’s book.

3. Usability Test Assignment
Part II: Usability Test & Report
Identify any usability problems by testing at least three subjects, who should be available easily like your room mates or family members.
Before testing, you should carefully develop appropriate tasks. The problems should be addressed in terms of usability perspective, especially time, effort, and error.
Based on your findings, you should suggest any modification for the improvement of usability.
words – tables & figures - 15% 
Due on 4/17

4. Goals of Usability Testing (1)
Of course, to ensure the creation of products that:
Are easy to learn and to use
Are satisfying to use
Provide utility and functionality that are highly valued by the target population
Specifically, …

5. Goals of Usability Testing (2)
Creating a historical record of usability benchmarks for future releases.
Minimizing the cost of service and hotline calls.
Increasing sales and the probability of repeat sales.
Acquiring a competitive edge since usability has become a market separator for products.
Minimizing risk.

6. Limitations of Usability Testing
Testing is always an artificial situation.
Test results do not prove that a product works.
Participants are rarely full representative of the target population.
Testing is not always the best technique to use.

7. Four Types of Usability Tests
Exploratory Test
Preliminary stage; skeleton
Assessment Test
Middle stage; most typical; body
Validation Test
Late stage; verification, just before release
Comparison Test
Different interfaces in the same product; with competitors’ products

8. Task Component and Description
Load paper into the copier.
Machine State
Copier with four labels attached and an empty cassette tray.
Successful Completion Criteria
Test subject loads paper into cassette after first fanning the stack of paper.
Benchmark
Load correctly within one minute.

9. Task Development Exercise
Tasks should be simple and specified.
Required Machine State should be comprehensive to avoid unexpected problems.
Successful Completion Criteria should be clear and specified.
Benchmark should be estimated reasonably.
Pilot test is required!

10. Useful Links for Usability Testing
List of Usability Evaluation Methods and Techniques
Designing for Usability
Using Paper Prototypes
The Usability Methods Toolbox
Usability First
Usability Is Good Business
Usability Professionals' Association

11. Typical Test Routine Introduction Pre-questionnaire Main Test
Make participants comfortable
Pre-questionnaire
Background info like age, sex, experience
Main Test
Talk Aloud
Post-questionnaire
Confirmation of test results
Debriefing
Wrap-up conversation

12. Different Testing Roles
Test monitor/administrator
Data logger
Timers
Video recording operator
Product/technical experts
Additional testing roles
Test observers

13. Role of the test monitor
During the test, the monitor is responsible for all aspects of administration including greeting the participant, collecting data, assisting and probing, and debriefing the participant. After the test, he or she needs to collate the day’s data collection, meet with and debrief other team members, and ensure that the testing is tracking with the test objectives.
Demo video!

14. Good test monitor Grounding in the basics of usability engineering
Quick learner
Instant rapport with participants
Excellent memory
Good listener
Comfortable with ambiguity
Flexibility
Long attention span
Empathic “people person”
“Big picture” thinker
Good communicator
Good organizer and coordinator

15. Typical Test Monitor Problems
Leading rather than enabling
Too involved with the act of data collection
Acting too knowledgeable
Too rigid with the test plan
Not relating well to each participant
Jumping to conclusions

16. Handling frustration Give up vs. Keep trying?
Getting valuable information from frustration by letting them keep trying.
How? - Empathize with the participant (smile!)
“I can sense you’re getting frustrated, but this is a particularly crucial part of the software/ documentation, would you please try a little bit longer?”
“I see you’re having a difficult time with this. This isn’t the first time that I’ve seen someone experience some difficulty here. Would you please continue on for five more minutes?”

17. Improve test monitoring skill
Learn the basic principles of human factor ergonomics
Learn from watching others
Watch yourself on tape
Work with a mentor
Practice monitoring
Learn to meditate
Practice “bare attention”