1. Interactive Systems Technical Design
Lecture #5
Usability engineering
[Source: Jakob Nielsen, Usability Engineering, Morgan Kaufmann, 1993]
ISTD 2003

2. Practical acceptability
What is Usability?
System acceptability
Social acceptability
Practical acceptability
Reliability
Compatibility
Usefulness
Cost …
Usability
Utility
Learnability
Satisfaction
Efficiency
Memorability
Errors
ISTD 2003

3. Usability Attributes
Learnability: The system should be easy to learn so that the user can rapidly start getting some work done with the system.
Efficiency: The system should be efficient to use, so that once the user has learned the system, a high level of productivity is possible.
Memorability: The system should be easy to remember, so that the casual user is able to return to the system after some period of not having used it, without having to learn everything all over again.
Errors: The system should have a low error rate, so that users make few errors during the use of the system, and so that if they do make errors they can easily recover from them. Further, catastrophic errors must not occur.
Satisfaction: The system should be pleasant to use, so that users are subjectively satisfied when using it; they like it.
ISTD 2003

4. Usability Engineering Lifecycle
Know the user
Individual user characteristics
The user’s current and desired tasks
Functional analysis
The evolution of the user and the job
Competitive analysis
Setting usability goals
Financial impact analysis
Parallel design
Participatory design
Coordinated design of the total interface
Apply guidelines and heuristic analysis
Prototyping
Empirical testing
Iterative design
Capture design rationale
Collect feedback from field use
ISTD 2003

5. Usability Heuristics Simple and natural dialogue
Simplify, less is more
Match the users’ task in as natural way as possible
Speak the users’ language
Employ users’ language and terminolory
Mappings and metaphors
Minimize user memory load
Consistency
Feedback
Response time (0.1 -> 1.0 -> 10 seconds)
System failure
ISTD 2003

6. Usability Heuristics (cont.)
Clearly marked exits
Do not trap the user
Shortcuts
Accelerators for experienced users
Good error messages
Clear language, precise, constructive, polite
Prevent errors
Confirmations (”do you really want to delete all files?”)
Avoid modes
Help and documentation
Most users do not read manuals
Search -> Understand -> Apply
ISTD 2003

7. Case Study: Assessing Usability of Pointing Devices
button OK OK
W1=4
W2=3
D2=3.5
D1=5
cursor A) B)
In which case it is more difficult to point the button with the cursor?
ISTD 2003

8. Fitts’ Law Index of difficulty = log2(2D/W)
D = distance to the object
W = width of the object
Time to point = C1 + C2 (Index of difficulty)
C1, C2 = device specific constants
A) Index of difficulty = log2(2*5/4) = log2(2.5)
B) Index of difficulty = log2(2*3.5/3) = log2(2.33)
ISTD 2003

9. Usability Testing User testing with real users
Fundamental usability assessment method
Provides direct information about …
How the system is used
What are the exact problems
ISTD 2003

10. Usability Testing (cont.)
Methodological pitfalls
Reliability: would the test produce the same result if repeated?
Caused by huge individual differences between test users
Validity: does the result actually reflect the usability issues that one wants to test?
Caused by using wrong users or tasks
ISTD 2003

11. Usability Testing (cont.)
Test plan
Goals, schedule, budget, pilot tests, …
Usability_Problems_Found(i) = N(1 – (1- P)i)
i = number of test users
N = total number of usability problems
P = probability of finding any single usability problem with any single
test user
ISTD 2003

12. Usability Testing (cont.)
Getting test users
Novice vs. expert users
Between-subjects vs. within-subjects testing
Choosing experimenters
Previous experience on usability testing is valuable
Ethical aspects of tests with human subjects
”System is being tested, not the user”
Respect the test user and his/her privacy
Choosing test tasks
Representative tasks
Conducting the test
Preparation
Introduction
The test itself
Debriefing
ISTD 2003

13. Other Usability Assessment Methods
Heuristic evaluation
Performance measurement
Thinking aloud
Observation
Questionnaires
Interviews
Focus groups
Logging actual use
User feedback
ISTD 2003

14. Heuristic Evaluation (”Discount Usability Engineering”)
Evaluation of a system using a set of heuristic rules or guidelines
Lifecycle stage: Early design, ”inner cycle” of iterative design
Users needed: None
Main advantage: Finds individual usability problems. Can address expert user issues.
Main disadvantage: Does not involve real users, so does not find ”surprises” related to their needs.
ISTD 2003

15. How to Conduct Heuristic Evaluation
Involve multiple evaluators (3-5)
Each evaluator independently analyzes the system wrt. a list of recognized usability principles
Aggregate evaluators’ feedback
ISTD 2003

16. Ten Usability Heuristics by Nielsen
Visibility of system status
The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.
Match between system and the real world
The system should speak the users' language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.
User control and freedom
Users often choose system functions by mistake and will need a clearly marked "emergency exit" to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.
Consistency and standards
Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.
Error prevention
Even better than good error messages is a careful design which prevents a problem from occurring in the first place.
ISTD 2003

17. Ten Usability Heuristics by Nielsen (cont.)
Recognition rather than recall
Make objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.
Flexibility and efficiency of use
Accelerators -- unseen by the novice user -- may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.
Aesthetic and minimalist design
Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.
Help users recognize, diagnose, and recover from errors
Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.
Help and documentation
Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user's task, list concrete steps to be carried out, and not be too large.
ISTD 2003

18. Performance Measurement
Quantitative and/or qualitative measurements when the system is used for completing a specific task or tasks: number of errors, time needed, proportion of satisfied/frustrated users, etc.
Lifecycle stage: Competitive analysis, final testing
Users needed: At least 10
Main advantage: Hard numbers. Results easy to compare.
Main disadvantage: Does not find individual usability problems.
ISTD 2003

19. User uses the system while continuously thinking out loud
Thinking Aloud
User uses the system while continuously thinking out loud
Lifecycle stage: Iterative design, formative evaluation
Users needed: 3-5
Main advantage: Pinpoints user misconceptions. Cheap test.
Main disadvantage: Unnatural for users. Hard for expert users to verbailize.
ISTD 2003

20. Observation
Observation of the user working, interfering with them as little as possible
Lifecycle stage: Task analysis, follow-up studies
Users needed: 3 or more
Main advantage: Ecological validity; reveals user’s real tasks. Suggests functions and features.
Main disadvantage: Appointments hard to set up. No experimenter control.
ISTD 2003

21. Users are presented with a written questionnaire
Questionnaires
Users are presented with a written questionnaire
Lifecycle stage: Task analysis, follow-up studies
Users needed: At least 30
Main advantage: Finds subjective user preferences. Easy to repeat.
Main disadvantage: Pilot work needed (to prevent misunderstandings). Data may not be reliable.
ISTD 2003

22. Users are interviewed and presented with a set of questions
Interviews
Users are interviewed and presented with a set of questions
Lifecycle stage: Task analysis
Users needed: 5
Main advantage: Flexible, in-depth attitude and experience probing.
Main disadvantage: Time consuming. Hard to analyze and compare.
ISTD 2003

23. Users are brought together to discuss and identify issues
Focus Groups
Users are brought together to discuss and identify issues
Lifecycle stage: Task analysis, user involvement
Users needed: 6-9 per group
Main advantage: Spontaneous reactions and group dynamics.
Main disadvantage: Hard to analyze. Low validity.
ISTD 2003

24. Logging Actual Use
Automatic collection of statistics about the detailed use of the system
Lifecycle stage: Final testing, follow-up studies
Users needed: At least 20
Main advantage: Finds highly used (or unused) features. Can run continuously.
Main disadvantage: Analysis programs needed for huge mass of data. Violation of users’ privacy.
ISTD 2003

25. Collection of feedback from the users using the system
User Feedback
Collection of feedback from the users using the system
Lifecycle stage: Follow-up studies
Users needed: Hundreds
Main advantage: Tracks changes in user requirements and views.
Main disadvantage: Special organization needed to handle replies.
ISTD 2003