1. Formative Evaluation
cs3724: HCI

2. User Interface Metrics
Ease of learning
learning time, …
Ease of use
perf time, error rates…
User satisfaction
surveys…
Not “user friendly”

3. Scenario-Based Design
ANALYZE
analysis of
stakeholders,
field studies
claims about current
practice
Problem scenarios
Scenario-Based Design
DESIGN
Activity
scenarios
metaphors,
information
technology,
HCI theory,
guidelines
iterative
analysis of
usability
claims and
re-design
Information scenarios
Interaction scenarios
PROTOTYPE & EVALUATE
summative
evaluation
formative
evaluation
Usability specifications

4. Usability Engineering
Reqs Analysis
Design
Evaluate
Develop

5. Usability Engineering
Reqs Analysis
Design
Evaluate
Develop
many iterations

6. Usability Engineering
Formative evaluation
Summative evaluation

7. Claims Analysis + advantages - disadvantages E.g. file management:
+ fast learning
+ spatial learning aid memory
switching between devices
long to locate a known file…
+ flexible, power
+ fast for experts
Learning commands
aesthetics lacking?...

8. Usability Evaluation Analytic Methods: Empirical Methods:
Heuristic Evaluation
Models: GOMS, Fitts Law
Empirical Methods:
Usability Testing
Wizard of Oz
Controlled Experiment

9. Nielsen’s 10 Heuristics Visible status, feedback wysiwyg
User control, undo, exits wizards
Familiar, speak user’s language Acrobat error msg
Consistent, standards Word, PPT
Recognition over recall web nav, phone menu
Efficient, expert shortcuts Word bold
Aesthetic, minimalist phone book
Prevent errors HomeFinder
Error recovery undo, back
Help, task-based IIS help doc

10. Speak the User’s Language

11. Help documentation
Context help, help doc, UI

12. Applying Heuristics
Lets look at a website…

13. Early Usability Testing
Formative: helps guide design
Early in design process
when architecture is finalized, then its too late!
A few users (3-5)
Identify usability problems, incidents
Qualitative feedback from users
Quantitative usability measures

14. Usability Test Procedure
Wizard of Oz: act out the system
Benchmark task set
No cheating! Don’t teach them how to use the UI
Repeat:
Give user a task
Ask user to “think aloud”
Observe, note mistakes and problems
Avoid interfering, hint only if completely stuck
Interview
Verbal feedback
Questionnaire
30 min - 1 hour / user

15. Analyze Results Initial reaction: Mature reaction:
“stupid user!”, “that’s developer X’s fault!”, “this sucks”
Mature reaction:
“how can we redesign UI to solve that usability problem?”
the user is always right
Identify usability problems
Learning issues: e.g. can’t figure out or didn’t notice feature
Performance issues: e.g. arduous, tiring to solve tasks
Subjective issues: e.g. annoying, ugly
Problem severity: critical vs. minor
Redesign

16. In action: Pizza Ordering System
Scenario: VT students, studying/busy/tired, poor, wants discounts, starving, credit card / hokie card/ no cash, 2 in together, half-n-half, split cost
Volunteers!
Design ideas
Wizard of Oz usability test
Redesign

17. Pizza R Us Drag your toppings: Whole peperoni Half-n-half mushrooms
pineapple
checkout
PRICE: $10.00

18. Size of pizza: First half: 2nd half: Toppings: Toppings: Price: 0.00
large
Small
First half:
Toppings:
Peperoni
Mushrooms
Pineapple
Price: 0.00
2nd half:
Toppings:
Peperoni
Mushrooms
pineapple

19. % pza –ppm –large -…