1. Evaluation
ECE 695
Alexander J. Quinn
March 30, 2018

2. Types of evaluation Lab study in-person Log analysis remote
think aloud protocol
Log analysis remote
A-B testing
Remote observation remote
monitor screen
Ethnography in-person, in-situ
embed with users for extended period of time
Interviews in-person
e.g., contextual interview
Cognitive walk-through no users
Heuristic evaluation no users

3. Jakob Nielsen’s heuristics
Match the real world
Consistency & standards
Help & documentation
User control & freedom
Visibility of system status
Flexibility & efficiency
Error prevention
Recognition, not recall
Error reporting, diagnosis, and recovery
Aesthetic & minimalist design
… and/or your own heuristics
// Credit:

4. Ben Shneiderman’s Golden Rules
Consistency
Shortcuts
Feedback
Dialog closure
Error prevention and easy recovery
Undo
User in control
Reduce short-term memory load
Credit: +

5. Heuristic evaluation Evaluators are experts ≥ 5 evaluators
1-2 hours to examine application
Rate severity on a controlled scale
Severity
I don't agree that this is a usability problem at all 1
Cosmetic problem only: need not be fixed unless extra time is available on project 2
Minor usability problem: fixing this should be given low priority 3
Major usability problem: important to fix, so should be given high priority 4
Usability catastrophe: imperative to fix this before product can be released

6. Exercise We will do a simplified heuristic evaluation.
Go to your app store and download any free “outliner” app on your phone
Task: Create an outline with at least 3 items
Use the evaluation form to note flaws found
Note: In proper heuristic evaluation, we would have ≥5 evaluators per app spending 1-2 hours
Severity
I don't agree that this is a usability problem at all 1
Cosmetic problem only: need not be fixed unless extra time is available on project 2
Minor usability problem: fixing this should be given low priority 3
Major usability problem: important to fix, so should be given high priority 4
Usability catastrophe: imperative to fix this before product can be released