Formative Evaluation cs3724: HCI

User Interface Metrics Ease of learning learning time, … Ease of use perf time, error rates… User satisfaction surveys… Not “user friendly”

Scenario-Based Design ANALYZE analysis of stakeholders, field studies claims about current practice Problem scenarios Scenario-Based Design DESIGN Activity scenarios metaphors, information technology, HCI theory, guidelines iterative analysis of usability claims and re-design Information scenarios Interaction scenarios PROTOTYPE & EVALUATE summative evaluation formative evaluation Usability specifications

Usability Engineering Reqs Analysis Design Evaluate Develop

Usability Engineering Reqs Analysis Design Evaluate Develop many iterations

Usability Engineering Formative evaluation Summative evaluation

Claims Analysis + advantages - disadvantages E.g. file management: + fast learning + spatial learning aid memory switching between devices long to locate a known file… + flexible, power + fast for experts Learning commands aesthetics lacking?...

Usability Evaluation Analytic Methods: Empirical Methods: Heuristic Evaluation Models: GOMS, Fitts Law Empirical Methods: Usability Testing Wizard of Oz Controlled Experiment

Nielsen’s 10 Heuristics Visible status, feedback wysiwyg User control, undo, exits wizards Familiar, speak user’s language Acrobat error msg Consistent, standards Word, PPT Recognition over recall web nav, phone menu Efficient, expert shortcuts Word bold Aesthetic, minimalist phone book Prevent errors HomeFinder Error recovery undo, back Help, task-based IIS help doc

Speak the User’s Language

Help documentation Context help, help doc, UI

Applying Heuristics Lets look at a website…

Early Usability Testing Formative: helps guide design Early in design process when architecture is finalized, then its too late! A few users (3-5) Identify usability problems, incidents Qualitative feedback from users Quantitative usability measures

Usability Test Procedure Wizard of Oz: act out the system Benchmark task set No cheating! Don’t teach them how to use the UI Repeat: Give user a task Ask user to “think aloud” Observe, note mistakes and problems Avoid interfering, hint only if completely stuck Interview Verbal feedback Questionnaire 30 min - 1 hour / user

Analyze Results Initial reaction: Mature reaction: “stupid user!”, “that’s developer X’s fault!”, “this sucks” Mature reaction: “how can we redesign UI to solve that usability problem?” the user is always right Identify usability problems Learning issues: e.g. can’t figure out or didn’t notice feature Performance issues: e.g. arduous, tiring to solve tasks Subjective issues: e.g. annoying, ugly Problem severity: critical vs. minor Redesign

In action: Pizza Ordering System Scenario: VT students, studying/busy/tired, poor, wants discounts, starving, credit card / hokie card/ no cash, 2 in together, half-n-half, split cost Volunteers! Design ideas Wizard of Oz usability test Redesign

Pizza R Us Drag your toppings: Whole peperoni Half-n-half mushrooms pineapple checkout PRICE: $10.00

Size of pizza: First half: 2nd half: Toppings: Toppings: Price: 0.00 large Small First half: Toppings: Peperoni Mushrooms Pineapple Price: 0.00 2nd half: Toppings: Peperoni Mushrooms pineapple

% pza –ppm –large -…