1. Evaluating User Interfaces Walkthrough Analysis Joseph A. Konstan konstan@cs.umn.edu

2. CSci 51152 October 10 n Introduction to Evaluation n Cognitive Walkthrough n Other Evaluation Methods

3. CSci 51153 Interface Development Methodology n Prototype and Iterate u keep iterating until it is good enough u evaluate along the way to assess n What is Good? What is Good Enough? u set usability goals u should relate to tasks

4. CSci 51154 Casual Iteration n Find major usability problems u missing features u user confusion u poor interaction n Try interface with specific tasks u first use designers, then move towards users u observe overall usage

5. CSci 51155 Casual Iteration n Remember the goal u don’t defend the interface u don’t bias the tests towards the interface n If possible, allow user exploration u may even lead to capturing new tasks n Consider alternative ways to fix a problem

6. CSci 51156 Limits of Casual Iteration n Does not indicate when to stop n Financial trade-offs n Justification of delay

7. CSci 51157 Usability Goals and Measures n Concrete, quantitative measures of usability u learning time u use time for specific tasks and users u error rates u measures of user satisfaction n Comparative usability goals u compare with prior versions or competitors

8. CSci 51158 Things to Watch n Goals should be realistic u 100% is never realistic n Many goals go beyond the application UI u training, manuals n Testing goals should help improve the UI u detail--not just good/bad

9. CSci 51159 Exercise: Setting Usability Goals n In project groups, come up with 2 usability goals for your project u discuss the feasibility of testing these goals F what is needed for the test F when in the process can they be tested? F how much effort, user preparation/training, etc.? F what would you learn from the test?

10. CSci 511510 Interface Evaluation n Goals of interface evaluation u find problems u find opportunity for improvement u determine if interface is “good enough”

11. CSci 511511 With or Without Users n Users are expensive and inconsistent u usability studies require several users u some users provide great information, others little n Users are users u cannot be simulated perfectly n Best choice--Both

12. CSci 511512 Evaluation Without Users n Quantitative Methods u GOMS/keystroke analysis u back-of-the-envelope action analysis n Qualitative Methods u expert evaluation u cognitive walkthrough u heuristic evaluation

13. CSci 511513 Walkthrough Analysis n Economical interface evaluation u low-fidelity prototype u development team F users optional n Effective, if u goal is improvement, not defense u some team members skilled u proper motivation

14. CSci 511514 Cognitive Walkthrough n Goals u imagine user’s experience u evaluate choice-points in the interface u detect confusing labels or options u detect likely user navigation errors n Start with a complete TCUID scenario u never try to “wing it” on a walkthrough

15. CSci 511515 Tell a Believable Story n How does the user accomplish the task n Action-by-action n Based on user knowledge and system interface

16. CSci 511516 Best Approach n Work as a group u don’t partition the task n Be highly skeptical u remember the goal! n Every gap is an interface problem

17. CSci 511517 Who Should Do the Walkthrough n Designers, as an early check n Team of designers & users u remember: goal is to find problems u avoid making it a show n Skilled UI people may be valuable team members

18. CSci 511518 How Far Along n Basic requirements u description or prototype of interface u know who users are (and their experience) u a task description u a list of actions to complete the task (scenario) F DO NOT try to create the action list on the fly! n Viable once the scenario and interface sketch are completed

19. CSci 511519 How to Proceed n For each action in the sequence u tell the story of why the user will do it u ask critical questions F will the user be trying to produce the effect? F will the user see the correct control? F will the user see that the control produces the desired effect? F will the user select a different control instead? F will the user understand the feedback to proceed correctly?

20. CSci 511520 Walkthroughs are not Perfect n They won’t find every problem u limited by nature F new users who know what task they need to accomplish F biased towards correct action sequence u limited in implementation F hard to shed the expertise of evaluators n A useful tool in conjunction with others

21. CSci 511521 Exercise: Cognitive Walkthrough Analysis n In non-project groups of 3-5 n Users and Task to be announced n Scenario developed jointly n Perform walkthrough u identify problems u estimate error probabilities (25% intervals) n Remember who your users are!

22. CSci 511522 GOMS/Keystroke Analysis n Formal action analysis u accurately predict task completion time for skilled users n Break task into tiny steps u keystroke, mouse movement, refocus gaze u retrieve item from long-term memory n Look up average step times u tables from large experiments

23. CSci 511523 GOMS/Keystroke Analysis n Primary utility: repetitive tasks u e.g., telephone operators u benefit: can be very accurate (within 20%) u may identify bottlenecks n Difficulties u challenging to decompose accurately u long/laborious process u not useful with non-experts

24. CSci 511524 Back-of-the-Envelope Action Analysis n Coarse-grain u list basic actions (select menu item) u each action is at least 2-3 seconds u what must be learned/remembered? u what can be done easily? u documentation/training? n Goal is to find major problems u Example: 1950’s 35mm camera

25. CSci 511525 Expert Evaluation n Usability specialists are very valuable u double-specialists are even better n An inexpensive way to get a lot of feedback n Be sure the expert is qualified in your area

26. CSci 511526 Looking Ahead n Next week: Heuristic Evaluation n Walkthroughs Due u “raw” notes notes from each step of walkthrough copy of prototype used, markups copy of scenarios used (note changes or fixes) u processed results 1-2 pages of issues identified, solutions not needed