Evaluating User Interfaces Walkthrough Analysis Joseph A. Konstan

CSci October 10 n Introduction to Evaluation n Cognitive Walkthrough n Other Evaluation Methods

CSci Interface Development Methodology n Prototype and Iterate u keep iterating until it is good enough u evaluate along the way to assess n What is Good? What is Good Enough? u set usability goals u should relate to tasks

CSci Casual Iteration n Find major usability problems u missing features u user confusion u poor interaction n Try interface with specific tasks u first use designers, then move towards users u observe overall usage

CSci Casual Iteration n Remember the goal u don’t defend the interface u don’t bias the tests towards the interface n If possible, allow user exploration u may even lead to capturing new tasks n Consider alternative ways to fix a problem

CSci Limits of Casual Iteration n Does not indicate when to stop n Financial trade-offs n Justification of delay

CSci Usability Goals and Measures n Concrete, quantitative measures of usability u learning time u use time for specific tasks and users u error rates u measures of user satisfaction n Comparative usability goals u compare with prior versions or competitors

CSci Things to Watch n Goals should be realistic u 100% is never realistic n Many goals go beyond the application UI u training, manuals n Testing goals should help improve the UI u detail--not just good/bad

CSci Exercise: Setting Usability Goals n In project groups, come up with 2 usability goals for your project u discuss the feasibility of testing these goals F what is needed for the test F when in the process can they be tested? F how much effort, user preparation/training, etc.? F what would you learn from the test?

CSci Interface Evaluation n Goals of interface evaluation u find problems u find opportunity for improvement u determine if interface is “good enough”

CSci With or Without Users n Users are expensive and inconsistent u usability studies require several users u some users provide great information, others little n Users are users u cannot be simulated perfectly n Best choice--Both

CSci Evaluation Without Users n Quantitative Methods u GOMS/keystroke analysis u back-of-the-envelope action analysis n Qualitative Methods u expert evaluation u cognitive walkthrough u heuristic evaluation

CSci Walkthrough Analysis n Economical interface evaluation u low-fidelity prototype u development team F users optional n Effective, if u goal is improvement, not defense u some team members skilled u proper motivation

CSci Cognitive Walkthrough n Goals u imagine user’s experience u evaluate choice-points in the interface u detect confusing labels or options u detect likely user navigation errors n Start with a complete TCUID scenario u never try to “wing it” on a walkthrough

CSci Tell a Believable Story n How does the user accomplish the task n Action-by-action n Based on user knowledge and system interface

CSci Best Approach n Work as a group u don’t partition the task n Be highly skeptical u remember the goal! n Every gap is an interface problem

CSci Who Should Do the Walkthrough n Designers, as an early check n Team of designers & users u remember: goal is to find problems u avoid making it a show n Skilled UI people may be valuable team members

CSci How Far Along n Basic requirements u description or prototype of interface u know who users are (and their experience) u a task description u a list of actions to complete the task (scenario) F DO NOT try to create the action list on the fly! n Viable once the scenario and interface sketch are completed

CSci How to Proceed n For each action in the sequence u tell the story of why the user will do it u ask critical questions F will the user be trying to produce the effect? F will the user see the correct control? F will the user see that the control produces the desired effect? F will the user select a different control instead? F will the user understand the feedback to proceed correctly?

CSci Walkthroughs are not Perfect n They won’t find every problem u limited by nature F new users who know what task they need to accomplish F biased towards correct action sequence u limited in implementation F hard to shed the expertise of evaluators n A useful tool in conjunction with others

CSci Exercise: Cognitive Walkthrough Analysis n In non-project groups of 3-5 n Users and Task to be announced n Scenario developed jointly n Perform walkthrough u identify problems u estimate error probabilities (25% intervals) n Remember who your users are!

CSci GOMS/Keystroke Analysis n Formal action analysis u accurately predict task completion time for skilled users n Break task into tiny steps u keystroke, mouse movement, refocus gaze u retrieve item from long-term memory n Look up average step times u tables from large experiments

CSci GOMS/Keystroke Analysis n Primary utility: repetitive tasks u e.g., telephone operators u benefit: can be very accurate (within 20%) u may identify bottlenecks n Difficulties u challenging to decompose accurately u long/laborious process u not useful with non-experts

CSci Back-of-the-Envelope Action Analysis n Coarse-grain u list basic actions (select menu item) u each action is at least 2-3 seconds u what must be learned/remembered? u what can be done easily? u documentation/training? n Goal is to find major problems u Example: 1950’s 35mm camera

CSci Expert Evaluation n Usability specialists are very valuable u double-specialists are even better n An inexpensive way to get a lot of feedback n Be sure the expert is qualified in your area

CSci Looking Ahead n Next week: Heuristic Evaluation n Walkthroughs Due u “raw” notes notes from each step of walkthrough copy of prototype used, markups copy of scenarios used (note changes or fixes) u processed results 1-2 pages of issues identified, solutions not needed