Extended Cognitive Walkthrough Judy Kay CHAI: Computer human adapted interaction research group School of Information Technologies

2 Overview Predictive method (with caveat Grigoreanu et al)Grigoreanu et al Cognitive Walkthrough Benefits Disadvantages (About 50,000 results from Google)

3 Postconditions for this class Describe the uses of Cognitive Walkthrough Describe the processes for conducting Cognitive Walkthrough analyses Describe advantages and limitations Ability to perform a Cognitive Walkthrough study Justify the use of Cognitive Walkthrough in the overall testing of a pervasive computing application

4 Focus on learnability Appropriate for novice or casual users cf Think-Aloud? Does not focus on speed cf Think-Aloud? Sequence is not known prior to inspection Assesses user success and recovery from errors Conducted by experts cf Think-Aloud?

5 Model of Exploratory Learning User has task –0. will user understand this sub-task is needed? Explores system for useful elements –1. will correct action be obvious? eg button visible –2. will user understand instructions? eg user understands the label on the button Selects one to try User interprets system response –3. will user know if progress has been made?

Extended Cognitive Walkthrough Takes account of mental model…..

7 User has task –0. will user understand this sub-task is needed (given their mental model) Explores system for useful elements –1. will correct action be obvious? –(given their mental model) –2. will user understand instructions? – (given their mental model) Selects one to try User interprets system response –3. will user know if progress has been made? –(given their mental model)

8 Example with basic CW Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training Abstract user goals: Buy a ticket to the required destination Determine whether I can afford to buy the ticket to a particular destination Adapted from Newman and Lamming, Interactive System Design, 1995 )

9 Example with basic CW Design a cash-operated machine for quick, easy purchase of train tickets by passengers, without training Designer goal breakdown to subgoals: Determine fare to pay –indicate destination –indicate one-way or return Dispense ticket –pay money –get ticket and change Adapted from Newman and Lamming, Interactive System Design, 1995 )

10 The interface – lo-fi prototype

Concrete user task Class activity: define 3 concrete tasks

12 The interface – lo-fi prototype

Concrete user task Buy a one-way ticket to Town Hall

14 The task: Buy a one-way ticket to Town Hall

15 How designer wants it to work – method 1 where use does not use keypad Click destination ie Town Hall Click journey type ie one way

16 0. Task? –Buy one-way ticket to Town Hall 1. Is correct action obvious? –Two possibilities: Destination One way / return –Will user know both must be set?

17 How to Help the User? Indicate extra information needed Indicate steps 1 and 2 can be done in either order Give some feedback on effect of each select action Reorganise layout so that Steps 1 and 2 are followed by the fare display

We alter our lo-fi prototype and check this

19 2. Will user understand instructions? 3. Will user interpret machine action correctly?

20 2. Will user understand instructions? –Yes – due to labels and instructions 3. Will user interpret machine action correctly? –Yes (buttons light up, new state appears)

21

22 1. Correct action obvious? –Yes 2. Will user understand instructions? –Yes 3. Will user interpret machine action correctly? –Yes

23 Paying 1. Correct action obvious? 2. Will user understand instructions? 3. Will user interpret machine action correctly?

What are the cases to consider for payment (in cash) Consider case where user has exact change

25 Paying Designer intends user to Pay in money and click Click 4. Press for ticket User then lifts flap to collect the ticket

26 Paying – exact change case 1. Correct action obvious? –Yes 2. Will user understand instructions? –Yes 3. Will user interpret machine action correctly? –Unclear (no feedback on money accepted so far)

We alter our lo-fi prototype and check this

28 Revised design gives feedback on amount paid so far

Consider other cases eg user realises they have insufficient money

30 Buying a Ticket: Insufficient Money? 1. Correct action obvious?

31 Summary of Flaws (so far) Confusion about need for steps 1 and 2 No feedback on amount deposited No means to get money back So far …...

32 Goals and Tasks In this example: Goal: buy a ticket Sub-goals: (determine fare) and pay User tasks: concrete cases used in CW

33 Extended cognitive walkthrough Adding user's mental model What does user believe? How do you find this out? What did we assume about the users mental model? What differences are there in the MM for: A novice user An expert user

Class activity: List aspects of the users MM that would be relevant to the train ticket interface

What are the implications of some likely cases: user familiar with existing interface user familiar with a different bus ticket interface

Class activity on Extended CW:

37 Interface is info3315 website Task INFO3315 student has just started semester and wants to determine the deadline for the first assessed work Define the relevant mental model

38 Mental Model Class web site is at There is a lecture and a lab each week for most classes There is assessed practical work for most classes There are fixed deadlines for such work

39 Task sequence Go to class web site is at [This is the home page] See the heading Labs and Deadlines Scan down to Deadline: Prototype demos This is listed under 19 Sep, Week 8

40 User has task –0. will user understand this sub-task is needed (given their mental model) Explores system for useful elements –1. will correct action be obvious? –(given their mental model) –2. will user understand instructions? – (given their mental model) Selects one to try User interprets system response –3. will user know if progress has been made? –(given their mental model)

41 User has task –0. will user understand this sub-task is needed –Yes Explores system for useful elements –1. will correct action be obvious? –Perhaps not –may go to assessment page as mental model primes them for the word Assessment and that page does not have dates –may miss the Deadline in the heading –2. will user understand instructions? –Perhaps not –may well see Sep 19 (lecture date) and think that is the deadline Selects one to try User interprets system response –3. will user know if progress has been made? –In this simple task the user does not need to take action (so we can fit all this on a slide)

42 Potential GOTCHAs Need to have a prototype that is complete enough for a walkthough of an interesting concrete case But this is a technique for early in design process, where designer is ready to change it You then need to define: the users, and their relevant MMs a good set of tasks the correct steps (intended by designer) Evaluator must imagine peoples thoughts on first using this interface Keep referring to the mental model Carefully assess vocabulary/text in terms of mental model Repeat process over the tasks, and for each important class of mental model

43 Returning to the reading…. A very impure CW that call ICW Additional forms of CW Expert evaluator leads the process Works with development team reps Agile process Some Validation (cf lab trial of SSCW part) Two stages: Earlier is SSCW – Simplified Streamlined Later involves users in SPW – Simplified Pluralistic

44 Summary of uses Relatively inexpensive in our very, very lightweight approach Desk check –No users –Better with expert evaluators Generally applicable Novice, casual, intermittent users Focus on learnability

45 Summary of usefulness Really useful technique, even for designer Better if done by Outsider Expert But students and non-expert evaluators still can gain from using it Part of early usability evaluation because Low cost No users needed