Cognitive Walkthrough Jon Kolko Professor, Austin Center for Design

Cognitive Walkthrough A method for evaluating the learnability of a product, based on a theory of problem solving in unfamiliar situations. 2

Cognitive Walkthrough A method for evaluating the learnability of a product, based on a theory of problem solving in unfamiliar situations. COGNITIVE WALKTHRU IS… Intended for first-time use Assumes systems don’t require training Is based on a credible story of a hypothetical user 3

Problem Solving in Unfamiliar Situations When confronted with a unique, new situation, people leverage a problem solving theory based on exploration and progressive behavior refinement. We determine what controls are available for us to take action We select the control that makes the most sense towards incremental progress We perform the action using the control We evaluate the results of our action to see if we have made positive, negative, or neutral progress 4

Cognitive Walkthrough: Getting Ready Prepare a prototype of any fidelity Identify your user, assuming they are a system novice Identify the main goals this user will have with the system Identify and list each step through the product that support the user’s main goals Extract (or make) a screen shot that directly maps to each step in the walkthrough 5

Cognitive Walkthrough: The Evaluation Walk through the tasks screen by screen. At each screen, ask these questions, and answer them with a credible story: Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect they are trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 6

Cognitive Walkthrough: The Evaluation Will the user try to achieve the right effect? The user of the system has an end goal in mind, but needs to accomplish various tasks to complete it. Will they even know to perform the specific steps along the way? The user wants to print a document. Will they know that they first need to select a printer? The user wants to manipulate an object in Illustrator. Will they know that they first need to change to “manipulation” mode? 7

Cognitive Walkthrough: The Evaluation Will the user notice that the correct action is available? Is the option visible and on the screen, or at least in a place the user will likely look? The user wants to perform an action, but it’s buried in a menu The user wants to perform an action, but must swipe to another screen first The user wants to perform an action, but they have to tap and hold to make a button appear 8

Cognitive Walkthrough: The Evaluation Will the user associate the correct action with the effect trying to be achieved? Is a label or icon worded in a way that the user expects? The user wants to perform an action, but they don’t realize that a given icon is how the action is initiated The user sees a menu with a variety of buttons, and isn’t sure which to press 9

Cognitive Walkthrough: The Evaluation If the correct action is taken, will the user see that progress is being made towards their goal? Is there any feedback showing that the user selected the right option or is making forward momentum? The user arrives at a screen, but it has nothing to do with what they selected The user arrives at a screen, but it’s unclear how or why they got there 10

Cognitive Walkthrough: For Example Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that user is trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 11

Cognitive Walkthrough: For Example Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that user is trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 12

Cognitive Walkthrough: For Example Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that user is trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 13

Cognitive Walkthrough: For Example Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that user is trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 14

Cognitive Walkthrough: For Example Will the user try to achieve the right effect? Will the user notice that the correct action is available? Will the user associate the correct action with the effect that user is trying to achieve? If the correct action is performed, will the user see that progress is being made towards their goal? 15

Logging Cognitive Walkthrough Findings Unique Identifier Unique Screen ID Problem Description Severity (1-5, 5 is high) Frequency (1-5, 5 is common) Proposed Solution JK_1 1.4.A The icon necessary to launch the menu is not recognizable 3 4 Include a better icon, or include a tooltip that shows where to click; alternatively, use a text button for important actions (“Menu”) JK_2 1.4.B It is not clear that the goals will appear, or where to find them Make the goal area more prominent Define a severity and frequency rating to indicate the relative impact of the critical incident and the number of times this incident is likely to be identified by a user Use a unique identifier that combines the initials of the evaluator with a running number tally (JK = Jon Kolko, 1 = incident number one) 16

Cognitive Walkthrough A method for evaluating the learnability of a product, based on a theory of problem solving in unfamiliar situations. COGNITIVE WALKTHRU IS… Intended for first-time use Assumes systems don’t require training Is based on a credible story of a hypothetical user RUNNING A COGNITIVE WALKTHROUGH/ Prepare a prototype of any fidelity Identify your user Identify the main goals this user will have with the system Identify and list each step through the product that support the user’s main goals Extract (or make) a screen shot that directly maps to each step in the walkthrough Walk through the screens and ask the four questions. Catalog the problems in a spreadsheet Propose changes to the problems Prepare a presentation that describes the top findings 17