Copyright 1999 all rights reserved Evaluating Paper Prototypes n How do you do this? –What are the considerations? –What are the steps? ?
Copyright 1999 all rights reserved Evaluating Paper Prototypes n Before –Preparing for the evaluation –Selecting tasks n During –Teaching the user –Recording what happens n After –Using the results
Copyright 1999 all rights reserved Preparing for the Evaluation n Create a presentable design representation that is legible and neat n Select a representative user n Think through what you will do with the user –Define tasks for the user –Decide on explanations & instructions –Decide how you will observe & record what happens
Copyright 1999 all rights reserved Selecting Tasks n Select enough tasks to cover the range of expected use –Use the requirements to help determine the range –Go for variety first, comprehensiveness second “Use the faucet to wash your hands.”
Copyright 1999 all rights reserved Selecting Tasks n Make tasks specific but do not tell the user how to operate the interface –Describe the result not the steps to get there Right: “Assume the sink is full of bits of food from having rinsed the dishes. Wash these bits down the drain.” Wrong: “Move the faucet to the left, right, front and back while it is on power wash.”
Copyright 1999 all rights reserved Teaching the User n Try to avoid teaching the user –Better interfaces require less instruction n Reflect real use of the system –Will the user have instructions? – Will the user get training? n Teaching changes the evaluation –Bad teaching can ruin a good design –Good teaching can save a bad design
Copyright 1999 all rights reserved Do NOT Teach the User if... n There will not be instructions for the user under real conditions n The task is perceived as so easy that the user expects to do it without instruction n The user will not be using the interface on any regular basis
Copyright 1999 all rights reserved Do Teach the User if... n There will be instructions for the user under real conditions n The interface provides new capabilities unfamiliar to the user n The interface has many different functions and capabilities Warning! If you use instructions, write them beforehand
Copyright 1999 all rights reserved Observing and Recording n Know what you want to observe –And be alert for the unexpected n Do not rely on memory –Be prepared to record what happens
Copyright 1999 all rights reserved Time n Record the time to perform various parts of the task n Materials –Stopwatch –Chart to record the start time for each subtask
Copyright 1999 all rights reserved Errors n Categorize each error the user makes –Typing errors –Path error - additional steps or steps in an unexpected order –Command error - an incorrect action for the task the user is trying to do –Motor error - user selects wrong object accidentally
Copyright 1999 all rights reserved Thinking Aloud about the Task n Asked the user to talk while doing the task – Have them tell what they are doing as they perform the task n Have them read a set of instructions aloud before they begin the task –Helps to get them going –Can use the task description for this n Remind them to talk if they stop –But be low-key about it
Copyright 1999 all rights reserved User Understanding n Check user understand after she or he has done the tasks –Give the user a short test on the interface Multiple choice testMultiple choice test –Have the user describe to you how to do various tasks with the interface
Copyright 1999 all rights reserved User’s Perception n Don’t ask how they liked it overall n Ask specific questions about the interface Right: “Do you think the temperature is easy to adjust?” Wrong: “Rate on a scale from one to five, how easy you found the interface to use.”
Copyright 1999 all rights reserved Watch out for Response Bias! n User’s want you to feel good about your design –If you ask them if they like it, they will say YES –Ask them instead what parts they like & dislike n Users want to be successful –Norman on tendency of people to see problems as their fault
Copyright 1999 all rights reserved Using the Results n There is no value unless you apply the result of the evaluation n Use each type of observation for insight into the design n Listen to the user, but do not completely abandon your own common sense
Copyright 1999 all rights reserved Time n Does it take too long to do the task? n Reduce the number of steps –Favor frequent tasks –Simple tasks should be simple Combine temperature change and turning the water “on” Build a one-step key for hand washing button and cold drinks
Copyright 1999 all rights reserved Errors n Does your user make a lot of errors? n Avoid designs that require precise motor performance –Avoid very fine scales on sliders –Avoid multiple level cascading menus –Avoid moving from one end of the screen to the other and back, multiple times
Copyright 1999 all rights reserved Confusion n Is the user confused? –Long pauses on user talking aloud? –Subtask take longer than others? –Moving from one selection to another? –Vague explanation of the interface?
Copyright 1999 all rights reserved Confusion n Cures for confusion –Put sequential steps near each other Put together selections that are likely to occur togetherPut together selections that are likely to occur together –Change labels - make the mapping better Add a faucet icon showing what shower looks likeAdd a faucet icon showing what shower looks like –Modify the conceptual model Draw a picture of a sink for specifying the faucet directionsDraw a picture of a sink for specifying the faucet directions
Copyright 1999 all rights reserved Total Confusion n Is the user lost? –User cannot complete the task –User succeeds only with great difficult or a lot of help
Copyright 1999 all rights reserved Total Confusion n Select a new conceptual model –Ask the user what she or he thinks is the model for the interface –Ask the user to ignore the interface you have designed and describe how they might perform the tasks you have specified