How to Conduct Usability Testing: In 9 Easy Steps By Jennifer L. Bowie

What is Usability? “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) “A function of particular users performing particular tasks in a particular environment” (Smith et al. 68) The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4) The “people who use the product can do so quickly and easily to accomplish their own tasks” (Dumas and Redish 4) User-centered design, not “user- friendly” User-centered design, not “user- friendly”

What is Usability Testing? An empirical study of a product’s usability by observing actual users do real tasks with the product An empirical study of a product’s usability by observing actual users do real tasks with the product Involves: Involves: Real users Real users Real tasks Real tasks Specific usability goals/concerns Specific usability goals/concerns Observing and recording the testing Observing and recording the testing Data analysis Data analysis

Step 1: Analyze your Actual Users & Choose Profiles Who are your actual users? You may need to break your users into typical user categories. Consider: Who are your actual users? You may need to break your users into typical user categories. Consider: Demographics of your users (age, sex, race, education level, cultural background, socioeconomic status,…)? Will these differences impact use? Demographics of your users (age, sex, race, education level, cultural background, socioeconomic status,…)? Will these differences impact use? Experience level (with the product, with products of the same genre, with required technology,...)? Experience level (with the product, with products of the same genre, with required technology,...)? Other things: motivation, learning style, subject matter knowledge, location of use, physical characteristics, people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities), … Other things: motivation, learning style, subject matter knowledge, location of use, physical characteristics, people with disabilities or impairments (from color blindness and learning disabilities to more severe disabilities), …

Step 1: Analyze your Actual Users & Choose Profiles con. Create user profiles: Create user profiles: Break users into clear subgroups Break users into clear subgroups Profile/Define the characteristics of each subgroup Profile/Define the characteristics of each subgroup Choose user profiles to test: Choose user profiles to test: Ideally users from all major profiles will be tested Ideally users from all major profiles will be tested If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues If limited testing: Choose profiles based on highest number of users in that profile or profiles that you think may have the greatest usability issues

Step 2: Select Test Points What is your objective? What test points “get you there”? What is your objective? What test points “get you there”? Test Task/Procedure- look for tasks with a high chance or high cost of user failure Test Task/Procedure- look for tasks with a high chance or high cost of user failure Test Terminology- consider level of user Test Terminology- consider level of user Test your document design strategies- consider cueing patterns, heading/layout, navigation, extraordinary features, how text and visuals relate... Test your document design strategies- consider cueing patterns, heading/layout, navigation, extraordinary features, how text and visuals relate...

Step 3: Choose Type of Test Task (can-they-do-it): Requires users to perform a procedure Task (can-they-do-it): Requires users to perform a procedure Terminology (can-they- understand-it): Requires users to provide a summary of what they have read or definitions of key terms Terminology (can-they- understand-it): Requires users to provide a summary of what they have read or definitions of key terms Document Design (can-they-find- it): Requires users to use mocked- up portions of the manual to find information on key topics Document Design (can-they-find- it): Requires users to use mocked- up portions of the manual to find information on key topics

Step 4: Set Performance Objectives Consider how you will come up with your data Consider how you will come up with your data Time related: time taken to perform procedure or find a topic Time related: time taken to perform procedure or find a topic Error related: number of errors made during a procedure, number of time passage re-read, number of tries Error related: number of errors made during a procedure, number of time passage re-read, number of tries Objectives should be task individualized Objectives should be task individualized

Step 5: Select your Users Actual users: always best Actual users: always best Similar users: not as best, may have to find ways to make them more even with a actual testers (give them more information, have them pretend they know less) Similar users: not as best, may have to find ways to make them more even with a actual testers (give them more information, have them pretend they know less) Non-Similar users: very different and the worse types. May have to do a lot of work to make them more appropriate Non-Similar users: very different and the worse types. May have to do a lot of work to make them more appropriate

Step 6: Create Test Plan Create a written plan for the testing: Create a written plan for the testing: Choose order of tasks: start easy Choose order of tasks: start easy Create written test materials: Create written test materials: Task list for users Task list for users Written welcome speech/Intro to be read to user Written welcome speech/Intro to be read to user Consent forms Consent forms Observation forms Observation forms Pre-task and post task questionnaires & interview questions Pre-task and post task questionnaires & interview questions Other materials Other materials Define team member’s roles: Define team member’s roles: Facilitator/Briefer (necessary): Often only team member to interact with users Facilitator/Briefer (necessary): Often only team member to interact with users Observation recorder/note taker (necessary) Observation recorder/note taker (necessary) Camera operator (optional) Camera operator (optional) Help desk operator (optional) Help desk operator (optional) Test administer (optional) Test administer (optional) Create a script/plan for the actual test Create a script/plan for the actual test

Steps 7-9: Work your Data 7. Record Information Accurately: timing, copious notes, possible recordings… 8. Interpret the Data: Calculate data Calculate data Analyze the data Analyze the data Determine cause of problems Determine cause of problems Determine scope/severity of problems Determine scope/severity of problems Determine what needs to be fixed/changed Determine what needs to be fixed/changed 9. Incorporate the feedback into the Design

Good Luck & Have Fun! Where to find out more: Barnum Usability Testing and Research Barker Writing Software Documentation, Chapter 6 “Conducting Usability Tests” Hom “General Concepts of Usability Testing”