Find out what they like, and how they like it, And let ‘em have it just that way. Fats Waller Usability Testing 101 Gina Pearson, Web Manager January 29, 2003

Today’s Goals  Primary purpose is to learn how ERS is integrating usability testing into its Web design & development process  Review why user-centered design and testing (usability engineering) are important  Explain the usability testing process at ERS

Today’s Goals  You will learn how to:  Assemble your test team  Recruit & select test participants  Formulate a test plan  Create testing tasks & scenarios  Facilitate a test session  Test without a lab  Record, compile & analyze results  Communicate results effectively

Why Is Usability Important?

Ballot Usability What should have been addressed? –Font size –Lighting –Text vs. background color –Instructions –Layout –Act of voting (punching ballot card)

Ballot Usability  Costs  $20,000 to run usability tests addressing  Layout  Multiple votes  Dimpled ballots  About $1 billion spent by both candidates to try to get elected

Failure of Web Sites  Users could find information only 42% of time (15 large commercial sites - Spool study)  Sixty-two percent of web shoppers gave up looking for an item (Zona study)  Only 51% of sites complied with simple Web usability principles (Forrester study of 20 major sites)

User-Centered Design  Philosophical & Practical Approach  User-centric - not developer, designer nor organization-centric  Who are the users?  What information do they want?  Based on research and data not opinions  Testable & verifiable

The Traditional Development Process

User-Centered Design  A methodical approach to producing a web site or any user interface. It is a practical and systematic way to deliver a product that works for users.  Test early & often  Build a little, test a little, fix a little  At ERS, we are incorporating usability testing & other user assessment techniques as a regular part of our web design & development process

User-Centered Design  Types of user assessment techniques  Interviews  Ethnographic observation / field research  Focus groups  User personas  Expert (heuristic) reviews  Usability testing  Customer satisfaction surveys  Customer trend analysis  Search engine query analysis  What else?

Myths About Usability Testing  Usability testing:  Is time consuming, expensive & otherwise resource intensive  Typically involves hiring a specialized consulting company  Requires a formal testing lab  Must be videotaped  Always involves writing a detailed analysis and report of results

Diagnostic User Testing Find out what is and is not working well on the site. In a diagnostic usability test, you usually want to answer questions like these:  Do users complete a task successfully?  If so, how fast do they do each task? Is that fast enough to satisfy them?  What paths do they take in trying? Do those paths seem efficient enough to them?  Where do they stumble?— What problems do they have?— Where do they get confused?  What words or paths are they looking for that are not now on the site?

Methods  Assemble test team  Administrator  Facilitator  Observers / recorders  Video recordings operator (optional)

Methods  Define goals & scope of what you are testing  Recruit & compensate participants  Develop testing tasks & script  Set up testing environment  Draft participation agreement  Rehearse & refine test protocol  Conduct tests  Analyze & report out results

Methods  Test Environment

Methods  How many participants are needed for a usability test?  It depends!  Typical range is from users in each test  Our approach: 3 users per test but conduct several rounds of testing throughout an application or feature’s development  If you have different potential user groups (for example, small business owners, unemployed, blue-collar workers), try to include representatives of all these groups

Methods  Participant recruitment & compensation  Typically time and labor intensive - consider outsourcing (average $200 per participant)  Go through professional networks & contacts for volunteers  Conduct testing at conferences & other events with volunteer representatives from your target audiences  Always give volunteers a small token of your appreciation (t-shirt, mug, etc)

Methods  Task construction: where do ideas for tasks come from?  Concerns about information architecture, design, terminology, functionality, etc that have surfaced among the product development team  Trend analysis of s from site customers  Search engine query analysis  Trend analysis of call center inquiries  What else?

Methods  Constructing usability testing tasks  Find or scavenger hunt tasks  Verb-based tasks  Scenario-based tasks

Methods  Conducting the Test (about 2 hours)  Pre-test briefing for observers  Greet participant & administer participation agreement  Introduce recorder & observers  Explain testing process to participant  Administer pre-test interview & activities (if any)  Facilitate testing tasks  Administer post-test interview questions  Involve observers in general Q&A session at the end  After participant leaves, debrief recorder & observers

Methods  Developing the testing script & facilitating the test  3 parts of a usability test:  Pre-test interview  Testing tasks (“think out loud” methodology)  Post-test interview  See another sample script at:  pt.doc pt.doc

Methods  Characteristics of a Good Facilitator  Quick learner  Ability to establish instant rapport  Empathic “people person”  Good listening skills  Patient  Flexible & adaptable  Long & focused attention span  Impartial: not wedded to a particular design

Methods  Analyzing & reporting the test results using the KJ method  Assemble your testing team immediately after the last test for a brainstorming session.  Ask each participant to write down each major observed problem on an index card or sticky note.  Put all index cards or sticky notes on a large wall or board, no discussions!  Read each others cards silently, add additional problems.

Methods  Analyzing & reporting the test results using the KJ method  Sort problems by area. Eliminate duplicates if total agreement.  Name each group of problem areas.  Vote for most important problems, each participant has ten votes.  Assign one team member the task of writing up the results of brainstorming session.  Disseminate results of brainstorming session to development team via

For Further Information  Contact:  Gina Pearson, Web Manager  Brenda Powell, Web Analytics Manager

For Further Information Don't Make Me Think! A Common Sense Approach to Web Usability Steve Krug The book every manager should read! Handbook of Usability Testing: How to Plan, Design & Conduct Effective Tests By Jeffrey Rubin Practical, step-by-step guidelines, requires no engineering or human factors training. A Practical Guide to Usability Testing By Joseph Dumas & Janice Redish Premier text in the field.

For Further Information Jakob Nielsen ( Why You Only Need to Test With 5 Users Success Rate: The Simplest Usability Metric Usability Metrics

For Further Information Jakob Nielsen ( Cost of User Testing a Web Site When to Outsource the Recruiting of Test Users g.html

For Further Information Jared Spool ( Using Paper Prototyping to Manage Risks Usability Labs: Our Take Eight Is More Than Enough

For Further Information Interaction Architect 13 Common Objections Against User Requirements Analysis & Why You Should Not Believe Them cle b.htm Observation Methods & Tips for Usability Testing icle shd.htm

For Further Information DialogDesign ( Ethics in Human Computer Interaction Usability Test of ple

For Further Information Usability.gov  Why Should I Do Iterative Usability Testing?  How Can I Encourage People in My Organization to Conduct Usability Engineering and Testing?  Do I Need a Lab to Do Usability Testing?  How Many Participants Are Needed for a Usability Test?  How Much Does It Cost to Do Usability Testing?

For Further Information Training Opportunities  National Cancer Institute Usability Seminar Series & Education Training Program,  Society for Technical Communication WDC Chapter,  User Interface Engineering  Human Factors International  User Experience Conferences (Nielsen Norman Group)