1. Copyright © 2005, Pearson Education, Inc. An Instructor’s Outline of Designing the User Interface 4th Edition by Ben Shneiderman & Catherine Plaisant Slides developed by Roger J. Chapman

2. Copyright © 2005, Pearson Education, Inc. Chapter 4 Expert Reviews, Usability Testing, Surveys, and Continuing Assessment

3. Copyright © 2005, Pearson Education, Inc. Introduction Designers fail to evaluate their designs adequately Experienced designers know that extensive testing is a necessity Evaluation plan addresses: –stage of design (early, middle, late) –novelty of project (well defined vs. exploratory) –number of expected users –criticality of the interface (life-critical medical system vs. museum exhibit support) –costs of product and finances allocated for testing –time available –experience of the design and evaluation team Length of time spent varies ( nature of project ) Cost of project ( 5% - 20% of project)

4. Copyright © 2005, Pearson Education, Inc. Expert Reviews formal expert reviews are more effective entail one-half day to one week effort –a lengthy training period may be required to explain the task domain or operational procedures Expert review methods: 1.Heuristic evaluation 8 golden rules 2.Guidelines review need to master guidelines first 3.Consistency inspection terminology, color, fonts, schemes, layouts 4.Cognitive walkthrough need individual as well as group (for discussion) 5.Formal usability inspection present interface and discuss merits and weaknesses

5. Copyright © 2005, Pearson Education, Inc. Expert Reviews (cont.) scheduled at several points in the development process –when experts are available –when the design team is ready for feedback 3-5 expert reviewers are necessary Caveat: –experts may not have an adequate understanding of the task domain or user communities – experienced expert reviewers are unaware of problems encountered by typical users, especially first-time users

6. Copyright © 2005, Pearson Education, Inc. Usability Testing and Laboratories designed to find flaws in the user interfaces

7. Copyright © 2005, Pearson Education, Inc. Usability Testing and Laboratories (cont.) The emergence of usability testing and laboratories –since the early 1980s –shift to user needs –Usability became important Usability testing benefits –sped up many projects –produced dramatic cost savings Typical modest usability lab –two 10 by 10 foot areas –one for the participants to do their work and another, separated by a half-silvered mirror, for the testers and observers Participants chosen to represent the intended users –background in computing –experience with the task –motivation, education, and ability with the natural language used in the interface

8. Copyright © 2005, Pearson Education, Inc. Usability Testing and Laboratories (cont.) Videotaping participants performing tasks is often valuable for later review and for showing designers or managers the problems that users encounter. Forms of usability testing: –Paper mockups –Discount usability testing –Competitive usability testing –Universal usability testing –Field test and portable labs –Remote usability testing –Can-you-break-this tests Limitations –Emphasizes first time usage –Provides limited coverage of interface features

9. Copyright © 2005, Pearson Education, Inc. Ethics Testing can be a distressing experience –pressure to perform, errors inevitable –feelings of inadequacy –competition with other subjects Golden rule –subjects should always be treated with respect

10. Copyright © 2005, Pearson Education, Inc. Managing Subjects Ethically Before the test –Don’t waste the user’s time Use pilot tests to debug experiments, questionnaires etc Have everything ready before the user shows up –Make users feel comfortable Emphasize that it is the system that is being tested, not the user Acknowledge that the software may have problems Let users know they can stop at any time –Maintain privacy Tell user that individual test results will be kept completely confidential –Inform the user Explain any monitoring that is being used Answer all user’s questions (but avoid bias) –Only use volunteers user must sign an informed consent form

11. Copyright © 2005, Pearson Education, Inc. During the test –Don’t waste the user’s time Never have the user perform unnecessary tasks –Make users comfortable Try to give user an early success experience Keep a relaxed atmosphere in the room Coffee, breaks, etc Hand out test tasks one at a time Never indicate displeasure with the user’s performance Avoid disruptions Stop the test if it becomes too unpleasant –Maintain privacy Do not allow the user’s management to observe the test Managing Subjects Ethically

12. Copyright © 2005, Pearson Education, Inc. After the test –Make the users feel comfortable State that the user has helped you find areas of improvement –Inform the user Answer particular questions about the experiment that could have biased the results before –Maintain privacy Never report results in a way that individual users can be identified Only show videotapes outside the research group with the user’s permission Managing Subjects Ethically

13. Copyright © 2005, Pearson Education, Inc. Survey Instruments Written user surveys for usability tests and expert reviews familiar, inexpensive and generally acceptable perform pretest with a pilot sample to improve results Keys to success 1.Clear obtainable goals –Tied to components of Objects and Action Interface model –Users could be asked for their subjective impressions about specific aspects of the interface –task domain objects and actions –syntax of inputs and design of displays 2.Develop focused items that help attain the goals

14. Copyright © 2005, Pearson Education, Inc. Survey Instruments (cont.) Other information that can be collected –users background age, gender, origins, education, income –experience with computers specific applications or software packages, length of time, depth of knowledge –job responsibilities decision-making influence, managerial roles, motivation –personality style introvert vs. extrovert, risk taking vs. risk aversive, early vs. late adopter, systematic vs. opportunistic –reasons for not using an interface inadequate services, too complex, too slow –familiarity with features printing, macros, shortcuts, tutorials –their feeling state after using an interface confused vs. clear, frustrated vs. in-control, bored vs. excited).

15. Copyright © 2005, Pearson Education, Inc. Surveys (cont.) Online surveys –avoid the cost of printing –the extra effort needed for distribution and collection of paper forms –can remind those who have not filled out the survey Many people prefer to answer a brief survey displayed on a screen, instead of filling in and returning a printed form, –potential bias in the sample.

16. Copyright © 2005, Pearson Education, Inc. A cceptance Test the customer or manager sets objective and measurable goals for hardware and software performance If the completed product fails to meet these acceptance criteria, the system must be reworked until success is demonstrated. Need specific, measurable criteria for the user interface: –Time to learn specific functions –Speed of task performance –Rate of errors by users –Human retention of commands over time –Subjective user satisfaction Objective is to uncover as many problems as possible in the prerelease phases

17. Copyright © 2005, Pearson Education, Inc. A cceptance Test (cont.) In a large system, there may be eight or 10 such tests to carry out on different components of the interface and with different user communities. Once acceptance testing has been successful, there may be a period of field testing before national or international distribution.

18. Copyright © 2005, Pearson Education, Inc. Evaluation During Active Use Successful active use requires constant attention from dedicated managers, user-services personnel, and maintenance staff Methods 1.Interviews and focus group discussions –individual users: pursue specific issues of concern. –Group discussions: determine the universality of comments. 2.Continuous user-performance data logging –patterns of system usage –Speed of user performance –Rate of errors –Frequency of request for online assistance 3.Online or telephone consultants 4.Online suggestion box or e-mail trouble reporting 5.Discussion and newsgroups

19. Copyright © 2005, Pearson Education, Inc. Controlled Psychologically-oriented Experiments (cont.) Scientific method as applied to human-computer interaction : –Deal with a practical problem: consider the theoretical framework –State a lucid and testable hypothesis –Identify a small number of independent variables –choose the dependent variables that will be measured –Judiciously select subjects and carefully or randomly assign subjects to groups –Control for biasing factors (non-representative sample of subjects or selection of tasks, inconsistent testing procedures) –Apply statistical methods to data analysis –Resolve the practical problem, refine the theory, and give advice to future researchers

20. Copyright © 2005, Pearson Education, Inc. Controlled Psychologically-oriented Experiments (cont.) Need to develop efficiency measurements and techniques for evaluation Controlled experiments can help fine tuning the human-computer interface of actively used systems. Performance could be compared with the control group. Dependent measures could include performance times, user- subjective satisfaction, error rates, and user retention over time.

21. Copyright © 2005, Pearson Education, Inc. Summary Definite need to evaluate user interfaces Expert reviews provide comprehensive evaluation Usability labs provide practical user interaction Survey instruments provide much information but should be administered ethically Acceptance tests should contain explicit measurable criteria to ensure project requirements have been satisfied Evaluation of interfaces during active use ensures higher level of user satisfaction Psychologically oriented experiments provide empirical evidence for evaluating user interfaces