1. Chapter 15: Design and Evaluation in the Real World: Communicators and Advisory Systems Group 4: Tony Masi, Sam Esswein, Chris Troisi, Brian Rood

2. Introduction Text books make design and usability testing processes sound straightforward and step-by-step. This is never the case in real world applications. It is only when you become involved in the actual design project that the pressure, trade-offs, and demands influence the way the design project is carried out. In the real world, design and evaluation are VERY closely integrated. You don’t do one without the other.

3. Key Concepts Show how design and evaluation are brought together in development of interactive products Show how design and evaluation are brought together in development of interactive products Show how different combinations of design and evaluation methods are used in practice Show how different combinations of design and evaluation methods are used in practice Describe the various design trade-offs and decisions made in real-world situations Describe the various design trade-offs and decisions made in real-world situations

4. Key Issues The number of iterations through the design- evaluate cycle depend on the requirements of the project The number of iterations through the design- evaluate cycle depend on the requirements of the project Many practical issues and unexpected events must be dealt with by the design team Many practical issues and unexpected events must be dealt with by the design team No two projects are ever exactly the same; each will face a different set of constraints, demands, and crises No two projects are ever exactly the same; each will face a different set of constraints, demands, and crises

5. Real World Usability Cases http://www.asktog.com/columns/042ButterflyBallot.htmlhttp://www.asktog.com/columns/042ButterflyBallot.html - Florida Butterfly Ballot http://www.asktog.com/columns/042ButterflyBallot.html http://www.digitwireless.com/http://www.digitwireless.com/ - FasTap Keypad http://www.digitwireless.com/ http://www.consult-me.co.uk/csc-case-studies.htm http://www.consult-me.co.uk/csc-case-studies.htm - Some other usability case studies http://www.consult-me.co.uk/csc-case-studies.htm

6. Designing Mobile Communicators Nokia Nokia Communicator for adultsCommunicator for adults Philips Philips Communicator for childrenCommunicator for children

7. Background Mobile communicators are devices that commonly push the limits of technology. They combine many functions such as those listed to the right into one small device. Therefore, a key challenge is how to make these devices useable and affordable to most people. Send and receive telephone calls, email, faxes, and other messages Send and receive telephone calls, email, faxes, and other messages Keep contact info, journal entries, calendars, and other notes Keep contact info, journal entries, calendars, and other notes Watch stock and other finance reports Watch stock and other finance reports

8. Background Other considerations: Device should be small and lightweight Device should be small and lightweight Made of light materialsMade of light materials Small enough to fit in pocket or small bagSmall enough to fit in pocket or small bag Software must work with limited screen size and memorySoftware must work with limited screen size and memory What will user be doing while using the device? What will user be doing while using the device? Device should work in all conditionsDevice should work in all conditions Various noise levels Various noise levels Various lightings Various lightings Device should take little effort to operate so user can concentrate on other things at the same timeDevice should take little effort to operate so user can concentrate on other things at the same time like driving? like driving? Hands free mode?Hands free mode? Simple operations should be one-handedSimple operations should be one-handed Answer call Answer call Browse internet Browse internet

9. Background Other considerations: Device should account for distractions that may occur Device should account for distractions that may occur Interaction sequences should be easy to return to and continue after interruptionInteraction sequences should be easy to return to and continue after interruption Internet trade off: How long should the device remain connected?Internet trade off: How long should the device remain connected? Tasks tend to be time-critical, triggered by other people or events, relatively brief, low in terms of attention to be applied to the task, and very personal Tasks tend to be time-critical, triggered by other people or events, relatively brief, low in terms of attention to be applied to the task, and very personal Flow among tasks should be smoothFlow among tasks should be smooth Easy flow between related functions such as contact list and telephone functionEasy flow between related functions such as contact list and telephone function Must be simple and not involve much training (if any) Must be simple and not involve much training (if any) Needs to be robust and reliable Needs to be robust and reliable

10. Designing Nokia’s Mobile Communicator What kind of lifecycle? What kind of lifecycle? Iterative user-centered approachIterative user-centered approach Top level design concept cycle (Fig 15.2)Top level design concept cycle (Fig 15.2) Which methods to use? Which methods to use? Ethnographic researchEthnographic research Scenarios and task models (Fig 15.3, 15.4)Scenarios and task models (Fig 15.3, 15.4) Confidential product issues: Confidential product issues: First in the market is keyFirst in the market is key Evaluation must be very limitedEvaluation must be very limited No real usersNo real users

11. Physical aspects: Physical aspects: Screen sizeScreen size Number of buttons versus functionalityNumber of buttons versus functionality Soft keys with changing functions or values Soft keys with changing functions or values Hard coded keys that always return same value Hard coded keys that always return same value Consistency issues Consistency issues Internal consistency (within mobile software)Internal consistency (within mobile software) External consistency (with desktop software)External consistency (with desktop software) User testing User testing None before release (confidentiality)None before release (confidentiality) Summative testing & questionnaires afterSummative testing & questionnaires after Designing Nokia’s Mobile Communicator

12. Designing Philips’ communicator for children design cycle: iterative and evolutionary design cycle: iterative and evolutionary which methods: low-fidelity prototyping participatory design interface metaphors which methods: low-fidelity prototyping participatory design interface metaphors physical aspects: color, shape, size, robustness pen input bags to protect screen physical aspects: color, shape, size, robustness pen input bags to protect screen

13. Designing Philips’ communicator for children user involvement: children involved throughout prototypes evaluated constantly invaluable insights for the designers user involvement: children involved throughout prototypes evaluated constantly invaluable insights for the designers lessons learned: agree on assumptions in requirements think of follow-on projects early on users are not designers act quick and dirty if necessary lessons learned: agree on assumptions in requirements think of follow-on projects early on users are not designers act quick and dirty if necessary

14. Case study 2: A telephone response information system (TRIS) Interactive voice response systems are common in government offices and large companies. Do you know of examples that you have used? Interactive voice response systems are common in government offices and large companies. Do you know of examples that you have used? Why are these systems often so frustrating to use? Forming a mental model is difficult because there is no visual feedback and the user must remember the menu structure Why are these systems often so frustrating to use? Forming a mental model is difficult because there is no visual feedback and the user must remember the menu structure Many menus and deep menus are particularly difficult Many menus and deep menus are particularly difficult

15. Why was TRIS difficult to use? Having to remember the menu structure. Having to remember the menu structure. The programmers traded computational elegance for usability, e.g., the system asked for social security number and employee identification number, confusing users who did not have both. The programmers traded computational elegance for usability, e.g., the system asked for social security number and employee identification number, confusing users who did not have both. TRIS was comprised of different systems each with its own interaction style. Users were not told this but when they moved between the systems they experienced sudden, unexplained changes. TRIS was comprised of different systems each with its own interaction style. Users were not told this but when they moved between the systems they experienced sudden, unexplained changes.

16. How was TRIS evaluated? A combination of techniques were used: - a review of the literature provided information about problems with interactive voice response systems - expert reviews - GOMS analysis of the proposed redesign A combination of techniques were used: - a review of the literature provided information about problems with interactive voice response systems - expert reviews - GOMS analysis of the proposed redesign The redesign was implemented - usability tests confirmed that the redesigned system offered better usability than the original design The redesign was implemented - usability tests confirmed that the redesigned system offered better usability than the original design

17. Why was using different methods valuable? The evaluators were able to build-up a broad picture of usability problems. The evaluators were able to build-up a broad picture of usability problems. Using GOMS and heuristic evaluation they could explore the potential benefits of the redesigned system. Using GOMS and heuristic evaluation they could explore the potential benefits of the redesigned system. User testing enabled them to confirm that the redesigned system offered better usability. User testing enabled them to confirm that the redesigned system offered better usability. User satisfaction questionnaires confirmed that users preferred the redesigned system. User satisfaction questionnaires confirmed that users preferred the redesigned system.

18. Key points Design involves trade-offs Design involves trade-offs Design space for making changes when upgrading a product is limited Design space for making changes when upgrading a product is limited Cycles of rapid prototyping and evaluation allow designers to examine alternatives Cycles of rapid prototyping and evaluation allow designers to examine alternatives Simulations are useful when evaluating systems used by large numbers of people Simulations are useful when evaluating systems used by large numbers of people Piecing together evidence from a variety of sources can be valuable Piecing together evidence from a variety of sources can be valuable