1. CS 5150 1 CS 5150 Software Engineering Lecture 12 Usability 2

2. CS 5150 2 Course Administration Test 2 Answer books can be collected from the reception at 301 College Avenue Mid-semester break Next lecture is Wednesday, October 14

3. CS 5150 3 Usability: The Analyze/Design/Evaluate Loop Evaluate ? Design Build Analyze requirements User testing

4. CS 5150 4 Evaluation Iterative improvements during development. Making sure that a system is usable before launching it. Iterative improvements after launch. Categories of evaluation methods: Analytical evaluation: without users (not in CS 5150) Measurements on operational systems Empirical evaluation: with users

5. CS 5150 5 Evaluation How do you measure usability? Usability comprises the following aspects: Effectiveness – the accuracy and completeness with which users achieve certain goals Measures: quality of solution, error rates Efficiency – the relation between the effectiveness and the resources expended in achieving them Measures: task completion time, learning time, clicks number Satisfaction – the users' comfort with and positive attitudes towards the use of the system Measures: attitude rating scales From ISO 9241-11

6. CS 5150 6 Evaluation based on Measurement Basic concept: log events in the users' interactions with a system Examples from a Web system Clicks (when, where on screen, etc.) Navigation (from page to page) Keystrokes (e.g., input typed on keyboard) Use of help system Errors May be used for statistical analysis or for detailed tracking of individual user.

7. CS 5150 7 Evaluation based on Measurements Analysis of system logs Which user interface options were used? When was was the help system used? What errors occurred and how often? Which hyperlinks were followed (click through data)? Human feedback Complaints and praise Bug reports Requests made to customer service

8. CS 5150 8 The Search Explorer Application: Reconstruct a User Sessions

9. CS 5150 9 Evaluation with Users Testing the system, not the users! Stages of evaluation with users: Prepare Conduct sessions Analyze results User testing is time-consuming, expensive, and essential.

10. CS 5150 10 Evaluation with Users: Preparation Determine goals of the usability testing “Can a user find the required information in no more than 2 minutes?” Write the user tasks “Answer the question: how hot is the sun?” Recruit participants Use the descriptions of users from the requirements phase to determine categories of potential users

11. CS 5150 11 Usability Laboratory Concept: monitor users while they use system Evaluators User one-way mirror

12. CS 5150 12 Evaluation with Users: Sessions Conduct the session –Usability Lab –Simulated working environment Observe the user –Human observer(s) –Video camera –Audio recording Inquire satisfaction data

13. CS 5150 13 Evaluation with users: Results analysis If possible, use statistical summaries. Pay close attention to areas where users –were frustrated –took a long time –could not complete tasks Respect the data and users' responses. Do not make excuses for designs that failed. Note designs that worked and make sure they are incorporated in the final product.

14. CS 5150 14 Evaluation Example: Eye Tracking

15. CS 5150 15 Evaluation Example: Eye Tracking

16. CS 5150 16 Refining the Design based on Evaluation Do not allow evaluators to become designers Designers are poor evaluators of their own work, but know the requirements, constraints, and context of the design: Some user problems can be addressed with small changes Some user problems require major changes Some user requests (e.g., lots of options) are incompatible with other requests (e.g., simplicity) Designers and evaluators need to work as a team

17. CS 5150 17 Usability: Design Tensions in Networked Systems Designers wish to control what the user sees, but users wish to configure their own environments. Accessibility requires that designers do not control parameters such as font size. Client computers and network connections vary greatly in capacity Client software may run on various operating systems, which may not be the current version. Be explicit about the assumptions you make about the user's computer, Web browser, etc.

18. CS 5150 18 Usability and Cost Good usability may be expensive in hardware or special software development User interface development may be a major part of a software development project Programming environments provide powerful user interface toolkits Costs are multiplied if a user interface has to be used on different computers or migrate to different versions of systems Web browsers provide a general purpose user interface where others maintain the user interface software

19. CS 5150 19 Changes in user interface design Examples of change: 1995 to 2009

20. CS 5150 20 1990

21. CS 5150 21 1995

22. CS 5150 22 2003

23. CS 5150 23 2009

24. CS 5150 24 2001

25. CS 5150 25 2003

26. CS 5150 26 2009

27. CS 5150 27 1995

28. CS 5150 28 2006

29. CS 5150 29 1995

30. CS 5150 30 2003

31. CS 5150 31 2009

32. CS 5150 32 1995

33. CS 5150 33 2006