1. The Information School of the University of Washington Information System Design Info-440 Autumn 2002 Session #15

2. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 2 Agenda Usability testing –Video example –Assignment #4 –Continue with principles Lab –Bring your paper prototypes, notes & ideas

3. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 3 Admin Announcements –Will schedule some kind of Visio lab It will be very collaborative We’ll teach each other

4. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 4 Upcoming Next week –Read Nielsen, Chapters 5 & 6 –Heuristic evaluations Prototyping project –Bring prototypes to lab –1 st iteration of prototype (25 Nov) Quiz #4 –25 November –Chapter #6 only Assignment #4 –December 4

5. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 5 Review Usability –Purposes? Science Design Social Video excerpt –What did you learn?

6. The Information School of the University of Washington Video

7. The Information School of the University of Washington Assignment #4

8. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 8 Four parts Part I: –Heuristic evaluation Part II: –Task analysis Part III: –Task performance estimate Part IV: –Usability evaluation

9. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 9 Deliverable A usability report –Outline is given in assignment Target audience –Product design team, which includes business people, engineers, visual designer, and information architect Overall objective: –Understand the efficiency of finding movie times online

10. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 10 Part III: Objective Derive an estimate for how long it takes to discover movie times using: –Telephone –News paper

11. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 11 User goal Find the playing times for movie, M, at University District, Seattle, Washington, between 6-9pm on Thursday Movies are: –M1: 8 mile –M2: Harry Potter

12. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 12 Newspaper: Procedure Select newspaper Start state: –Newspaper folded/closed End state: –You find the exact time ‘Reset’ Newspaper

13. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 13 Telephone: Procedure Use this number: –Xxx Start state –Phone on hook End state –You find the exact time ‘Reset’ the phone

14. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 14 Data collection Each person in the class will generate this table: User IDTrialMethodTime Dgh-mem1news40 Dgh-mem2news67 Dgh-mem2tele40 Dgh-mem1tele67

15. The Information School of the University of Washington Part IV

16. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 16 Part IV: Main objective Determine which of three movie sites is the most usable The sites: –Yahoo: http:/movies.yahoo.com –Lycos: http://entertainment.lycos.com/movies/http://entertainment.lycos.com/movies/ –MSN http://www.movietickets.com/default.asp?afid=msn

17. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 17 User goal Find the playing times for movie, M, at University District, Seattle, Washington, between 6-9pm on Thursday Movies are: –M1: 8 mile –M2: Harry Potter

18. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 18 Experimental design Participant #1 –Complete goal for M1 –Complete goal for M2 Participant #2 –Complete goal for M2 –Complete goal for M1

19. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 19 Procedure Go to jump page –http://courses.washington.edu/i440a/movie.html Have participant read task –Tell participant the movie & what site to use End state: –User tells you the time the movie is playing ‘Return’ to jump page

20. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 20 Part IV: Data collection Each person in the class will generate this table: User IDTrialMethodTime Dgh-P1m1Yahoo40 Dgh-P1m2Yahoo67 Dgh-P2m2Yahoo30 Dgh-P2m1Yahoo87

21. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 21 Everyone generates 8 records… User IDTrialMethodTime Dgh-mem1news40 Dgh-mem2news67 Dgh-mem2tele40 Dgh-mem1tele67 Dgh-P1m1Yahoo40 Dgh-P1m2Yahoo67 Dgh-P2m2Yahoo30 Dgh-P2m1Yahoo87

22. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 22 The GOAL: Class bar-chart (Avg. time to find movie info)

23. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 23 Select your site: Write your name on handout Yahoo MSN Lycos

24. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 24 Part I: Heuristic Evaluation Using Nielsen guidelines, carry out a heuristic evaluation of the movie site, M. Read chapter 5 of Nielsen

25. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 25 Part II: Task Analysis Suppose you are sitting on your couch and you decide "I want to go to a movie with my friend Joe". Develop a detailed task analysis for solving this goal. The goal is satisfied when you are sitting with Joe in the theatre.

26. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 26 Report format Executive summary –A maximum of two hundred words on what you did and what you learned The objective –Propose an objective for this assignment Heuristic evaluation –Present the heuristic evaluation Task analysis –Present the task analysis

27. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 27 Report format (cont) The participants –Describe briefly who the participants were The task –Describe the goal that you gave the participants and how you timed them Findings –Include the graph here, a discussion of it, and your conjectures about task completion times Issues and recommendations –List a maximum of three issues that you discovered and your recommendation for solving the issues

28. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 28 Report format (cont) Discussion of methods –Discuss what it was like to observe the users. Did anything surprise you? Did they change your understanding of the goal and tasks? How did the usability evaluation compare to the heuristic evaluation? References –If required Appendix A –Include a table showing the raw data

29. The Information School of the University of Washington Break

30. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 30 Reasons to test Develop understanding of user goals –Before beginning a project –You observe people working with systems –Goal: Identify tactics, strengths, weaknesses, etc. Competitive tests (X vs Y) –You believe a competitor is better than you –Determine which is more usable –Goal: Uncover best practices at competitor site

31. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 31 Reasons to test Alternative interfaces (X 1 Vs X 2) –You have two alternative versions of a UI –Goal: Determine which is the better direction Test-and-iterate –Iterative development process –Goal: You seek incremental improvement

32. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 32 Usability process 1.Decide on objectives 2.Decide on participant profile & develop scenario 3.Analyze system & make predictions 4.Decide on participant goals 5.Prompt participants to complete goals 6.Carefully observe 4-6 participants Identify critical incidents Measure performance (time, errors, etc.) 7.Debrief participants 8.Prioritize issues Team discussion Develop presentations, reports, etc.

33. The Information School of the University of Washington A huge number of details are important – We will consider only a few

34. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 34 Some details Human subject ethics Participants Creating good tasks Being a good moderator Think aloud protocols

35. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 35 Human subject ethics Guidelines –Acknowledge that that system is being tested, not the participant (remind repeatedly) –Tell the participant that she is free to leave at any time –Reveal who is watching & what is being recorded –Do not report results such that a participant is identified –Avoid telling the participant that he is making mistakes or doing things wrong –Acknowledge participants efforts but in a neutral fashion Bottom line: Treat people with great respect Read: Nielsen (pp. 181-185) *on quiz*

36. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 36 Participants Participant profile –Computer/net experience –Experience with system –Interests Example –You want to study an online teen magazine –Consider 15 year old girls who use IM Anyone else The most important decision to make

37. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 37 Creating good tasks Clear beginning and end states Easily stated Cover target areas of the system Consider –Find the 5-day weather forecast for Toronto –Here’s $75.00 -- use eBay to buy something

38. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 38 Being a good moderator

39. The Information School of the University of Washington Copyright David Hendry (INFO-440 session 15 - 11/20/2002) 39 Think aloud