1. User Interface Evaluation Usability Inquiry Methods http://jthom.best.vwh.net/usability/ http://www.cs.umd.edu/~zzj/UsabilityHome.html

2. Usability Inquiry Methods  Usability experts learn about the users’ likes, dislikes, needs, etc. of the system through:  Observation  Verbal questioning  Written questioning  Widely used in practice.  Different methods have different costs, but in general, this is relatively cheap.

3. Usability Inquiry Methods  Contextual Inquiry  Field Observation  Questionnaires  Interviews  Focus Groups  Logging Actual Use

4. Contextual Inquiry  Contextual Inquiry is also known as proactive field studies.  Applicable Stages:  Requirements & Design  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 2.

5. Contextual Inquiry  Usability Issues Covered  Effectiveness: No  Efficiency: No  Satisfaction: No  Quantitative Data is NOT collected.  Can NOT be conducted remotely.  Can be used on any system.

6. Contextual Inquiry  What is it?  Before designing the system, the expert(s) visit the users’ workplace and question them.  This should occur before any design has been done.

7. Contextual Inquiry  How can I do it?  Determine who your users are.  Go visit them where they work.

8. Contextual Inquiry  How can I do it?  Talk to them about the system  How do they currently do their job?  How would you like to do your job?  What do you like about the current system/method?  What don’t you like about the current system/method?  http://jthom.best.vwh.net/usability/context.htm  http://www.cs.umd.edu/~zzj/FieldStu.htm

9. Field Observation  Applicable Stages:  Test & Deployment  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 2.

10. Field Observation  Usability Issues Covered  Effectiveness: Yes  Efficiency: No  Satisfaction: Yes  Quantitative Data is NOT collected.  Can NOT be conducted remotely.  Can be used on any system.

11. Field Observation  What is it?  Usability experts observe users in the field using the system/product.

12. Field Observation  How can I do it?  Go to the users’ workplace and simply observe.  Things to look for:  What is the user’s mental model?  Are the users using it the way you expect?  You don’t want them to know you are evaluating them.  http://www.cs.umd.edu/~zzj/FieldObs.htm

13. Questionnaires  Applicable Stages:  Any stage of development (depending on the questions)  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 2.

14. Questionnaires  Usability Issues Covered  Effectiveness: Yes  Efficiency: No  Satisfaction: Yes  Quantitative Data is NOT collected.  Can be conducted remotely.  Can be used on any system.

15. Questionnaires  What is it?  Written lists of questions that you distribute to your users.

16. Questionnaires  How can I do it?  Develop a list of questions on paper, web, email, etc. and give the questionnaire(s) to the users.  The users will answer the questions and return the questionnaires to you.  http://jthom.best.vwh.net/usability/question.htm  http://www.acm.org/~perlman/question.html

17. Interviews  Applicable Stages:  Any stage of development (depending on the questions)  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 1.

18. Interviews  Usability Issues Covered  Effectiveness: Yes  Efficiency: No  Satisfaction: Yes  Quantitative Data is NOT collected.  Can be conducted remotely.  Can be used on any system.

19. Interviews  What is it?  You interview users and ask them questions.

20. Interviews  How can I do it?  Develop a list of questions for the users.  Meet with the users, individually.  Ask them the questions and log the responses  Written and/or taped

21. Interviews  How can I do it?  Interview Tips:  Clearly define this is an interview.  Ask open ended questions to get the user talking. Yes-No questions are bad.  Begin with less demanding topics and progress to more difficult topics.  Don’t ask questions to support your belief or hypothesis.  Do not answer your own questions.  Do not agree or disagree … remain neutral.

22. Interviews  How can I do it?  Probes: used to encourage the subjects to continue speaking, or to guide their response in a particular direction  Addition Probe  Encourages more information or clarifies certain responses from the test users. Either verbally or nonverbally the message is, "Go on, tell me more," or "Don't stop."

23. Interviews  How can I do it?  Reflecting Probe  Uses a nondirective technique, encourages the test user to give more detailed information. The interviewer can reformulate the question or synthesize the previous response as a proposition.  Directive Probe  Specifies the direction in which a continuation of the reply should follow without suggesting any particular content. A directive probe may take the form of "Why is the (the case)?"

24. Interviews  How can I do it?  Defining Probe  Requires the subject to explain the meaning of a particular term or concept.  http://jthom.best.vwh.net/usability/surveys.htm  http://www.cs.umd.edu/~zzj/Interview.htm

25. Focus Groups  Applicable Stages:  Test & Deployment  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 6.

26. Focus Groups  Usability Issues Covered  Effectiveness: Yes  Efficiency: No  Satisfaction: Yes  Quantitative Data is NOT collected.  Can be conducted remotely with email, bulletin boards, conference calls, etc.  Can be used on any system.

27. Focus Groups  What is it?  A group of users are gathered to talk about the system.  The expert acts as the moderator.  Should conduct more than 1 focus group.

28. Focus Groups  How can I do it?  Bring a group of users together and begin.  Collect data  http://www.cs.umd.edu/~zzj/FocusGro.htm

29. Logging Actual Use  Applicable Stages:  Test & Deployment  Personnel  Usability Experts, approximately 1.  Developers, 0.  Users, 6.

30. Logging Actual Use  Usability Issues Covered  Effectiveness: Yes  Efficiency: Yes  Satisfaction: Maybe  Quantitative Data is NOT collected, in most cases.  Can be conducted remotely.  Can be used on most systems.

31. Logging Actual Use  What is it?  The computer automatically collect usage data.  You could ask the user to log their usage, but that’s not practical.

32. Logging Actual Use  How can I do it?  Usability Logger  http://www.usabletools.com/  Automatic capture of keyboard, mouse, etc.  VideoCam and other products.

33. Logging Actual Use  Facts On Logging Actual Use  You know exactly what the user is doing.  You don’t know why, but you do know what, when, where.  You don’t know how the user feels.  http://www.cs.umd.edu/~zzj/Logging.htm

34. Usability Inquiry Methods most useful  Contextual Inquiry  Field Observation  Questionnaires  Interviews  Focus Groups (least useful)  Logging Actual Use