1. Utah School of Computing HCI Validation Richard F. Riesenfeld University of Utah Fall 2009 Lecture Set 16

2. Utah School of Computing slide 2 Student Name Server Fall 2009 Major Considerations - 1 Stage of design -Conceptual, preliminary, detail Novelty of project -Do we know what we are doing? Number of expected users -How important is this? -How amenable to change will it be?

3. Utah School of Computing slide 3 Student Name Server Fall 2009 Major Considerations - 2 Criticality of the interface -Are lives at stake if there are problems? Cost of product -Allocation for testing Time available for testing Experience of designers and evaluators

4. Utah School of Computing slide 4 Student Name Server Fall 2009 Expert Review methods - 1 Heuristic evaluation -Experts critique it wrt established criteria Guidelines review -Does it meet “spec” -Can be an overwhelming list -Bureaucratic approach

5. Utah School of Computing slide 5 Student Name Server Fall 2009 Expert Review methods - 2 Consistency inspection -Experts check of style, function, form, etc. Cognitive walkthrough -Experts perform role of users -Try to assess its success from experience

6. Utah School of Computing slide 6 Student Name Server Fall 2009 Expert Review methods - 3 Formal usability inspection -Moot court -Countervailing opinions -Can be unweildy

7. Utah School of Computing slide 7 Student Name Server Fall 2009 Comparative Evaluations - 1 Different experts see different issues -Can get caught with conflicting advice* -Limit the number of experts Get a “bird’s-eye” view in the beginning -Throw images on a wall, etc. * “ * “ For every PhD, there is an equal and opposite PhD”

8. Utah School of Computing slide 8 Student Name Server Fall 2009 Comparative Evaluations - 2 Formal (statistical) methods -Form a hypothesis -Determine dependent variables -Identify independent variables * “

9. Utah School of Computing slide 9 Student Name Server Fall 2009 Usability Labs, etc Hard to employ because of time and money constraints in product development -Development cycle schedule -Budgets -Corporate/Cultural Attitude

10. Utah School of Computing slide 10 Student Name Server Fall 2009 Controlled Experients Statistical testing -Establish level of statistical significance -At the 95% confidence level we know… Usability testing -Find flaws in the interface through more informal (inconclusive) methods -Empirical methods

11. Utah School of Computing slide 11 Student Name Server Fall 2009 Human Subjects -1 Careful, “This isn’t Kansas anymore!” Many new dimensions need attention Human respect and dignity -Voice generated check outs violated privacy  Military has NO privacy  Other cultures treat matters differently

12. Utah School of Computing slide 12 Student Name Server Fall 2009 Human Subjects -2 Real LEGAL issues, so get it right! -Informed consent -Understand your liability -Get it in writing, copy to each party Government, or institutional rules -We are not accustomed to this -Need cognizant approvals  IRBs, etc.  Research proposals, etc

13. Utah School of Computing slide 13 Student Name Server Fall 2009 Observations methods Have subjects “think aloud” -Will subjects be honest, etc. Use video recording Field tests -Study the successes/failures of the interface -Getting access -Reliance on memories  “How is it going?” (We tend to react to most recent)

14. Utah School of Computing slide 14 Student Name Server Fall 2009 Destructive Testing Hey, can you break this? Good for security Good for games Durability testing appropriate for some environments -ATM in high crime area -Military -Students, they can’t resist a challenge

15. Utah School of Computing slide 15 Student Name Server Fall 2009 Competitive Testing -1 Consumers Union, Road & Track, style -Take several into lab and have a “shoot out” Expensive Takes skill (like a movie review) -Depends on the criteria -Depends on good and representative judgment

16. Utah School of Computing slide 16 Student Name Server Fall 2009 Competitive Testing -2 Major Limitations -Limited coverage of features -Depends on initial user experiences

17. Utah School of Computing slide 17 Student Name Server Fall 2009 Surveys Tricky business, can lead to nearly any conclusion -Population selection -Question choices -Size -Leading questions, other bias Negative bias – users with complaints

18. Utah School of Computing slide 18 Student Name Server Fall 2009 Online Surveys More issues…

19. Utah School of Computing slide 19 Student Name Server Fall 2009 Conclusions HCI is a new game Not exact science Old methods not entirely applicable Need newer, faster, light weight, flexible, informal, subjective, intelligent approaches

20. Utah School of Computing slide 20 Student Name Server Fall 2009 Recommendations Use good judgment Trust good judgment -Yours -Others, whom you trust Be open to criticism and suggestion

21. Utah School of Computing slide 21 Student Name Server Fall 2009 Interpretation What is being said? What is the real issue? What is the real fix? RSI is a problem -Pain -Was cause keyboard or mouse? -Need different devices, or speech, or simply a better mouse and keyboard?

22. Utah School of Computing END HCI Validation End Lecture Set 16