1. University of Washington HCDE 518 & INDE 545 Design Research HCDE 518 & INDE 545 Winter 2012 With credit to Jake Wobbrock, Dave Hendry, Andy Ko, Jennifer Turns, & Mark Zachry

2. University of Washington HCDE 518 & INDE 545 Agenda  Announcements  Class Activity – Heuristic Evaluation  Lecture – Design Research  Break – 5 mins  Lecture & Discussion – A research project  Discussion Questions – Ben & Sarah  Next Class

3. University of Washington HCDE 518 & INDE 545 Announcements  R9, A3 due next Monday  P4 (demo & report) and P5 (presentation) on Wednesday  Questions?

4. University of Washington HCDE 518 & INDE 545 Class Activity: Heuristic Evaluation  Electronic voting machine  Download prototype:  http://courses.washington.edu/hcde518/lectures/12- AccuvoteWithPrinter.swf http://courses.washington.edu/hcde518/lectures/12- AccuvoteWithPrinter.swf  Download form:  http://courses.washington.edu/hcde518/lectures/12-HeuristicEvalForm.xlsx http://courses.washington.edu/hcde518/lectures/12-HeuristicEvalForm.xlsx  Nielsen’s heuristics:  http://www.useit.com/papers/heuristic/heuristic_list.html http://www.useit.com/papers/heuristic/heuristic_list.html  Use form and Nielsen's heuristics to evaluate the voting interface in groups of 2-3

5. University of Washington HCDE 518 & INDE 545 DESIGN RESEARCH VS. DESIGN PRACTICE Credit to Andy Ko, who also created the sketches

6. University of Washington HCDE 518 & INDE 545 What have we been doing so far?  Mostly design practice, with some support from research on design methods  Learning to think like a designer

7. University of Washington HCDE 518 & INDE 545 What is design thinking? There are at least seven things… 1.explicit problem reformulation 2.divergent/convergent thinking 3.exploitation of failure 4.externalization of ideas 5.emotional distance from ideas 6.group critique of ideas 7.articulation of rationale

8. University of Washington HCDE 518 & INDE 545 Problem Formation & Perspective Taking  Design is inherently about changing existing situations to preferred ones.  But how one formulates this preference affects the solutions we consider. For example, consider these…  How can we make lectures more efficient?  How can we spend less time in class?  How can we class time more productive?

9. University of Washington HCDE 518 & INDE 545 Convergent & Divergent Thinking  Our natural tendency is to judge; we see something wrong and we point it out (or we think it, at the very least)  A critical skill is being able to defer judgment, and leave yourself space to consider alternatives that may appear unworkable at first glance  When divergent thinking is interrupted by critique, we stop seeking relationships between ideas and cease combining ideas

10. University of Washington HCDE 518 & INDE 545 Acceptance & Exploitation of Failure  By generating hundreds of possible solutions to a problem, even solutions that are risky or unfavorable, one can learn a great deal about what makes one particular idea effective  generate alternatives  expect failure  utilize failure

11. University of Washington HCDE 518 & INDE 545 Externalization of Ideas  When we keep ideas in our minds, we are inherently limited by our capacity to imagine details  Ideas can be externalized as writing, drawings, or other forms  The sketching of ideas facilitates a dialog between designers and their decisions by forcing them to express details, often revealing which parts of an idea are still ill- or undefined

12. University of Washington HCDE 518 & INDE 545 Emotional Distance from Ideas  It’s easy to fall in love with our own ideas: we invest our time and our passions into them and become invested  But there’s no greater hindrance to quality than emotional attachment to an idea  It prevents us from seeking feedback, from listening to feedback, and from exploring superior alternatives

13. University of Washington HCDE 518 & INDE 545 Group Critique of Ideas  Ideas need diverse perspectives to evolve  One way to obtain these perspectives is through group critique, where designers present externalizations of their ideas and solicit feedback.  Group critiques help designers question their assumptions, but they also help synthesize feedback, by allowing people to build on each others’ critiques and to learn from each other’s struggles

14. University of Washington HCDE 518 & INDE 545 Articulation of Rationale  While externalizing one’s ideas creates the opportunity for critique, it does not guarantee it  A central part of design thinking is justifying one’s design choices  The act of explaining decisions forces you to evaluate your decisions and identify what evidence, principles, or knowledge to used to motivate the choice

15. University of Washington HCDE 518 & INDE 545 What ties these all together? Theory of Mind  “Theory of mind is the ability to attribute mental states—beliefs, intents, desires, pretending, knowledge, etc.—to oneself and others and to understand that others have beliefs, desires and intentions that are different from one's own” [Premack, D. G. & Woodruff, G. 1978]  Related to empathy

16. University of Washington HCDE 518 & INDE 545 Theory of Mind + Design Thinking  to formulate a problem, you have to view the world from several people’s perspectives  to defer judgment, you have to entertain multiple perspectives on a problem  to exploit failure, you have to assess your ideas independent from you  to externalize ideas, you have to imagine them from someone’s perspective  to keep distant from your ideas, you have to discount your perspectives on it in favor of users  to use critiques, you have to view opinions from other people’s perspectives  to articulate rationale, you have to imagine details from other people’s perspectives

17. University of Washington HCDE 518 & INDE 545 Implications  What can you do to be more empathetic?  learn to listen  expose yourself to other cultures  judge yourself more than you do others  reflect on your work as you work  The more you do these things, the better able you’ll be to solve other people’s problems

18. University of Washington HCDE 518 & INDE 545 If that’s design practice, what’s design research?

19. University of Washington HCDE 518 & INDE 545 What’s the difference between...  Bank of America designing a new mobile banking application  designs it  builds it  deploys it  maintains it  Intent is to provide a reliable, usable, desirable service (and make $$$)  A Ph.D. student designing a new mobile application  designs it  builds it  deploys it  ?  why would a Ph.D. student do this?

20. University of Washington HCDE 518 & INDE 545 To learn something about human behavior  Designs can facilitate the answering of research questions that were previously unanswerable  how do people manage their money?  what does daily knowledge of net assets do to spending behavior?  how do different visualizations of spending behavior affect anxiety about retirement?  how does a banking application that criticizes people’s spending affect spending behavior?  how do software developers manage coordinate their work relative to deadlines

21. University of Washington HCDE 518 & INDE 545 Example: ShutEye  How can we get people to think more about the things that impact their sleep?  We designed a simple mobile app that ran on the background of people’s phones  Studied system use for 4 weeks, interviewed people before and after to assess their understanding of sleep factors  Also did assessments of whether people’s sleep improved

22. University of Washington HCDE 518 & INDE 545 To learn something about designing systems  Attempting to design something that has never been built before can result in generalizable knowledge about how systems can be designed  a banking application that predicts spending in a novel way might teach us how to predict other types of behaviors  a mobile app that is has a customizable user interface may teach us how to make other kinds of application’s user interfaces customizable  a mobile app with novel security features might  result in security features applicable to other  applications

23. University of Washington HCDE 518 & INDE 545 Example: Lullaby  How can we design a system that might help correlate people’s sleep environment to their sleep quality?  My PhD student Matt Kay and I designed and built an entire application to answer this question  The application itself had to coordinate multiple sensor streams, figure out how to mitigate privacy concerns, not disrupt people’s sleep  We are finding that people are identifying aspects of their sleep they hadn’t previously realized

24. University of Washington HCDE 518 & INDE 545 Research = generalizability?  For design to be research, perhaps the activity must have implications beyond the design itself  new knowledge  new possibilities

25. University of Washington HCDE 518 & INDE 545 Research = reporting?  What if you never describe it to anyone?  There could be 1,000’s of new discoveries bout humans and technology lurking inside Google  If they never report on these, are they research?

26. University of Washington HCDE 518 & INDE 545 Research = reflecting?  What if you describe your design, but you don’t analyze it?  Google engineers could explain the design behind Google Books, but what would we know about it, other than how to recreate it?  What do we know about its merits and limitations relative to other media?

27. University of Washington HCDE 518 & INDE 545 The design in design research is secondary  The point of designing something in a research context is rarely to create an actual thing the goal is to produce knowledge about how to create similar things or to enable the investigation of questions about people and the natural world  However, designing things in research often has a the nice benefit of creating useful and/or novel things  People can actually use Lullaby or ShutEye to improve their sleep  Both of these could be turned into commercial products and make money  But it’s usually not researchers who do it

28. University of Washington HCDE 518 & INDE 545 Three ways of knowing (of many)  qualitative methods help us observe causality through rigorous, holistic, subjectivity  the observer is the instrument, designing the process of observation  quantitative methods help us test causality through rigorous, narrow, objectivity  We design objective measurements to quantify  design methods help us manipulate causality through rigorous, specific interventions  the object is a probe, changing behavior, enabling us to observe or measure change

29. University of Washington HCDE 518 & INDE 545 What would make your designs research?  Did your designs reveal new knowledge?  Did they teach you how to design similar things?  Does this knowledge generalize?  Did you report this knowledge?  Did you reflect on how this changes the knowledge we had previously?

30. University of Washington HCDE 518 & INDE 545 BREAK – 5 MINUTES

31. University of Washington HCDE 518 & INDE 545 Discussion Questions - Ben  1. Is it allowed for a design team to make up their own Heuristics? If so what are the benefits? Draw backs? (In other words ask users to look for specific issues that they anticipate or recognize)  2. When following a heuristic, people can say if it breaks it. When is the best time to ask them for advice on how to make it better, inline or at the end?  3. What other applications see such a drastic law of diminishing returns (after 5 people...) it seems like a pretty logarithmic decline.  4. How do you handle disagreements between evaluators?  5. Nielson indicates that a good evaluator – someone who easily identifies user issues – is just as likely to find a hard to find “hard to find” issues as a “poor” evaluator, why do you think this is?

32. University of Washington HCDE 518 & INDE 545 Discussion Questions - Ben  6. Of the 10 Heuristics listed by Nielsen what do you think major companies do the least, why?  7. As an engineer the notion that performing the same experiment twice and not drawing the same conclusion being a good thing (according to Forlizzzi) is difficult to swallow. Do you agree that a good HCI study should concentrate on relevance rather that validity?  8. Where do you think a traditional designer is more useful in HCI in the front end development or back end usage?  9. When performing a Heuristic evaluation how do you focus a participant away from creeping featurism and toward the design as it is. Especially in the hi fidelity proto-type stage.  10. Is it better to ask your users to categorize their comments according to Nielsons severity ratings, or based on your interpretation of the extent of the issue they highlighted to bin them yourself (as the designer)?

33. University of Washington HCDE 518 & INDE 545 Discussion Question - Sarah  Nielsen provides severity ratings for usability problems. Often these are subjective based on the evaluator. Nielsen recommends using multiple evaluators to avoid this. Can you think of a way to make this less subjective?  Are there any gaps in the model provided by Forlizzi et al?  Nielsen provides the ten usability heuristics. These were developed before touch screens. Can you think of any guidelines that need to be added in order to effectively evaluate technology on touch screens?  According to Forlizzi et al. interaction design research in HCI is changing. How will this affect the field of HCI?

34. University of Washington HCDE 518 & INDE 545 Next Class Topics  Monday, March 5 th  Trends in UCD  Discussants: Stephan & Sarah  Wednesday, March 7 th  Final project presentations & demos  Course evals  Upcoming Work  R9, A3

35. University of Washington HCDE 518 & INDE 545 GROUP PROJECT TIME