1. User Research HCDE 518 & INDE 545 Winter 2012
With credit to Jake Wobbrock, Dave Hendry, Andy Ko, Jennifer Turns, & Mark Zachry

2. Agenda Announcements Lecture & Discussion – Design Methods
Break – 5 mins
Lecture & Discussion – CI & Ethnography
Next class

3. Announcements P0, R1, R2 graded This Wednesday – no class meeting
A1 shortly
This Wednesday – no class meeting
Group work day
A2 deadline extended to next Monday
Next week is normal (I hope!)

4. Design Methods

5. Observing People What do we “see”? Opportunities for new designs
Breakdowns
Workarounds
Mismatches between what users say and do

7. Relying on what users say
Can we rely on what users say about what they want in a new design?
Very carefully
Henry Ford: “If I had asked my customers what they wanted, they would have said a faster horse."
It is better to watch what they do than to go only on what they say
Mismatches may hold keys to new designs

8. Users’ words are unreliable
People are notoriously bad at predicting what they would use or would prefer when it is only hypothetical
They can much better respond to actual, concrete things, or make comparisons
This highlights the importance of observation and of prototypes

9. Users can however… Tell you what they are doing right now
Tell you how they are feeling right now
Tell you what their goal is right now

10. IDEO Method Cards
Available from William Stout publishers ($49)

11. LEARN from the facts you gather

12. LOOK at what users really do

13. ASK users to help

14. TRY it yourself

15. Triangulation All user research techniques have their own limitations
Use multiple techniques to fully understand a design scenario
Choose techniques that account for the weaknesses of each other
Choose techniques to cover both a breadth and depth of the user experience

16. Design Activity – 10 minutes
Using the Method Cards, come up with two methods that could be useful in each of the following contexts, and two that would not be useful for the following design question. Make sure they triangulate!
How can cell phones help older adults communicate with their young grandchildren over a distance?
Cards:

17. What did you come up with?

18. A2 – Look, Learn, Ask, Try Similar to what you just did!
You’ll be given 3 design scenarios and you’ll be asked to come up with 4 methods that would be appropriate and 1 that would not
How can a new system support communication for emergency room nurses?
How can a mobile system help long-distance bicyclists to find restaurants and amenities?
How can a video game help educate kids in Grades 1-5 on healthy eating
You’ll be asked to explain your choices
Due next Monday (1/30)

19. Diana’s Discussion Questions
1.     Is it better to start with looking, asking, learning or trying?
2.     Which method(s) from the 51 cards best addresses the appearance of an artifact?
3.     Which method(s) from the 51 cards best addresses the culture of a population?
4.     What method(s) is less time consuming? In general, how much time does a designer needs to save for this phase?
5.     Methods like surveys are easy to mass distribute, whereas interviews are more difficult to obtain. How meaningful is the information from one another?

20. Diana’s Discussion Questions
6.     If design shapes human experience, why not let the user come up with the first prototype? Wouldn’t that say more about how they perceive the product?
7.     Goals like organic work flow cannot always be achieved. From the four orders of design, what should take priority?
8.     When does the designer develop the working hypothesis? (That is, what kind of research is originally done?)
9.      How does thinking of technology as systematic thinking differ to thinking of it as products?
10.  Most of the reading seem to describe ideal processes for knowing people and designing, why are they not followed?

21. Break – 5 minutes

22. P1 – Due Monday, Feb. 8th Define stakeholders
Choose and execute 3 user research techniques (make sure they complement each other)
Cannot be from the “Try” category
Define design requirements
Create personas
Re-visit design question

23. Ethnography
Observational science attempts to understand a group or individual objectively
Understand the subject of study from the outside in a way that can be explained to “anyone”
Generate “thick description” painting a vivid holistic picture
Ethnography attempts to understand a group or individual phenomenologically
Understand the subject of study as the subject of study understands himself/herself

24. Design Ethnography Quicker than traditional ethnography
Usually days, weeks, or months, not years
Sometimes called “concurrent ethnography”
The ethnography is being done at the same time that design is under way
Goal is to generate insights for informing inspiring design
Translating from raw field data to design ideas can be difficult

25. Four Ethnographic Principles

26. Natural Settings Conducted in the setting of the participant
Focus on naturally occurring, everyday talk and action
Cannot use laboratory or experimental settings to gather this type of data

27. Holism Behavior can only be understood in its larger social context
That is, holistically

28. Descriptive
Study how people actually behave, not how they ought to behave
Defer judgment
Data is not usually quantitative, but qualitative

29. Subjects' Point-of-View
See through participants' eyes in order to grasp how they interpret and act in their world
Phenomenological

30. Personal Example Studying autism therapists
Trained to be a therapist, acted as one for 6 months
Immensely helpful in gaining trust and access to users
Could easily test out prototypes

31. Contextual Inquiry
What does context refer to?

32. Key Concepts
“The users cannot describe what they really do because they are not conscious of it and do not reflect on it.”
“The core premise of Contextual Inquiry is very simple: go where the customer works, observe the customer as he or she works, and talk to the customer about the work. Do that, and you can’t help but gain a better understanding of your customer.”

33. Principles of Contextual Inquiry
Must be done in the setting of the participant
Partnership
Master/apprentice model; investigator is humble
Interpretation
Observed facts must be regarded for their design implications. Raw facts without interpretation aren't very useful
Focus
Themes that emerge during the inquiry. You can't pay attention to all facets of someone's work at all times!

34. Master/Apprentice You are the apprentice The informant is your master
What does this relationship imply?
Keen observation
Unafraid to ask questions
Eager to learn
Admire the master
Aspire to see the world as they do
Adopting the master/apprentice model during your CI will mean you don't have to pre-prepare a set of interview questions
Reduces pressure to “get it right.”

35. Interviewing in CI
Go for concrete details obtained in-context, not abstract generalities
Don't ask participants to summarize their work
Ask them specific details about real, concrete, observable things
Have them “think aloud” as they work through their tasks
Pepper them with short, easily answerable questions
Avoid high-level philosophical questions that will just cause them to “talk” instead of “do”

36. Interpretation Checking
It is good to regularly check your interpretations
“I saw you just do X. Is that because of Y?”
“I believe X. Is that correct?”
“If you had a technology that did X, would that solve the problem we just encountered?”
As long as you check your interpretations in context, participants will respond accurately
Outside of context, they may be more inclined to agree or answer in generalities rather than specifics

37. Ways to Mess Up a CI Not being inquisitive/nosy enough
If you have the impulse to ask, do it right away!
Overly disrupting the task
Questions are great, but don’t ask so many so fast that the participant stops doing their tasks.
Turning it into a regular interview
If you could have done it in a coffee shop, you didn’t do a contextual inquiry.
Failing to be discrete
Participants must feel safe, free, and anonymous.
Failing to respect your participants
Failing to observe closely and take good notes
Over-focusing on the wrong details
Slipping into abstraction
Keep it concrete, in the work, in the details.

38. Summary
The Design Cards, Design Ethnography, and CI are all ways of understanding your users’ goals and needs better
Focus on:
Understanding what they’re trying to accomplish
How they do it now
What works
What doesn’t work -> Design opportunity!
More info -> See backup slides

39. Siri’s Questions

40. Case Studies Huang & Truong Tee, Brush, & Inkpen
Questionnaire and Interviews
Focus on understanding how and why people dispose of their old mobile phones
Identify design opportunities
Tee, Brush, & Inkpen
Interviews, family trees
Understand ways that families currently communicate

41. Next Class Topics Wednesday, January 25th Monday, January 30th
No class lecture – group work time
Monday, January 30th
Personas
Discussant - Patrick
Upcoming Work
Get started on P1 user research!
A2, R4

42. Group Project time

43. backup

44. Data Recording Approaches
Notes
Notes + still camera
Notes + Audio
Notes + Audio + still camera
Video
What are the advantages and disadvantages to each of these?

45. Interviews
Purpose: Collect detailed information about tasks, activities, technologies. Understand the why behind activities, rather than the what. Suitable for relatively small number of people (5 – 30) - Shoot for ~12 to reach data saturation, though not always feasible

46. Interviews Unstructured Broad questions concerning some general area
Structured Narrow questions concerning specific area
Semi-structured Balance between broad/narrow questions
Focus groups Group discussion around a topic

47. Creating an Interview Guide
Who do you need to interview and why?
Demographic questions
Open questions
Closed questions
Activities
Sketching
Demonstrations
Reliability and validity
Be careful about leading questions

48. Running an Interview 0. Recruitment Introduction Warm up session
Main session
Cool-off period
Closing session

49. Example – Huang & Truong

50. Questionnaires/Surveys
Purpose: Deepen understanding by collecting information from a broad range of people
Suitable for large number of people
20 – 1,000+

51. Questionnaires Be clear on the goal Keep it short
Open and closed questions
Rating scales (e.g. Likert)
Be sure to pilot your questionnaire and expect to iterate 3-4 times
Online or on paper?

52. Example – Huang & Truong

53. Observations in the Field
Helps you understand people’s context, task, activities, goals
What people do is different than what people say they do
People are notoriously poor and describing their work

54. What to Pay Attention To
Key features
Questions
Space
Actors
Activities
Objects / workarounds
Acts
Events / triggers
Time
Goals
Feelings
How is the physical space adapted to the job?
What are the key constraints on the job?
Where are strategic decisions made?
Where are tactical decisions made?

55. Examples What might you observe about each case?
Is observation appropriate?
A nurse as she treats patients in a hospital
A teacher while he is teaching a classroom full of kindergartners
A team of emergency medical technicians working on a case
A writer working on a book

56. Laboratory Studies
Useful for studying and recording the details of how people perform:
Goals
Tasks
Action sequences
Disadvantages: Not the user’s natural environment, missing context
Better for doing tests of prototypes where detail is important

57. Lab Studies

58. Indirect Observations
Diary studies
Experience Sampling studies
Interaction logs

59. Diary Studies
Have users carry a diary with them to answer specific questions about activities throughout the day when certain activities occur

60. Example PAL diary study Fits in one’s pocket
Record when PAL is desirable

61. Experience Sampling Studies
Have users carry around a device that has them answer questions at given intervals
Cell phone, PDA, SMS
Good for getting in-the-moment assessments
Example: page user once every 3 hours and ask them to fill out a short survey on their current activity and rate sleepiness level on a scale from 1 to 7

62. Example – Track Your Happiness
Go to TrackYourHappiness.org and sign up

63. Diary Study vs. Experience Sampling
Both can collect similar types of information
Diary is less intrusive
ES can be more reliable at getting regular data
User is less likely to forget
Another alternative – context-based sampling
MyExperience Toolkit -

64. Interaction Logs / Usage Logs
Use software to automatically log interactions with a system
e.g., number of clicks, time spent on task, etc.
e.g., pages visited within a site: Google Analytics
Can also be used on other platforms using interesting sensors to sense context of use
Example: Logging steps with a pedometer

65. Example
Using Bluetooth tags to track people’s distance from their cell phones
Intent: Inform design of “always on” ubiquitous computing applications

66. Case Studies Huang & Truong Tee, Brush, & Inkpen
Questionnaire and Interviews
Focus on understanding how and why people dispose of their old mobile phones
Identify design opportunities
Tee, Brush, & Inkpen
Interviews, family trees
Understand ways that families currently communicate

67. Questions What were your overall thoughts?
What did you like about the approach?
What would you have changed about the approach?
Why do you think they chose the approach they did?