1. Lecture 2: Discovering what people can't tell you: Contextual Inquiry and Analysis Methodology
Brad Myers
/ / : Introduction to Human Computer Interaction for Technology Executives
Fall, 2010, Mini 2

2. Enrollment = 100 students!
94% Masters

3. Pick Devices for Assignments
Note about using your own company’s products
OK, if can evaluate it with users for HW #1, and redesign for the rest of the HW’s
But no sharing of projects – individual
Random order for currently enrolled & wait-listed students
Have choices from some distance and absent students, will go in order
If late to class, go to end of the line

4. Some Usability Methods
Contextual Inquiry
Contextual Analysis (Design)
Paper prototypes
Think-aloud protocols
Heuristic Evaluation
Affinity diagrams (WAAD)
Personas
Wizard of Oz
Task analysis
Cognitive Walkthrough
KLM and GOMS (CogTool)
Video prototyping
Body storming
Expert interviews
Questionnaires
Surveys
Interaction Relabeling
Log analysis
Focus groups
Card sorting
Diary studies
Improvisation
Use cases
Scenarios
Cognitive Dimensions
“Speed Dating” …

5. Contextual Inquiry and Analysis/Design
One method for organizing the development process
We teach it to our MS and BS students
Seems to be very successful
Hartson-Pyla text: Chapter 3,6
(doing things in a different order than text)
Originally described in book:
H. Beyer and K. Holtzblatt Contextual Design: Defining Customer-Centered Systems. San Francisco, CA:Morgan Kaufmann Publishers, Inc. ISBN:
Another book (doesn’t work as well):
K. Holtblatt, J. BurnsWendell, and S. Wood Rapid Contextual Design: A How-to Guide to Key Techniques for User-Centered Design. San Francisco, CA:Morgan Kaufmann Publishers, Inc.

6. User Study Methods & the different fields they come from
Questionnaires, Interviews
Social Psychology
Focus Groups
Business, marketing technique
Laboratory studies
Experimental Psychology
Think-aloud protocols
Cognitive Psychology
Participant/observer ethnographic studies
Anthropology

7. Contextual Inquiry & Analysis/Design
An evolving method
A kind of “ethnographic” or “participatory design” method
Combines aspects of other methods:
Interviewing, think-aloud protocols, participant/observer in the context of the work
Next step: Contextual Analysis (Hartson-Pyla term)
Beyer-Holtzblatt call it “Contextual Design”
Also includes diagrams (“models”) to describe results

8. “Contextual Inquiry” Interpretive field research method
Depends on conversations with users in the context of their work
“Direct observation” when possible
When not possible
cued recall of past experience, or
recreation of related experience
Used to define requirements, plans and designs.
Drives the creative process:
In original design
In considering new features or functionality

9. Why Context? Design complete work process Integration!
Fits into “fabric” of entire operations
Not just “point solutions” to specific problems
Integration!
Consistency, effectiveness, efficiency, coherent
Design from data
Not just opinions, negotiation
Not just a list of features

10. Who? Users Interviewers: “Cross-functional” team Between 6 – 20
Representative of different roles
Note: may not be people who will be doing the purchasing of the system
E.g., if for an enterprise; public kiosk
Interviewers: “Cross-functional” team
Designers
UI specialists
Product managers
Marketing
Technical people

11. Where? Design is a group activity
Shared across different groups
Useful to have a designated, long-term space for the project team
Interviews at user site

12. Key Concepts in Contextual Inquiry
Understand users' needs in their work or living environment
Partnership
Work with users as co-investigators
Interpretation
Assigning meaning to the observations

13. Context
Definition:
The interrelated conditions within which something occurs or exists
Understand work in its natural environment
Go to the user
Observe real work
Use real examples and artifacts
“Artifact”: An object created by human workmanship
Interview while she/he is working
Context exists even when not a “work” activity
Use “work” here just to mean “doing something”
Can be home, entertainment, etc.

14. Key distinctions about context
Interviews, Surveys, Focus Groups
Summary data & abstractions
What customers say
Subjective
Limited by reliability of human memory
What customers think they want
Contextual Inquiry
Ongoing experience & concrete data
What users do
Objective
Spontaneous, as it happens
What users actually need

15. Elements of User's Context: Pay Attention to all of these
User's work space
User's work
User's work intentions
User's words (language used)
Tools used
How people work together
Business goals
Organizational and cultural structure

16. Standard Contextual Inquiry: Work-based Interview
Use when:
Product or process already exists
Or a near competitor’s
User is able to complete a task while you observe
Work can be interrupted

17. Interview Recording and Note-Taking
Do record interview
Video recordings
Screen capture software with laptop microphone for user
When to take notes?
Note taking can help you pay closer attention
Notes lead to faster turn-around
Do not let it interfere with interviewing
Usually would use a second person
How to record?
What the user says – in quotes
What the user does – plain text
Your interpretation – in parentheses
Write fast!

18. Reasons for variation on the standard work-based interview
Different goals
Designing a known product
Know the competition
Addressing a new work domain
Study what replacing
Designing for a new technology
Types of tasks that make work-based inquiry impractical
Intermittent – instrument or keep logs
Uninterruptible – video and review later
Extremely long – point sample and review

19. Some Alternative Contextual Inquiry Interview Methods
For intermittent tasks
In-context cued recall
Activity logs
For uninterruptible tasks
Post-observation inquiry
For extremely long or multi-person tasks
Artifact walkthrough
New technology within current work
Future Scenario
Prototype or prior version exists
Prototype/Test drive

20. Partnership Definition:
A relationship characterized by close cooperation
Build an equitable relationship with the user
Suspend your assumptions and beliefs
Invite the user into the inquiry process

21. Why is Partnership Important?
Information is obtained through a dialog
The user is the expert.
Not a conventional interview or consultant relationship
Alternative way to view the relationship: Master/Apprentice
The user is the “master craftsman” at his/her work
You are the apprentice trying to learn

22. Establishing Partnership
Share control
Use open-ended questions that invite users to talk:
"What are you doing?"
"Is that what you expect?"
"Why are you doing...?"
Let the user lead the conversation
Listen!
Pay attention to communication that is non-verbal

23. Analysis
In the moment: Simultaneous data collection and analysis during interview
Post interview:
Using notes, tapes, and transcripts
Analysis by a group:
Integrates multiple perspectives
Creates shared vision
Creates shared focus
Builds teams
Saves time

24. Defining the Tasks
In a real Contextual Inquiry, user decides the tasks
Investigate real-world tasks, needs, context
But you still must decide the focus
What tasks you want to observe
That are relevant to your product plan
But for Assignment 1, you will have to invent some tasks

25. Test Tasks Task design is difficult part of usability testing
Representative of “real” tasks
Sufficiently realistic and compelling so users are motivated to finish
Can let users create their own tasks if relevant
Appropriate difficulty and coverage
Should last about 2 min. for expert, less than 30 min. for novice
Short enough to be finished, but not trivial
Tasks not humorous, frivolous, or offensive
Easy task first, progressively harder
But better if independent
Remember: Not asking their opinions

26. Test Script Example: Useful to have a script
Make sure say everything you want
Make sure all users get same instructions
Should read instructions out loud
Ask if users have any questions
Make sure instructions provide goals only in a general way, and doesn’t give away information
Describe the result and not the steps
Avoid product names and technical terms that appear on the web site
Don’t give away the vocabulary
Example:
“The clock should have the right time”; not: “Use the hours and minutes buttons to set the time”

27. Example of CI Video of sample session with a eCommerce site:
Issues to observe
Interview of work in progress, in “context”
Actual session of doing a task
Not an interview asking about possible tasks, etc.
Note that focusing on expert behavior & breakdowns
Questions to clarify about routine, motivations
Why do certain actions: need intent for actions
Notice problems (“breakdowns”)
Notice what happens that causes users to do something (“triggers”)
E.g. appearance of error messages, other feedback, external events (phone ringing), etc.

28. Screen shots of important points in video