1. Observing Users Paul Bogen, LaShon Johnson, Jehoon Park

2. Overview Introduction Goals, questions and paradigms How to observe Data collection Indirect Observation Analyzing, interpreting, and presenting data

3. Introduction What is Observation? –Informing Design vs. Evaluation Aims of this Chapter –Why Observation? –How to Observe –How to use your observations –Issues with Observation

4. Goals, questions, and paradigms Why have a goal? What and when to observe? Approaches to observation –“Quick and dirty” –Usability testing –Field studies –Ethnography

5. How to observe – Controlled Role of Observer –Predetermined Location –Testing Equipment –Informed Consent Think-aloud technique

6. How to Observer – Field Studies Practitioner’s Framework –The person –The place –The thing

7. How to Observe – Field Studies Goetz and LeCompte –Who is present? –What is happening? –When does the activity occur? –Where is it happening? –Why is it happening? –How is the activity organized?

8. How to Observe – Field Studies Colin Robson –Space –Actors –Activities –Objects –Acts –Events –Goals –Feelings

9. How to Observe – Field Studies Field Checklist –State the initial goal and question clearly –Select a framework to guide your activity in the field –Decide how to record events –Be prepared to go through your records asap

10. How to Observe – Field Studies Field Checklist cont. –Separate personal opinion from what happened –Be prepared to refocus study –Think about gaining trust of those you observe –Think about how to handle sensitive issues

11. How to Observe – Field Studies Field Checklist cont 2. –Consider working as a team –Consider checking notes with the observed –Look at the situation from different perspectives

12. How to Observe - Ethnography Need of acceptance Ethnography vs. Participant Observation –Open Interpretivist –Theoretical Underpinning –Biases

13. How to Observe – Ethnography Ethnographic Checklist –Identify a problem or goal and ask questions –Just be there to observe –Collect a variety of data –Move between broad picture and specific questions –Use a holistic approach to data analysis

14. Data collection Note plus still camera Audio recording plus still camera Video Issues –Equipment –Flexibility –Completeness –Disturbance to users –Reliability –Analysis –Feedback to designers

15. Indirect Observation Diaries Interaction logging

16. Analyzing the data Kind of Data –Qualitative that is interpreted –Qualitative that is categorized Looking for incidents or patterns Categorizing Analyzing discourse –Quantitative that is collected Feeding the findings back into design