1. CSCD 487/587 Human Computer Interface Winter 2013 Lecture 14 Observation and Ethnography

2. Outline There are benefits and challenges of different types of observations How to observe as an onlooker, a participant, and as an ethnographer Observation can be Direct – You observe what users do or Indirect – You use diary studies and logging

3. What and When to Observe Observation is usable at all stages during product development Goals and questions determine the paradigms and techniques used Observers can be – Onlookers – Participants – Ethnographers Degree of immersion that evaluators adopt varies across a broad outsider-insider spectrum

4. Types of Observation Observation Controlled Environment (e.g. Lab) Field Environment (e.g. Natural) Outsider looking on “Quick and Dirty” or In usability testing “Quick and Dirty” or In field testing Insider N.A. Participant observation (e.g. in ethnography)

5. Approaches to Observation “Quick and Dirty” Observation – It can occur anywhere, anytime – Least formality involved Observation in Usability Testing – Video and interaction logs capture all user operations – One-way mirrors or remote TV screen – Data is used to analyze what users are doing and provide insight into users’ reactions – Can augment other techniques – Users can't always explain what they do Thus, limits of surveys and interviews !!! – Direct observation helps fill in details

6. Approaches to Observation Observation in Field Studies – Observer may be Passive observer, Participant observer or Ethnographer – Goal is to cause as little disruption as possible

7. Approaches to Observation Passive or Outside Observer Example: An observer in class of boys and girls whose primary job to keep track of how much time a particular technology is used by each gender Participant Observer Participates in social conventions of a group, combines participation with maintenance of professional distance for unbiased observation Ethnography Becomes one of the group, takes weeks, months or longer Inside information of community activity is obtained

8. How to Observe In Controlled Environment Practical Issues Need to Plan Advance – Decide where users are located so that equipment can be set up in advance – Decide about modes of data capture e.g. video, interaction logs – Equipment testing, audio and video – Get legal user consent – Prepare a script to guide users through set of questions – It is important to make users feel comfortable – Problem with this approach Observers do not know what users are thinking

9. Controlled Environment Think-Aloud Technique User is trying to perform certain task Observer wonders, what is going on, what is he thinking, why did he do that, Externalize thought process

10. Controlled Environment Think Aloud Technique

11. Checklist of things to Plan Before Going into Field Select a framework to guide activities in field Decide how to record events Think about how to gain acceptance and trust of users under observation Be prepared to refine and refocus study for same or new user groups Think about how to handle sensitive issues Consider checking notes with a participant and plan to look at situation from a different perspective

12. Frameworks for In the field There are Frameworks to guide observation, provides structure Simple – The person. Who? – The place. Where? – The thing. What? The Goetz and LeCompte (1984) framework: – Who is present? – What is happening? – When does the activity occur? – Where is it happening? – Why is it happening? – How is the activity organized?

13. Frameworks for In the field The Robinson (1993) framework –Space. What is the physical space like? –Actors. Who is involved? –Activities. What are they doing? –Objects. What objects are present? –Acts. What are individuals doing? –Events. What kind of event is it? –Goals. What do they to accomplish? –Feelings. What is the mood of the group and of individuals?

14. Data Collection Techniques Notes plus still camera – The least technical way of collecting data – Transcription the first step in data analysis – Photographs, sketches, etc. can be easily collected Audio recording plus still camera + – Less intrusive than video – More flexible and mobile – One drawback is transcribing the data which can be onerous if many hours have to be transcribed

15. Data Collection Techniques Video – Advantage of both audio and video data, but more intrusive – Attention becomes focused on what is seen through lens – Analysis of video can be long and time consuming – Sound may get muffled when recording in noisy conditions

16. Comparison Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley

17. Comparison (contd.) Table source: Preece, Rogers, Sharp: Interaction Design: Beyond Human-Computer Interaction. Wiley

18. Indirect Observation Diaries Diaries provide record of What users did, When they did it and What they thought about their interactions with technology Useful when users are scattered and unreachable Inexpensive and requires no special equipment or expertise They rely on participants being reliable and remembering to complete them

19. Indirect Observation Interaction Logging - Includes analysing data from key presses, mouse movements etc. - Usually synchronized with video and audio logs - Time stamped to calculate how much time a user spends on a particular task E.g. Explicit counters that record visits to a website If you want to find out if adding a bulletin board increases number of visits, comparison of traffic before and after is useful - Unobtrusive and large volumes of data can be collected

20. Ethnography

21. Ethnography’s is notion of participant observation –Idea that you learn about other people's cultural practices by –Going there, being there, and by doing it with them –Most traditional anthropologists who would consider themselves to be ethnographers have spent years living in other cultures with people, and not just watching what they do, but actually doing it too 21 Ethnography

22. Ethnography is an analytic perspective that is associated with doing interactive fieldwork - Term was coined or invented by anthropologist Bronislaw Malinowski circa 1922 to describe his approach towards understanding other people’s cultures through fieldwork - Term “ethnography” is often used to describe fieldwork in systems design

23. Design Scenario Task for our Ethnographic study: – Design an email system for the 21st Century networked home – We need to first look at how mail is currently delivered in the home

24. An example Handling mail in the home - Letterbox, pigeon hole, porch, front door - In other words a known in common delivery / collection point … … … - Where does mail arrive? Known in common collection point Contingent But known in common by household members

25. What happens next? Mail arrives then what? - Someone collects it. Who? - But can anybody open it? No - So it gets put some place where members can see that new mail has arrived - A known in common place where members can sort through it to see if any mail has arrived for them Known in common sorting point Just about anyone (even the dog :-)

26. 26www.id- book.com Who is it for? Specifically, how do we determine who is entitled to open particular mail items? - Name and …. - Logos, handwriting, postmarks, senders’ address, etc. All articulate entitlement to open (not necessarily name) Sorting the mail: members can see at-a-glance who it is for - Visible features of envelopes enable members to determine at-a-glance who an item is for and what to do with it junk it, leave it, open it

27. What if it is for someone else? What happens then? Do we leave the mail at the sorting point? - Members usually put mail for others in places that are relevant to them Relevant recipient point Places the recipient usually sits –At the kitchen table, on their favourite chair, on the arm of the sofa, or nest of tables next to it, etc. Or places where the recipient cannot miss it (e.g., outside the bedroom door of your teenage son) Mail items that are not for but may be of interest to other members are also placed in this way (postcards, letters, leaflets, etc.)

28. What do we do with it? Lots of mail requires that we do something with it (e.g., pay a bill). Members place mail to reflect its action status. - Mail requiring immediate action is placed such that members can see that at-a-glance By the mobile phone charger, or by the car keys, or in porch next to the front door, etc., if it’s has to be taken out of the home and dealt with Or by a computer if its to be dealt with online Immediate action point

29. What if a response isn’t needed? Not all mail requiring action requires that we respond to it – e.g., bank statements, insurance certificates, mortgage payment records, etc. - We may still need to take action on it though (e.g., checking and archiving). - Mail where action may need to be taken is placed to show that action is pending. Pending action point

30. 30www.id- book.com What if a response will wait? Other mail items require action but not until a later date – e.g., appointments, invitations, promotional offers, etc. - They are placed to display their relevance to up and coming events - e.g., on notice boards next to calendars, shopping lists, takeaway menus, etc. Pending event points

31. 31www.id- book.com What if it’s Special? Not all of the mail we receive is utilitarian bills, bank statements, appointments, We also receive “special” mail items: birthday cards, thank you cards, congratulations, etc. - Special items are placed to display their essentially social character On window sills, mantelpieces, cupboards, dressers, etc. Social display points

32. Naturally Accountable Organization The organization of mail handling in the home, observable and reportable by members 1. Collecting mail from a known in common delivery / collection point 2. Placing mail at a known in common sorting point 3. Sorting mail based on household member’s entitlement to do so, which is read off the visible features of envelopes and not just the name 4. Placing sorted mail items in places relevant to their recipients 5. Placing mail to indicate its action status Immediate action Pending action Pending event 6. Placing mail to display its special social status

33. 33www.id- book.com Design scenario revisited What does the naturally accountable organization of mail handling suggest about the design of email for the 21st Century networked home? – How might the field study “inform” design? What are the main organizational features or “findings” of the study? Thought experiment part two: who factored the distribution of email across, and its display at, different locations into their design sketches? - Can you see the value of looking at relevant activities in the real world? - Mail is distributed around the home - Mail is distributed through its physical placement and display - The display of mail performs various functions: At-a-glance it announces that new mail has arrived, that mail has arrived for you, that immediate action needs taking, and so on

34. Summary Many techniques for collecting user data Observation, survey, questionnaires, think aloud and ethnographic studies Need to capture user's interactions with the object of study Time, expense will certainly affect choice of method And, ultimately what you need to learn from the users

35. 35 End Usability testing with Earth based interface See Assignments page for short, fun assignment