1. Housekeeping Seminar Topics doodle poll on facebook page – sign up by tomorrow night (Fri) please, and I’ll make the schedule this weekend Option to propose a novel topic. One slot on the last day – I may free up another couple on other days depending on interest levels Post-mortem of 1 st MP1 Protocol testing Preparation Volunteers Those rating sheets Critiques

2. Agenda Today: Finish off contextual interviews Diary Studies Discussion of diary studies reading (assigned for Tues) Questionnaire handout (start in class – complete for Tuesday) Next Tuesday: Finish off Questionnaires, Methodology Matters

3. Contextual inquiry  Interviewees are interviewed in their context, when doing their tasks, with as little interference from the interviewer as possible.  Allows probing of “why?”  Can be real-time or record interesting actions for later discussion

4. “Typical” 4 phased approach  Traditional interview  Get an overview, establish trust, start recording  Switch to a master-apprentice relationship  Tell them what you want to observe  Make sure to establish when ok to interrupt  Observe, ask questions  Take notes  Balance need to understand with impact of interruptions  Summarization  Go over observations and your understanding with participant  Make sure that you go it right

5. Other ways of providing context  If natural observation not possible, can ask them to demonstrate specific tasks of interest  Can provide task scenarios and ask them to perform  “Think aloud” aloud protocols

6. Other ways of getting observational data  Logging  Screen recording (check out Camtasia)  Trace data

7. CSCI 4163/6610, Winter 2014

8. Advertising Diary  Did you remember to fill it out?  When did you do it?  As encountered?  In a batch?  Were the instructions clear?  What was easy/hard about it?  Do you think it captured your exposure to advertising?

9. Diary studies…  Participants collect data about events  As they happen  In the context of the event (in situ)  Can think of like a small, longitudinal questionnaire  Often used to prompt interview discussion (similar to observations in a contextual inquiry)  Can help understand rare/infrequent events

10. Data collection methods  Survey style forms  Paper (little training required, but hard to monitor)  Online entry forms  Google docs  Photographs, video  Digital/disposable camera, mobile phone  Audio recording  Voice recorder, mobile phone  Aggregating data online  Blogs (text, audio, video), Twitter  Rich and timely information

11. Type of data recorded  When  Date/time  Duration  Activity/task  What  Activity/task  feelings/mood  Context (environment/setting)

12. When is data recorded?  Randomly  In response to prompts  At specific intervals  Based on activity

13. General types of diary studies  Unstructured  Ask participants to report on everyday activities  Trying to elicit general themes  Structured  Ask participants to report on everyday activities by answering specific questions about the activity  Combination of question types  Can also be used as a form of usability tests and problem reports  Ask them to complete a task and report results, identify bugs, etc.

14. Variations based on purpose Feedback (now)Elicitation (later)  Record everything in situ  No follow up  Focus on the “what” data  Mostly structured  Can be burdensome on participants and researchers  Record aspects or triggers of an activity  Follow up with an interview to gain more information  Data captured is used to elicit the “why” and the how”  Mostly unstructured  Can be problems with participant recall

15. Cultural Probe  Diary study++  Used to not only record activities, but capture more of “felt life”  Stimulate thought as well as capture experiences  http://www.hcibook.com/e3/casestudy/cultural-probes/ - pack contained a small disposable camera and a listening glass that participants used to listen at walls and doors and write what they heard. It also contained a small solid state recorder packaged in a sleeve that said 'dream recorder'. http://www.hcibook.com/e3/casestudy/cultural-probes/  Was used by designers at the Royal College of Art, London, to study the way people see their own homes. The results were used to enable designers to get a 'feel' of the meaning of home for many people.  Experience sampling method (measure feelings, moods)  Technology Probes (prototypes, experimental artifacts)

16. Coordination Challenges  Keeping participants on track  Periodic reminders  Feedback about the level of detail in responses  Progressive incentives, surprise incentives/gifts (may or may not be allowable by BREB)  Adapting to changes  Start analysis as soon as first results arrive  May need to re-evaluate the diary format if the data being captured is not what you expected!

17. Advantages  Time efficient for researchers  Lower cost than direct observations  Can have a broad geographic distribution  Supports contextual reports over time  Natural environment  Can give rich data about contexts of use

18. Disadvantages  Participant fatigue  Missed responses (frequency data is generally unreliable – lower bound)  Study drop outs  “after the fact” reporting to catch up  Behaviour adjustment  Participants need reminders  Can be expensive for long durations  Volume of collected data can be overwhelming

19. Study protocol  Introductory letter  Consent form  Needs to motivate their participation  Very specific instructions  Can be helpful to give an example of a filled out form (take care not to prime/limit)  Diary form / collection aids

20. Forms of analysis  Track temporal patterns  Look for semantic patterns in visual data  Combine with interview analysis

21. Diary reading  When Participants Do the Capturing: The Role of Media in Diary Studies  What was their motivation?  What were their research questions?  What was their approach?  What were their findings?  What were the strengths of their study design?  What were the weaknesses?  Do you trust their findings?

22. Question writing exercise  1. Answer the questionnaire (these are BAD questions!)  2. Take a few minutes to think about what made the questions bad/hard to answer/hard to analyze  3. look at 6-17 on the questionnaire design checklist  For each checklist item, identify which questions are examples of poor design  4. On Tuesday, we will discuss this as a class