Housekeeping Seminar Topics doodle poll on facebook page – sign up by tomorrow night (Fri) please, and I’ll make the schedule this weekend Option to propose a novel topic. One slot on the last day – I may free up another couple on other days depending on interest levels Post-mortem of 1 st MP1 Protocol testing Preparation Volunteers Those rating sheets Critiques

Agenda Today: Finish off contextual interviews Diary Studies Discussion of diary studies reading (assigned for Tues) Questionnaire handout (start in class – complete for Tuesday) Next Tuesday: Finish off Questionnaires, Methodology Matters

Contextual inquiry  Interviewees are interviewed in their context, when doing their tasks, with as little interference from the interviewer as possible.  Allows probing of “why?”  Can be real-time or record interesting actions for later discussion

“Typical” 4 phased approach  Traditional interview  Get an overview, establish trust, start recording  Switch to a master-apprentice relationship  Tell them what you want to observe  Make sure to establish when ok to interrupt  Observe, ask questions  Take notes  Balance need to understand with impact of interruptions  Summarization  Go over observations and your understanding with participant  Make sure that you go it right

Other ways of providing context  If natural observation not possible, can ask them to demonstrate specific tasks of interest  Can provide task scenarios and ask them to perform  “Think aloud” aloud protocols

Other ways of getting observational data  Logging  Screen recording (check out Camtasia)  Trace data

CSCI 4163/6610, Winter 2014

Advertising Diary  Did you remember to fill it out?  When did you do it?  As encountered?  In a batch?  Were the instructions clear?  What was easy/hard about it?  Do you think it captured your exposure to advertising?

Diary studies…  Participants collect data about events  As they happen  In the context of the event (in situ)  Can think of like a small, longitudinal questionnaire  Often used to prompt interview discussion (similar to observations in a contextual inquiry)  Can help understand rare/infrequent events

Data collection methods  Survey style forms  Paper (little training required, but hard to monitor)  Online entry forms  Google docs  Photographs, video  Digital/disposable camera, mobile phone  Audio recording  Voice recorder, mobile phone  Aggregating data online  Blogs (text, audio, video), Twitter  Rich and timely information

Type of data recorded  When  Date/time  Duration  Activity/task  What  Activity/task  feelings/mood  Context (environment/setting)

When is data recorded?  Randomly  In response to prompts  At specific intervals  Based on activity

General types of diary studies  Unstructured  Ask participants to report on everyday activities  Trying to elicit general themes  Structured  Ask participants to report on everyday activities by answering specific questions about the activity  Combination of question types  Can also be used as a form of usability tests and problem reports  Ask them to complete a task and report results, identify bugs, etc.

Variations based on purpose Feedback (now)Elicitation (later)  Record everything in situ  No follow up  Focus on the “what” data  Mostly structured  Can be burdensome on participants and researchers  Record aspects or triggers of an activity  Follow up with an interview to gain more information  Data captured is used to elicit the “why” and the how”  Mostly unstructured  Can be problems with participant recall

Cultural Probe  Diary study++  Used to not only record activities, but capture more of “felt life”  Stimulate thought as well as capture experiences  - pack contained a small disposable camera and a listening glass that participants used to listen at walls and doors and write what they heard. It also contained a small solid state recorder packaged in a sleeve that said 'dream recorder'.  Was used by designers at the Royal College of Art, London, to study the way people see their own homes. The results were used to enable designers to get a 'feel' of the meaning of home for many people.  Experience sampling method (measure feelings, moods)  Technology Probes (prototypes, experimental artifacts)

Coordination Challenges  Keeping participants on track  Periodic reminders  Feedback about the level of detail in responses  Progressive incentives, surprise incentives/gifts (may or may not be allowable by BREB)  Adapting to changes  Start analysis as soon as first results arrive  May need to re-evaluate the diary format if the data being captured is not what you expected!

Advantages  Time efficient for researchers  Lower cost than direct observations  Can have a broad geographic distribution  Supports contextual reports over time  Natural environment  Can give rich data about contexts of use

Disadvantages  Participant fatigue  Missed responses (frequency data is generally unreliable – lower bound)  Study drop outs  “after the fact” reporting to catch up  Behaviour adjustment  Participants need reminders  Can be expensive for long durations  Volume of collected data can be overwhelming

Study protocol  Introductory letter  Consent form  Needs to motivate their participation  Very specific instructions  Can be helpful to give an example of a filled out form (take care not to prime/limit)  Diary form / collection aids

Forms of analysis  Track temporal patterns  Look for semantic patterns in visual data  Combine with interview analysis

Diary reading  When Participants Do the Capturing: The Role of Media in Diary Studies  What was their motivation?  What were their research questions?  What was their approach?  What were their findings?  What were the strengths of their study design?  What were the weaknesses?  Do you trust their findings?

Question writing exercise  1. Answer the questionnaire (these are BAD questions!)  2. Take a few minutes to think about what made the questions bad/hard to answer/hard to analyze  3. look at 6-17 on the questionnaire design checklist  For each checklist item, identify which questions are examples of poor design  4. On Tuesday, we will discuss this as a class