2. Diary studies Rikard Harr November 2010

3. © Rikard Harr 20103 Outline The Diary study: benefits, challenges and alternatives The papers: aims and use of diary study –Short summaries Group task

4. © Rikard Harr 20104 The Diary study Benefits An approach for studying people on the move and in situ (all) A method for understanding participant behavior that minimizes the effect of observers (Carter and Mankoff, 2005) Demands less resources than alternative methods (Carter and Mankoff 2005) Helps create empirical evidence Helps develop trust, a suitable first method (Hess and Wulf, 2009)

5. © Rikard Harr 20105 Alternatives for understanding everyday human behavior Observations Ethnographic approaches, video recorded Contextual inquiries Survey studies Experience Sampling Method (ESM) Probes

6. © Rikard Harr 20106 Challenges How to find a balance between feedback studies and media elicitation studies –People answer/record questions/things as things happen –People record events and are interviewed afterwards How to gather user entries in feedback studies –Dial to a voice mail system –Create audio recordings –Creating written entries Which media to use in media elicitation studies –Audio –Photo –Tangible objects

7. © Rikard Harr 20107 The papers and Diary studies Czerwinski et al. (2004) make use of it for characterizing how people interleave multiple tasks amidst interruptions Sohn et al. (2008) make use of it for understanding mobile information needs and how they are addressed Hess and Wulf (2009) structures it and add camera and pen (?) Carter and Mankoff (2005) explores the effect of using different media to capture events and how this affect the diary study method

8. © Rikard Harr 20108 Applying the method: Czerwinski Aim, to characterize how people interleave multiple tasks amidst interruptions Baseline survey Used diaries during a week Also captured users descriptions of their work 11 participants were used (3 female) All reported on participation in at least 3 projects Age from 25 to 50 A spreadsheet for each day was created with columns: Time of task start, difficulty switching to the task, what documents were included in the task, what was forgotten if anything, comments, and number of interruptions experienced A code scheme had been developed Two experimenters coded the first day diaries

9. © Rikard Harr 20109 Applying the method: Hess and Wulf Aim, to evaluate a structured media diary for capturing participants’ social video consumption Date, Context, Event, Persons, Comment, Picture, Sticker Recruited participants via an article in the newspaper 9 households, 17 participants Short introduction before handing out ”the boxes” Age, between 10 and 45 Range of study, 2-4 weeks Short semi-structured interview (10 minutes) Diaries were accepted very well 17 participants made 189 diary entries Strong differences in the level of detail in diaries Stickers were used in 15 cases Got photos from 6 out of 9 (1-6)

10. © Rikard Harr 201010 Applying the method: Carter and Mankoff Aim: seek to understand how context information and episodic memory prompts captured by participants vary with media, in what way different media supports recall and how different media affect the diary study process Wants to improve the technique, based on three studies –Observed and interviewed researchers in two studies –Conducted their own study Results Audio elicitation suffers from recognition problems, but encourage more discrete event capturing Tangible objects unravels users attitudes and beliefs better, but are not good for remembering events Photos are easiest to capture and recognize For studies in which detail is important audio/photo is preferable

11. © Rikard Harr 201011 Applying the method: Carter and Mankoff Location information does not add much for recalling an episode People are more keen on providing feedback answers if there are fewer events To support recall it is good to tag a captured event with a brief annotation Researchers should be able to review captured data as well as annotations before the interview Automatic time-stamping is important in all cases

12. © Rikard Harr 201012 Group task You want to study the disruptive effects of various forms of interruptions for information workers. This in order to provide implications for design of future ICT You have no budget but some time over You think that factors such as relation, location, situation matter How would you designs such a diary study and why? Motivate your decisions and prepare to present it If you absolutely want to apply another approach or method, do so and prepare to motivate why If necessary make own assumptions about the object of study

13. © Rikard Harr 201013 Applying the method: Sohn et al. Aim: To understand mobile information needs and how they are addressed Two-week diary study 20 participants (10 – 10), aged 19-58 Recruited through an online mail list and flyers Focused on type of information needs, methods to adress those needs and contextual factors that prompted needs Kept diaries when mobile Conducted introductory, mid-study and final interview with all

14. © Rikard Harr 201014 Applying the method: Sohn et al. Provided several examples of what to record Used a text messaging scheme Messages were sent to an email address which posted them on a website Text messages should be composed to support answering a set of questions at the end of the day Researchers sent five reminding messages every day At the end of the day participants logged in and answered six questions about their entries Applied a reward system 421 diary entries In interviews clarifications of unclear entries were made