1. The Logbook Probe: Facilitating auto-thick description for evaluation Workshop on Innovative Approaches for Evaluating Affective Systems Swedish Institute for Computer Science, Kista, Sweden 9 January 2006 Joseph ‘Jofish’ Kaye Culturally Embedded Computing Cornell Information Science jofish@cornell.edu

2. Unpacking an overpacked title The Logbook Probe: Facilitating auto-thick description for evaluation

3. Starting points Gaver et. al: Cultural Probes Geertz: Thick description Dourish, Wright & McCarthy, etc: experience-based HCI Iterative design: Design, Build, Evaluate..

4. Thick Description Winking isn’t just something your eye does It’s culturally embedded It’s only the thick description of the context and culture that lets us understand the role of the wink in sharing a conspiracy, or even parodying another sharing conspiracy. Geertz, C. (1973) Interpretation of Cultures Ch. 1

5. Cultural Probes Gives context around a situation Originally for inspiring the design part of the design  build  evaluate  iterative design cycle Repurposed here for inspiring the evaluation part of the cycle Gaver et. al. Cultural Probes, interactions 6(1) 1999

6. Experience-based HCI Excellent theoretical foundations –Dourish (2001) Where the Action Is –McCarthy & Wright (2004) Technology as Experience –etc. Significant work on designs in CHI, DIS, etc Lack of work on evaluation for this domain as compared to traditional HCI (GOMS!)

7. Object for evaluation: VIO VIO (Virtual Intimate Object): Software device represented as a small red circle in the taskbar of Window’s screen When circle is clicked, partner’s circle glows bright red, then fades over time. http://io.infosci.cornell.edu Kaye, Levitt, Nevins, Golden & Schmidt. Communicating Intimacy One Bit at a Time. Proceedings of CHI 2005, ACM Press.

8. Two studies to date I: 10 couples, 5 using VIO, 5 using hardware version. 1 week, paper logbook probes. II: 80+ initially, pre- and post- surveys, two weeks of Internet based logbook probes, four groups –VIO –VIO + daily survey –daily survey –only pre- and –post surveys Currently being analyzed.

9. Principles To get the subjects to give us a rich description of their experience To encourage reflection by the subjects (and researchers) to get more insight into the situation To defamiliarize subjects (and researchers) with their assumptions to get rich, novel descriptions

10. Three topics Questions/tasks about the technology being evaluated. Questions/tasks about the situation. Questions/tasks about instrument being used to gather information

11. And three kinds of questions (for organization, not canonical) Context: Where? When? Take a photo of… How long? How often? Metaphor: What TV show? What song? What colour? What metaphor? Valence: What’s the best? What’s the worse? What’s your favorite? What should we change? What shouldn’t we change? Draw the best, the worst.

12. Design chart ContextMetaphorValence Technology (VIO) Situation (Long distance relationship) Instrument (Probe / Survey)

13. Example questions What did you hate the most about using the VIO? Draw a picture of your ideal intimate object What colour is your relationship? Why? What TV show most represents your relationship? What song most represents your relationship? What metaphor would you use to explain VIO to people?

14. A final note: getting beyond usability Evaluating in context requires working in context VIO becomes ready-at-hand, not present-at-hand, and so evaluation must be in context… … which makes demonstrating this, in this context, at this workshop, hard! Must have significant, continuous use, otherwise most of what you get is usability issues – see Mobile Probes paper Hulkko et. al. Mobile Probes Proc. NordCHI ‘04

15. Thank you VIO software: http://io.infosci.cornell.eduhttp://io.infosci.cornell.edu Many thanks to Kia Hook, Jarmo Laaksolahti, Kirsten Boehner, Petra Sundstrom, Phoebe Sengers, Bill Gaver and the Culturally Embedded Computing Group Dots for use on Swedish names: ……..…