1. Ethnography in Designing Socio- Technical Environments Wednesday, March 15, 2006 Stefan Carmien Melissa Dawe Anja Kintsch

2. Computer Scientists and People Our formal training as technologists and designers often leaves gaps in how to do research in the socio side of socio-technical environments.. ? ?

3. What we want to share Our experiences in using ethnography as a tool to understand the context and the stakeholders in our work designing systems for persons with cognitive disabilities and support communities.

4. Ethnographic Methods What are they? –Who uses them & why What are their attributes What are the skills of an ethnographer? What artifacts do they produce?

5. Qualitative Ethnography We are only focusing on Qualitative aspects of ethnography, why? –Participant observation –Fieldnotes –Coding –Hypothesis generation –Semi-structured interviews Inspired by Grounded theory –Allow topics to emerge from raw data

6. Ethnography and L 3 D How we can use it Why we must use it

7. Three Projects Stefan - MAPS Melissa – Mobile Communication Study Anja - Smart Care

8. Stefan MAPS system research focus –How person with cognitive disabilities live & learn –How new AT is used Roles: –Person with cognitive disabilities –Caregivers Parent Job coach Transition team Group home staff –Study in pairs

9. Stefan - Methods –Semi- structured interviews At end of usability studies Exit interviews at end of MAPS introduction –Participant observation Several days just hanging out –Activities of daily life (ADL) Introduce MAPS Caregivers making scripts Three scripts –Controlled environment (home) –Less controlled environment –Open environment (out of home)

10. Stefan Methods (cont’d) –Each participant observation session was documented Fieldnotes (little note pad) Digital recorder –Post Observation write-up Transfer Fieldnotes to log Roughly transcribe recording to log –One hour observation = 1.7 hours entry in log –Each Dyad’s log was between 20 and 30 pages Coding & pattern extraction Illustrative quotations from recordings Each finished dyad informed the approach of the study of the remaining ones

11. Stefan - How I have used the data Experiencing the life of my population (both parts of the dyad) –Daily variations in ability –Details of what works and doesn't work in scripts (Voice, size of steps, internal scripts) –‘Battle of time’ of caregivers Design with Scenarios (usability & editor features) –Caregivers implicit requests –Caregivers hopes, expectations, & fears Left to do: –Code dyad studies –Re-code Melissa's interviews –Extract abstractions from coding –Use quotes in dissertation

12. Melissa: Mobile Communication Study Research Focus 1) How can technology support the mobile communication needs of young adults with moderate cognitive disabilities? 2) How can caregivers and individuals with cognitive disabilities authentically participate in the design process?

13. Melissa: Mobile Communication Study Methods Phase 1: Semi-structured interviews with families –Recruiting participants –Developing an interview guide –Conducting interviews in peoples’ homes –Coding –Qualitative analysis –Maintaining a relationship with participants

14. Melissa: Mobile Communication Study Methods (cont’d) Phase 2: Interviews and observations focused on mobile communication Phase 3: Technology “probe” study –Work with families (participatory design) to design UI for their child –8 week field trial to study co-evolution of users and technology –Technology-as-ethnographer through detailed logging –Design-in-use: requirements emerge through ‘doing’

15. Melissa: Mobile Communication Study Participatory Design Activity

16. Melissa: Mobile Communication Study working with AbleLink AbleLink “Pocket Ace” software is a cell phone application designed for individuals with cognitive disabilities In exchange for use of their software, I will provide usability feedback and feature requirements I have full access to source code, dev support

17. Anja: Observations on observing and interviewing Imagine! “Fact-finding investigation of factors related to caregiver activities, quality of life, and use of personal technologies among individuals with cognitive disabilities who are residents in group homes and community settings.” Inform the development of context-aware technologies. Face-to-face semi-structured interviews and in situ observations Part of RERC

18. Goals of the Questionnaire Made sure our questions reflected our research questions. Interview procedures and questions intended to collect quantitative as well as qualitative data Made questions similar for all respondents: managers, group home caregivers, parents, and clients.

19. Experiences with the Questionnaire Caregivers interpreted the questions differently than we intended. –We were looking for specific answers so we could say x out of y say z. - that didn’t happen. “Please describe a situation where a client who you know well can “almost” do something that’s important to them? …what would be needed to allow them to do the task independently?” - no answer involved A.T.

20. Timing challenges with the Questionnaire Once we settled on the questionnaire we were bound to it because we had to get 58 interviews completed and we were behind schedule.

21. Interviewing Caregivers Different group home caregivers responded to my presence differently: –Guardedness - are you a spy from management? –Defensiveness - I am good at my job! –You are here to fix things - let me tell you the problems –Open and confident –For some it was enjoyable for others it felt like a requirement Was not capture in any formal method

22. Observing when people with cognitive disabilities are present We planned on passive observation, but I ended up being participatory observer –Observer effect compounded: Don’t respect the boundaries of observer and observed - would come up and interact with me.

23. How to do research with people with cognitive disabilities - Potential Problems? Tangential thinking and lack of familiarity with the context Caregivers answering or interpreting too much When working with clients with limited speech. How do you offer choices without influencing the answer? Foils?

24. What are your most favorite things to do each week? Potentially correct choices as offered by Imagine! Potentially correct choices of daily activities but are not Imagine! classes or necessarily considered selectable events Foils: are offered by Imagine! but client not participating in these classes. Incorrect and absurd foils

25. What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise Stefan: –Understanding caregivers & relationships –Getting young adults ability & needs Vary over time Typical teenage issues uneven map of abilities (unexpected faculties) –Scenario based design

26. What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise Melissa: –Understand that for families, assistive technology includes kitchen timers, memo recorders, Yahoo! games, etc. –Learn the importance of mobile communication in supporting independence, social- connectedness, sense of safety for families –A design process where families guide the design of their own technology

27. What We Got Using Ethnographic Methods That We Could Not Have Gotten Otherwise Anja: –Compare what management sees as important issues verses what caregivers on the ground see as important –See the extreme limits of many clients in the group home environment - maybe it isn’t the best place to work on independence –See that group homes are like large families

28. Our goal We hope to spark a discussion with the larger L 3 D community that have used or are interested in using these methods.

29. Resources- Classes Classes: –ANTH 7300 -- Research methods in cultural anthropology –COMM 6030/001 -- Interpretive Research Methods –SOCY 5031/001 -- Research Design

30. Resources- Tools Digital recorders (Olympus) Transcribing software - Olympus DSS player (adaptive speed control) Coding tools: –HyperResearch – NVIVO

31. Resources- Texts Texts: –Analyzing and Interpreting Ethnographic Data, Lecompte –Analyzing Social Settings, Lofland & Lofland –Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, Strauss & Corbin –Contemporary Field Research, Emerson –Qualitative Research Methods, Berg –Research Methods in Anthropology, Bernard –Writing Ethnographic Fieldnotes, Emerson, Fretz, Shaw

32. Resources- People Local Expertise: –Leysia Palen –Tammy Sumner –Sophia Liu –Heinrich Schwartz (Comm Dept.) –Kathleen Tierney (Sociology Dept.)

33. What are they? Ethnography is "the study of people in their own environment". So it's a collection of methods that are intended to understand something about people, and involve some sort of fieldwork in their environment. Different methods are appropriate based on the setting, and the research questions. Who uses them and why? Historically, first were anthropologists. The classic method was participant observation in an extended study, often studying foreign cultures (think Margaret Mead living in Samoa). Now sociologists,communication studies researchers, HCI, even marketing departments, etc.,a wide array of researchers interested in some aspect of human behavior in a natural setting. What are the attributes?- -Studying people, some aspect of human behavior (including social behavior) - In the field -- it involves studying people in their environment- -Often involve data collection that is analyzed qualitatively, but also conducive to quantitative analysis (or some hybrid)- -Often used for exploratory research or early phases, when research questions are being defined- -Can be used in complement with other research methods (e.g. formal surveys, etc.) in "data triangulation" (gaining a more complete view by approaching the problem from different perspectives)- -Traditionally assumed the researcher did not alter the people or environment under study, but now there are types of ethnography where this is definitely a desired outcome (action research, design research like we do!) -What are the skills of an ethnographer?- -Depends on techniques used (interviews, observations, extended fieldwork, etc.), but in general: study design- doing the fieldwork- qualitative analysis (usually coding and analyzing large amounts of data)- writing (presentation of ethnographic research is usually texts, including articles, books, trend in 'novel-like' writing style) -What artifacts does it produce?- I kind of answered this above, usually texts, often books. The main 'artifact' in traditional ethno is a theory or theories about human behavior.- HCI has the unique quality of developing tools through ethnographic methods. Answers to the questions on slide 4