1. © deSouza, Dittrich, Sharp Practical Exercise Reflection What impressions did you get? Where did these impressions come from? How did you find this exercise? What (initial) conclusions might you draw from this data?

2. © deSouza, Dittrich, Sharp Doing Ethnography

3. © deSouza, Dittrich, Sharp Key principles Everything is strange –Question why? –Stop and reflect Members’ point of view –Developer themselves – why do that? No a priori expectations –Non judgmental –Data analysis and changing plans continuous BUT know your focus

4. © deSouza, Dittrich, Sharp 4 Duration of an Ethnographic study A traditional ethnographic study (in anthropology) has usually a 1-year duration (Nardi;1997:p363) for 2 reasons: –In primitive cultures, one needs to learn the language, adapt to life conditions (health, hygiene, etc). The researcher can even get sick! –In the academic system, one year is enough so that the student can graduate at some point In SE/CSCW/HCI, the focus is on work practices –1 year is not necessary; –6 weeks is enough to get good results, sometimes even less than that. –It depends on the context and research question.

5. © deSouza, Dittrich, Sharp Before the study Recruiting participants –Access –Building trust Ethical issues –Relationship between observer and observed –Recognised ethical code –Participants’ right to withdraw at any point –Informed consent - ? –Data protection and anonymity Complementary methods –Observation –Interviews

6. © deSouza, Dittrich, Sharp Observation

7. © deSouza, Dittrich, Sharp A matter of legitimacy One of the most important data collection tools of an ethnographer. Originally, it was almost a requirement for ethnographic research. It provides legitimacy: –“To say the Yanamami do this and the Borroro do that, and to be taken seriously, you have to have been there, seen them, and if not done it and brought back the T-shirt, at least captured and recorded their lives (these days on videotape). Fieldwork became de rigeur for ethnography and every practising anthropologist worth his or her salt had “his” (or “her”) tribe. This, then, is a cardinal thing to note about ethnography. Its practice is a particular form of legitimation. Ethnographers “know” in ways others don’t and can’t. And what they know derives in part from personal experience.” (Genevieve Bell, May 2004) As another data collection method it is helpful to triangulate the data.

8. © deSouza, Dittrich, Sharp Types According to the researcher engagement: –Participant: the researcher acts as the informants: writes code, attends meetings, discusses solutions. Example: Sharp et. al. (2004) –Non-Participant: the researcher only observes the informants. He is “fly in the wall”. Example: De Souza et. al. (2004); In any case, the researcher must document as much information as possible in his/her field notes;

9. © deSouza, Dittrich, Sharp Field Notes A description of events, people, interactions, tool usage, things listened, heard, experiences, impressions; Be as detailed as possible, i.e., write down the higher number of details possible; Separate observations and quotes from the informants from impressions and comments from the researcher (colours, symbols…); A private document that can only be shared within the research team.

10. © deSouza, Dittrich, Sharp Practical Tips Write your fieldnotes during the data collection [in the field] if at all possible; In the beginning of your study, write down as much as possible to the level of detail you need: –Meetings, interactions, tool usage, phone calls, process adherence, etc. Everything or anything can be relevant at this point. Often, at this point you do not know what is or is not important to your research Later, as your research progresses, field notes can be refined to focus on the “interesting” aspects only.

11. © deSouza, Dittrich, Sharp It is important that the ethnographer is introduced to the informants by a respected member of the group. In the beginning, there is a natural “resistance” to the ethnographer, but this will cease as time progresses; Often, ethnographers have “key informants”, i.e., experienced informants that introduce the ethnographer, facilitate his access, answer questions, provide explanations, and so on. –Finding and cultivating them is important!

12. © deSouza, Dittrich, Sharp Structuring frameworks to guide observation - The person. Who? - The place. Where? - The thing. What? The Goetz and LeCompte (1984) framework: - Who is present? - What is their role? - What is happening? - When does the activity occur? - Where is it happening? - Why is it happening? - How is the activity organized?

13. © deSouza, Dittrich, Sharp When should I stop observing? No obvious ending. Schedules often dictate when your study ends. Otherwise, stop when you stop learning new things. –when you start to see similar patterns of behavior being repeated, or –when you have listened to all the main stakeholder groups and understand their perspectives.

14. © deSouza, Dittrich, Sharp What to collect? Or the level of detail to collect? The researcher needs to decide the level of detail needed (Randall & Rouncefield 2004). –Gestures, eye gazes, movements as done by Heath & Luff (1992) in their study of subway control rooms; –Time spent in each activity performed by knowledge workers (Gonzales and Mark, 2004) and software developers (Goncalves, de Souza et al, 2009) This level of detail depends on the research questions. Every researcher needs to define what is needed.

15. © deSouza, Dittrich, Sharp What data to collect? (1) The following illustrative list (Crabtree, 2003, p. 53): Activity or job descriptions. Rules and procedures (etc.) said to govern particular activities. Descriptions of activities observed. Recordings of the talk taking place between parties involved in observed activities. Informal interviews with participants explaining the detail of observed activities. Diagrams of the physical layout, including the position of artefacts.

16. © deSouza, Dittrich, Sharp What data to collect? (2) Photographs of artifacts (documents, diagrams, forms, computers, etc.) used in the course of observed activities. Videos of artifacts as used in the course of observed activities. Descriptions of artifacts used in the course of observed activities. Workflow diagrams showing the sequential order of tasks involved in observed activities. Process maps showing connections between activities.

17. © deSouza, Dittrich, Sharp Data recording Notes, audio, video, photographs Notes plus photographs Audio plus photographs Video

18. © deSouza, Dittrich, Sharp Comparison of the three main approaches to data recording (1) CriterionNotes plus cameraAudio plus cameraVideo EquipmentPaper, pencil, and still camera. Handheld recorder with a good microphone. Headset useful for easy transcription. Video camera – handheld or fixed. Editing, mixing, and analysis equipment needed. Flexibility of use Very flexible. Unobtrusive. Flexible. Relatively unobtrusive. Needs positioning carefully to capture relevant activity. Obtrusive. Completeness of data Only get what note taker thinks is important and can record in the time available. Visual data is missing. Notes, photographs, and sketches can augment recording but need to be coordinated with audio recording for analysis. Collects thorough and detailed data, especially if more than one camera is used, but video material needs to be coordinated for analysis.

19. © deSouza, Dittrich, Sharp Comparison of the three main approaches to data recording (2) CriterionNotes plus cameraAudio plus cameraVideo Disturbance to users Very low.Low, but microphone needs to be positioned. Medium. Camera can be difficult to ignore. Preliminary field studies and piloting with participants can help to lessen the impact. Reliability of data May be low. Relies on humans making a good record and knowing what to record. High but external noise, e.g. fans in computers, can muffle what is said. Captures detail of relevant activities, provided camera is positioned appropriately. AnalysisTranscription straightforward. Rich descriptions. Transcribing data can be onerous or a useful first step in data analysis. Critical discussions can be identified. Transcription needed for detailed analysis. Can revisit permanent original record. Critical incidents can be identified and tagged. Software tools for detailed analysis. Can revisit permanent original record.

20. © deSouza, Dittrich, Sharp Ethnographic Interviews

21. © deSouza, Dittrich, Sharp Interviews An approach for data collection used by different research methods, including ethnographic, qualitative and even quantitative; There are different types of interviews and each one of these types is more often associated with a particular research method

22. © deSouza, Dittrich, Sharp Types of interviews In general, interviews can be classified along two continua (DeWalt and DeWalt, 2002): –The degree of control by the researcher and informants; and –The degree to which the stimuli (questions) presented to each informant are uniform.

23. © deSouza, Dittrich, Sharp Degree of control by researcher –In one extreme, the researcher has no control and he is a pure observer in an conversation. The informant has control about what he or she wants to talk about; and –In the other extreme, the researcher creates a questionnaire or survey with pre-defined questions with no accommodation for the concerns or understanding of an individual respondent.

24. © deSouza, Dittrich, Sharp Degree to which questions to each informant are uniform –In one extreme the survey is defined beforehand and is presented to all informants with no room for his/her specific concerns; and –In the other end of the continuum, each conversation is unique, there is no attempt to ask the same questions in the same way to each informant.

25. © deSouza, Dittrich, Sharp Types of Questions The distinction among the types of interviews is closely linked to the type of questions that are asked: –Closed questions: well-defined questions with a limited number of answers. The informant has to choose among this set of answers; –Open questions: ill-defined questions that require an opinion, an explanation, etc.

26. © deSouza, Dittrich, Sharp Example of Closed Questions Please indicate your level of agreement for the following statement: “My colleagues provide timely information about their changes in the source code that affect my work”. a.Strongly disagree b.Disagree c.Neutral d.Agree e.Strongly agree

27. © deSouza, Dittrich, Sharp Example of Open Questions Can you explain to me what do you do as a software architect? Please, could you identify for me the features in [the current prototype] that you think would make your own work more effective for the group?

28. © deSouza, Dittrich, Sharp Naive Questions (play dumb!) Over time the researcher becomes more and more familiar with the field. Despite that, he should avoid start making assumptions about what he knows about the field, he should not stop asking simple, even naive questions; –Avoid questions that confirm what you think you know! Everyone here uses Eclipse, right? –Instead ask the informant to explain something again to you, so that you can confirm what you think you know. Which software development tools are used here?

29. © deSouza, Dittrich, Sharp A large part of doing good field work is that ability to make yourself vulnerable and slightly stupid. That willingness to suspend early assumptions about the way the world works, and to let other people's ways of making meaning be the ones that make sense for you. (Genevieve Bell, May 2004)

30. © deSouza, Dittrich, Sharp If you have a CS degree or similar and is doing fieldwork in Software Engineering, the informants might make assumptions about what you already know (compilers, programming languages, etc). So, depending on your interest, you might even pretend to not know some of these aspects, i.e., play dumb!

31. © deSouza, Dittrich, Sharp Active Listening The researcher is primarily a listener who gains information in the terms used by the informants as the informant answers the questions. –This means, the researcher interrupts the informant as little as possible, and only when he thinks there is an interesting aspect that is not fully developed by the informant

32. © deSouza, Dittrich, Sharp Active Listening (2) The researcher is quiet, but is active. –He is aware of what is being discussed; –He is making [mental or jot] notes about what is said, who said it and whether that makes sense in the context of the project; –He is demonstrating his interest to what is being said and his respect to the ideas of the informant;

33. © deSouza, Dittrich, Sharp Sensitive Silence In some cultures, just 1 or 2 seconds of silence is enough to encourage informants to continue talking without the need of the researcher to intervene; –This silent can be used as a way to let an informant expand on a particular topic; Note that being quiet during an interview should not be underestimated, this in fact, is one of the more difficult skills to learn as one learns interviewing [DeWalt and DeWalt, 2002]

34. © deSouza, Dittrich, Sharp Feedback According to this technique, the researcher simply repeats what has been said by the informant to confirm his understanding and to encourage the informant to continue and make corrections;

35. © deSouza, Dittrich, Sharp Tell me more! “Tell me more!”. Use questions and comments that encourage the informant to continue talking about what happened, when happened, who was involved, etc: –Tell me more! –What else? –What happened? –What did you do? The idea here is to encourage the informant to keep talking about the same topic, instead of encourage him to change topics;

36. © deSouza, Dittrich, Sharp Quality assurance Pilot study –Learn from your mistakes Member checking –Replay your observations – you may have it wrong (e.g. red bar) Triangulation –By data collection method –By analysis approach

37. © deSouza, Dittrich, Sharp Practical Exercise Find a small group of people, and try to answer the question, “What are these people doing?” Watch for three to five minutes and write down what you observe. When you have finished, note down how you felt doing this, and any reactions in the group of people you observed. Use one of the structuring frameworks introduced this morning if you like. Observe more than one group if you want to try different approaches

38. © deSouza, Dittrich, Sharp Lunch

39. © deSouza, Dittrich, Sharp Practical Exercise Reflection What did you do? What were the people doing? How did you feel about doing this? Any reaction from the people you observed? If you were to observe the group again, how would you change what you did the first time?

40. © deSouza, Dittrich, Sharp In Software Engineering?

41. © deSouza, Dittrich, Sharp Data analysis Writing ethnography –Narrative Identifying Themes –Confirming and disconfirming –Many voices –Do I have to use grounded theory? Grounded Theory –Building a theory – what does it look like? Theoretical lenses –Distributed cognition, activity theory, …?

42. © deSouza, Dittrich, Sharp Writing ethnography An ethnography is a “post hoc” representation or account of what has been seen, heard and found “in the field”. Writing the ethnography is not just “writing up” the field notes. It involves their interpretation and analysis. (Dourish 1994)

43. © deSouza, Dittrich, Sharp Distributed cognition Used for analysing collaborative work –Ed Hutchins ship navigation and aeroplane flying –Ackerman and Halverson call centres –Adapted for use in HCI The people, environment & artefacts are regarded as one cognitive system Focuses on information propagation & transformation Can be used to answer ‘what if’ questions & breakdowns

44. © deSouza, Dittrich, Sharp The Documentary Method of Interpretation ©Rönkkö

45. © deSouza, Dittrich, Sharp ©Rönkkö What we need to observe to understand what is happening. (And what the members know.)

46. © deSouza, Dittrich, Sharp 46 Grounded Theory Authored by Glaser and Strauss in 1967; It does not require a prior theory about the data, that is, a set of hypotheses to be tested. –Instead, the goal of grounded theory is precisely to generate theory grounded exclusively on the existing data. It aims to develop a theory or explanation about what is going on in the field, or more specifically, what is available in the data collected. [Glaser & Strauss, 1967] and [Strauss & Corbin, 1997]

47. © deSouza, Dittrich, Sharp Grounded Theory: Overview Grounded theory is based on Coding, which is the analysis of the data; Field notes and transcriptions of interviews are coded to identify concepts and categories: –A concept names a phenomenon. It abstracts an event, object, or action/interaction that is significant to the researcher [Strauss and Corbin, 1998; p103]. –Categories are grouping concepts put together under a more abstract high order concept [Strauss and Corbin, 1998; p113]. 47

48. © deSouza, Dittrich, Sharp 48 Grounded Theory: coding Open coding –data is micro-analyzed (line-by-line) to identify categories Axial coding –categories were broken into subcategories. Whereas categories stand for phenomena, subcategories answer questions about the phenomenon, such as when, where, why, who, how, and with what consequences; –Identifies relationships between categories

49. © deSouza, Dittrich, Sharp 49 Grounded Theory: coding Selective coding –the most important categories are selected to be core categories, that is, the categories that will be used to describe the emerging theory

50. © deSouza, Dittrich, Sharp 50 Grounded Theory: Open Coding

51. © deSouza, Dittrich, Sharp Practical Exercise Look again at the data from this morning’s exercise and identify some codes/themes within the transcript