1 User-Centered Design and Development Instructor: Franz J. Kurfess Computer Science Dept. Cal Poly San Luis Obispo FJK 2005

2 Copyright Notice These slides are a revised version of the originals provided with the book “Interaction Design” by Jennifer Preece, Yvonne Rogers, and Helen Sharp, Wiley, I added some material, made some minor modifications, and created a custom show to select a subset. –Slides added or modified by me are marked with my initials (FJK), unless I forgot it … FJK 2005

3 484-W09 Quarter The slides I use in class are in the Custom Show “484-W09”. It is a subset of the whole collection in this file. Week 7 contains slides from Chapters 12 and 13 of the textbook. FJK 2005

4 484-W10 Quarter The set of slides I use in class is close to the one in the PowerPoint Custom Show “484-W09”. Since I’m using mostly Keynote now, I use the “Skip” feature to achieve a similar result. 4

5 Chapter 12 Observing Users FJK 2005

6 Chapter Overview Goals, Questions, and Paradigms Observation Data Collection Tracking Activities Analysis, Interpretation, and Presentation of Data FJK 2005

7 Motivation the actual observation of users as they perform tasks with the system may reveal information that is difficult to obtain otherwise technologies like video cameras and audio recorders make it fairly easy to capture many aspects of the observation the activities of users can also be tracked indirectly the analysis, interpretation and presentation is essential after the actual observation

8 FJK 2005 Objectives be familiar with the main observation methods be aware of the different perspectives of an on-looker, a participant, and an ethnographer appreciate the importance of analyzing, interpreting, and presenting data obtained from observations gain experience in performing and evaluating observations

9 Observing users

10 The aims Discuss the benefits & challenges of different types of observation. Describe how to observe as an on-looker, a participant, & an ethnographer. Discuss how to collect, analyze & present observational data. Examine think-aloud, diary studies & logging. Provide you with experience in doing observation and critiquing observation studies.

11 Activity: Observation establish teams of three to five –one or two participants –two or three observers participants select an activity to perform –do not reveal it to the observers hints are allowed –can, but does not have to be, computer-related –discuss with the observers what kinds of observation techniques would be useful participants perform the chosen activity –observers utilize or simulate the agreed-upon observation techniques observers collect, analyze, and interpret data –either as a team, or independently (might be more fun) –come up with a hypothesis –present it to the participants FJK 2005

12 What and when to observe Goals & questions determine the paradigms and techniques used. Observation is valuable any time during design. Quick & dirty observations early in design Observation can be done in the field (i.e., field studies) and in controlled environments (i.e., usability studies)

13 Roles of Observers outsiders looking on –sometimes through a window or one-way mirror –should the participant be aware of it? participating observers –the observer works with other participants –emphasis of the observer should be on observation, not on activities to perform the task ethnographers –intentional immersion into the “world” of the participants FJK 2005

14 Observation Frameworks The person. Who? The place. Where? The thing. What? Framework Examples –Goetz and LeCompte (1984) –Robinson (1993)

15 The Goetz and LeCompte (1984) Framework Who is present? What is their role? What is happening? When does the activity occur? Where is it happening? Why is it happening? How is the activity organized? 15

16 The Robinson (1993) Framework Space –What is the physical space like? Actors –Who is involved? Activities –What are they doing? Objects –What objects are present? Acts –What are individuals doing? Events –What kind of event is it? Goals –What do they to accomplish? Feelings –What is the mood of the group and of individuals? 16

17 Aspects to Consider goals and questions selection of framework, techniques data collection equipment to use acceptance and trust by participants handling of sensitive issues involvement of informants data analysis need for triangulation –different perspectives

18 Observing as an Outsider same technique as in usability testing more objective than participant observation often done in usability labs –layout of the facilities –equipment is in place recording is continuous analysis & observation almost simultaneous danger of drowning in data analysis –can be coarse or fine grained presentation –Video clips can be powerful for telling astory

19 Participant Observation and Ethnography debate about differences participant observation is key component of ethnography co-operation of people observed informants can be useful –but also sensitive data analysis is continuous interpretivist technique questions get refined as understanding grows reports usually contain examples

20 Activity: Observation in RPGs is it ethical to join a community for the main purpose of observation –with or without the knowledge of other participants –human observer or computer agent do you know of any incidents in games you’re participating in FJK 2005

21 Data Collection Techniques notes –often selective, and include some interpretation visual data –still camera –video audio data –tape recorder –usually included in video –computer tracking users: –diaries –interaction logging

22 Observation Activity: Review data collection techniques used –appropriate –worth the effort constraints imposed by the selected techniques –missed data –aspects not captured –observer bias –differences between observers analysis and interpretation –bias, differences, jumping to conclusions FJK 2005

23 Data Analysis qualitative data –interpreted and used to tell the ‘story’ about what was observed –categorized using techniques such as content analysis quantitative data –collected from interaction and video logs –presented as values, tables, charts, graphs and treated statistically.

24 Interpretive Data Analysis key events –drive the group’s activity patterns of behavior triangulation –testing of data sources against each other presentation –report findings in a convincing and honest way ‘rich’ or ‘thick descriptions’ –include quotes, pictures, and anecdotes software tools can be useful –NUDIST, nVivo (see –Ethnograph (see augmented FJK 2005

25 Looking for Patterns critical incident analysis –coarse analysis of the overall data set to identify important events –only those events are analyzed in more detail content analysis –systematic categorization of events –requires orthogonal categories (no overlap) discourse analysis –interpretation of dialog –also considers meaning, intention, context quantitative analysis (statistics) –means, standard deviation, etc. augmented FJK 2005

26 Key Points observation from outside or as a participant analyzing video and data logs can be time- consuming participant observation includes collections of comments, incidents, and artifacts ethnography is a philosophy with a set of techniques that include participant observation and interviews ethnographers immerse themselves in the culture that they study

27 ActiveWorlds …An ethnographic project for you … Join ActiveWorld.com if you have a high speed Internet connection or go to another chat room of your choice. 3-D chatroom environment –visit different environments –chat with the people that you meet there. Example: –The next slide shows the Yellowstone world.

28 ActiveWorld Yellowstone

29 ActiveWorld Activity Select one of the worlds to visit and choose an avatar (a graphical personification) to represent you. Spend one to two hours doing a participant observation study. Use one of the frameworks discussed in the previous slides to guide you and write a one-two page report about your study. Also notice and report on any usability issues you encounter and on user experiences in this environment. (Chapters 1, 4 and 5 discuss user experience criteria that are relevant.)