1. The effects of relevance of on-screen information on gaze behaviour and communication in 3-party groups Emma L Clayes University of Glasgow Supervisor: Prof. Anne Anderson Co-Supervisor: Jim Mullin BT Supervisor: Dr. David Hands Sponsored by UK ESRC and BT

2. Outline Background & Hypotheses Design and Method Results Summary Future work Conclusion

3. Representations: form and location Many studies concerning mediated communication (Finn et al, 1997) with different technologies, measures and tasks Little research on the impact of different representations (e.g Sellen, 1995; Parise et al, 1996)

4. Representations and relevance How does the form and location of representations impact on gaze and communication? How does this interact with the relevance of on screen information ( e.g. task related information and representations in terms of the task role)

5. Evaluation techniques in CMC Many different measures used to examine computer mediated communication (e.g. dialogue analysis, task performance, questionnaires) Additional factors (e.g. task, no of participants, technology) and different methods have led to conflicting results

6. Methodological issues Three-party groups Problem solving and social tasks Communication analysis, questionnaire responses and task performance Eye-tracking as an evaluation technique

7. Eye-tracking in Psychology and HCI Well established measure of human information processing (Rayner, 1998) Eye-gaze computer interfaces (Jacob, 1991) Recent studies examining gaze and CMC- Velichkovsky et al (1997), Mullin et al (2001)

8. Communication and Attention Eye-tracking as an evaluation technique: What do users attend to when using remote communication systems? Does the relevance of on screen information impact on gaze behaviour? Are patterns of gaze related to patterns of interaction within the group?

9. Eye-tracking in CMC Exploratory questions: Is it possible to obtain significant amounts of eye data during a non-restricted interaction? Are participants consistent in their patterns of gaze across a screen? Are these patterns meaningful?

10. Communication and Relevance Relevance of on screen information (e.g. video data and video links) Shared visual data more useful than video conference links (Daly-Jones et al 1998)

11. Communication and Relevance Relevance of other collaborators (e.g. task role and status) Status effects in mediated communication (Dubrovsky et al 1991, France et al, 2000)

12. Study 1 What on-screen features attract most gaze? Does the position of representations on screen impact on mediated communication? Does this interact with the type of task and relevance of information presented on screen?

13. Task 1-Problem-solving task Map task-collaborative problem-solving task Two Instruction Givers (IG1 and IG2) have to instruct one other person (Instruction follower-IF) on how to draw a route on their map Therefore, relationship between IG and IF more relevant to task success than IG1 and IG2

14. Task 1-Display Screen

15. Task 1-Hypothesis I Hypothesis I- Users will gaze more often at shared visual data ( i.e. the map) than video links of remote collaborators

16. Task 1-Hypothesis II Do participants look equally often at the IF and IG video images? Does the location of the video image affect gaze behaviour?

17. Task 1-Hypothesis III Do patterns of gaze reflect patterns of interaction within the group? I.e. Do participants talk more often to the person they look most often at?

18. Design 2 factor mixed design Relevance (map, video-IF, video-IG) within subjects Location (video-IF top, video-IF bottom) between subjects

19. Method Subjects- 10 groups of 3, 20 eye- tracked 2 maps, order of maps and location of IF video balanced 3 participants in different rooms-high quality audio and video links- Eye tracked participant always an IG

20. Results Screen divided into areas of interest for eye gaze analysis Percentage analysis on different areas of interest and different fixations

21. Eye Data-Pictorial analysis

22. Results-Hypothesis I Participants spent significantly more time looking at the map (72.4%) than video image of IF (12%) and IG (10%) Relevance of on-screen information impacts on gaze behaviour F(2,36)=258.15, p<.001

23. Results-Hypothesis II Do participants look more often at IF compared to IG and is this affected by location? Main effect of relevance (IF 12%, IG 10%) p<.05 No effect of location p>.05.

24. Results-Hypothesis II Significant interaction between relevance of video image and location of video image F(1,18)=5.73,p<.05 Participants looked more often at the instruction follower - only significant when the IF video was located in the top left of the screen (F=10.48, p<.05)

25. Results-Hypothesis III Do patterns of gaze reflect patterns of interaction within the group? Does the eye tracked participant direct more turns of speech to the IF or IG? Turn combination analysis: IG1-IG2, IG1-IF No effects of relevance (task role-IG2/IF), location of IF or an interaction between the two.

26. Results-Hypothesis III

27. Task 1-Summary Users look more often at shared visual data (map 72.4%) than video images (22%) Users gaze more often at IF than IG-only significant when IF video is located in top left of screen

28. Task 1-Summary Communication patterns reflect that IG directs almost equal number of turns to IG and IF- does not reflect patterns of gaze Positional effect for salient video images?

29. Task 2-Relevance and Status Realistic task-mediated business meeting Two confederates- one high status (boss) and one low status (marketing assistant) Video data presented not essential to complete task

30. Task 2-Relevance and Status Hypothesis I- Do participants gaze more often at shared data compared to video images when the information is relevant, but not essential to complete the task.

31. Task 2-Relevance and Status Hypothesis II- Do participants look more often at a high-status collaborator? Does the location of the high status video image impact on gaze behaviour?

32. Task 2-Relevance and Status Hypothesis III- do patterns of gaze reflect patterns of interaction within the group? I.e do participants talk more often to the person they look most often at?

33. Task 2-Design 2 factor mixed design Relevance (visual graphic, video-high status, video-low status) within subjects Location (video-high status top, video- high status bottom) between subjects

34. Task 2-Method 2 confederates, 20 eye-tracked participants 3 participants in different rooms-high quality audio and video links

35. Task 2-Display Screen

36. Results-Hypothesis I Participants spent significantly less time looking at the low status video image (18%) than both the task feature (24.5%) and the high status video image (35%) Relevance of on-screen information impacts on gaze behaviour (F(2, 36)=8.19, p<.05), no effect of location or an interaction.

37. Results-Hypothesis II Do participants gaze more often at the high status confederate compared to the low status confederate and does this interact with the location of video images? Only sig main effect of status F(1,18)=22.19,p<.05, no effect of location or an interaction Participants gazed more often at the high status confederate (35%) compared to the low status confederate (24.5%)

38. Results-Hypothesis III Do patterns of gaze reflect patterns of interaction within the group? Do participants direct more turns of speech to the high status confederate compared to the low status confederate?

39. Results-Hypothesis III Turn combination analysis: sig effect of status, no effect of location or an interaction Participant directs more turns of speech to high status confederate and this reflects patterns of gaze

40. Results-Hypothesis III

41. Task 2-Summary Participants gaze less often at low status video compared to high status video and task feature Participants gaze more often at high status video regardless of location Interaction in Task 1 not replicated- video images attract more gaze and main effect of status found Patterns of gaze reflect patterns of communication

42. Study 1-Summary Participants gaze more often at shared data in task 1, not in task 2 Significant interaction between location and relevance of video images in task 1, not in task 2 Overwhelming impact of status on gaze and communication in task 2 Communication patterns reflect patterns of gaze in Task 2, not Task 1

43. Study 1-Summary Eye tracking provides valuable information about mediated interaction Distribution of attention related to experimental task manipulations Positional effect for salient video images?

44. Study 1-Eye-tracking in CMC Exploratory questions about eye tracking answered Data capture rate high –managed to track on average 70% of participants who took part Average of 80% of fixations directed on screen during task Patterns of gaze consistent and meaningful

45. Future Work Same tasks, different design Video images placed in 4 corners of the screen, therefore 4 conditions for each task Task 1:Provisional results suggest interaction not replicated- always gaze more often at the map Task 2:Difference in gaze distribution to videos smaller when video images are placed on same side of the screen than when they are placed on opposite sides of the screen

46. Conclusion Eye tracking can be used successfully as an evaluation technique Task differences and status effects robust- reflected in both patterns of gaze and communication May be positional effect for salient video images?

47. Conclusion Implications for the design of remote communication systems Context in which system is to be applied very important e.g. social or problem solving Further research required on positional effects-may be used to enhance or reduce amount of gaze directed to representations