Coye Cheshire & Andrew Fiore // Computer-Mediated Communication Media Richness and Visual Interfaces 20 February 2013

Projects and Assignment #1  Assignment 1 is a short 2-3 page description of your group project idea and the division of labor within the group.  Due Feb. 27 (next week) at beginning of class (one assignment per group, 2 printed copies)  Groups will be signing up for a meeting with us to discuss the project the following Wednesday (Mar. 6).  2/20/131Cheshire & Fiore — Computer-Mediated Communication

Theories of mediated communication 2/20/13Cheshire & Fiore — Computer-Mediated Communication2

Cues Filtered Out 2/20/13Cheshire & Fiore — Computer-Mediated Communication3 unsealedprophecy.wordpress.com  Social presence: Lower bandwidth  Less warm, others seem less like people  Lack of non-verbal cues — disinhibition and hostility (e.g., flaming)  1:1 mapping between cues and social functions — missing cues, missing functions

Social Identity/Deindividuation Theory (Cues About Us, Not You or Me) Visual anonymity  “deindividuation”  salience to group identity “Overinterpreting” based on limited info could lead to greater social attraction based on in-group status; stereotyping of out-group. 2/20/13Cheshire & Fiore — Computer-Mediated Communication4

2/20/13Cheshire & Fiore — Computer-Mediated Communication5 [C]apacity to facilitate the formation of shared meaning within a given time interval. “ ” — Dennis & Kinney Media richness:

2/20/13Cheshire & Fiore — Computer-Mediated Communication6 A plausible ranking? Face-to-face Synchronous video Synchronous audio / asynch. video Synchronous text / asynch. audio Asynchronous text Richer Leaner

2/20/13Cheshire & Fiore — Computer-Mediated Communication7 Rich Elements of richness  Multiplicity of cues (bandwidth)  Immediacy of feedback  Use of natural language  Personal focus

2/20/13Cheshire & Fiore — Computer-Mediated Communication8 Lean Elements of richness  Multiplicity of cues (bandwidth)  Immediacy of feedback  Use of natural language  Personal focus

Channel: conduit for a particular type of info, e.g., for voice or text Cue: “any feature of the world, animate or inanimate, that can be used... as a guide to future action” (Donath 2007) — i.e., informative, not necessarily intentional Signal: a cue meant to indicate an otherwise hidden quality Channel Cue Signal Channels, cues, and signals 2/20/13Cheshire & Fiore — Computer-Mediated Communication9

2/20/13Cheshire & Fiore — Computer-Mediated Communication10 Some types of social cues Textual  Production cost to encode meaning equivalent to FTF in text Verbal Beyond FTF? Non-verbal

2/20/13Cheshire & Fiore — Computer-Mediated Communication11 Feedback Convey the receiver’s understanding to sender, who can adjust accordingly Type of feedback  Acknowledgment — understanding  Repair — correction or clarification  Proxy — completion Immediacy of feedback  Concurrent: synchronous nods, mm-hmms  a.k.a. backchannel  Sequential: brief interjection

Media choice vs. use (Cues to Choose By) What medium would you choose for a given task? vs. What medium “performs” best?  Media Richness (the theory) originally examined media choice and use in organizations. Claim: Managers should choose medium based on task to be effective. More ambiguous tasks are more efficient in richer media. But when might we want a “less rich” medium? 2/20/1312Cheshire & Fiore — Computer-Mediated Communication

2/20/13Cheshire & Fiore — Computer-Mediated Communication13 Media choice vs. media use Types of tasks  “Uncertain” — missing information  “Equivocal” — ambiguous interpretations “Best” medium for an (un)equivocal task  What do managers say they would choose?  What do they actually choose?  What yields the best performance? (And what is “best performance”?)

2/20/13Cheshire & Fiore — Computer-Mediated Communication14 Dennis & Kinney hypotheses  H1a: Performance improves as multiplicity of cues increases …  H1b: … more for more equivocal tasks.  H2a: Performance improves as immediacy of feedback increases …  H2b: … more for more equivocal tasks.

Dennis & Kinney experiment Tasks  Low-equivocality: SAT-type questions  High-equivocality: College admissions Media 2/20/13Cheshire & Fiore — Computer-Mediated Communication15 Cues: LowCues: High Feedback: Delayed Text chat (turn-based) Video (half-duplex) Feedback: Immediate Text chat (live typing) Video (full-duplex)

2/20/13Cheshire & Fiore — Computer-Mediated Communication16 Mean decision time High cues (AV)Low cues (CMC) TaskImmed.DelayedImmed.Delayed Low equiv High equiv Mean decision quality High cues (AV)Low cues (CMC) TaskImmed.DelayedImmed.Delayed Low equiv – 0.03– 0.14 High equiv.0.32– 0.03– 0.19– 0.05

2/20/13Cheshire & Fiore — Computer-Mediated Communication17 — Clark & Brennan (1991)

2/20/13Cheshire & Fiore — Computer-Mediated Communication18 “The extra facial cues [in the video conditions] do not appear to be that valuable, unless the decision was actually harder to make. Facial cues might be more important in a situation where feeling and emotion were bigger factors. In such a simple study, it is doubtful that anyone feels particularly strong about any opinion.” — Taeil “[The authors assume] that the same factors can be used to evaluate the performance of all tasks. Media richness can be effective for certain tasks but not others. There are tasks that don't require audio & video. A simple one-line text or may be sufficient. Some other tasks, audio & video can play a bigger role because the task itself requires real-time social interactions and exchange. The authors did touch on these concepts in the conclusions. I just wish that more of these were front-loaded and incorporated into the experiments.” — Joyce

Social Information Processing (Cues Filtered In) Walther (1992) re-examined early CMC research: “Given sufficient time and message exchanges for interpersonal impression formation and relationship development to accrue, and all other things being equal, relational [quality] in later periods of CMC and F2F communication will be the same.”  Users compensate for attributes of CMC (e.g., emoticons, timing of messages) 2/20/1319Cheshire & Fiore — Computer-Mediated Communication

Hyperpersonal communication (Cues Bent and Twisted) Contributing factors:  Sender:Selective self-presentation  Receiver: Perceived commonality, anticipated future interaction  Channel effects — control, time to consider responses  Feedback effects — confirmation of idealized perceptions Bottom line: Potential for hyper-positive (or hyper- negative) perceptions in CMC as compared to FTF 2/20/1320Cheshire & Fiore — Computer-Mediated Communication

2/20/13Cheshire & Fiore — Computer-Mediated Communication21 The sensorial parsimony of plain text tends to entice users into engaging their imaginations to fill in missing details while, comparatively speaking, the richness of stimuli in fancy [systems] has an opposite tendency, pushing users’ imaginations into a more passive role. — Curtis (1992) “ ”

Walther, Slovacek, & Tidwell /20/13Cheshire & Fiore — Computer-Mediated Communication22

2/20/13Cheshire & Fiore — Computer-Mediated Communication23 Long-term, no photos Short-term, photos Long-term, photos Short-term, no photos Social affinity

2/20/13Cheshire & Fiore — Computer-Mediated Communication24 Farnham & Riegelsberger 2004 Text profiles Photo profiles Gaming partner preference (1 = Don’t want to play with, 7 = Want to play with) Count

2/20/13Cheshire & Fiore — Computer-Mediated Communication25 The study of CMC effects is not best served by blanket statements about technology main effects on social, psychological, and interpersonal processes, nor by proclamations that online relationships are less rewarding than FTF ones. Rather, qualities of CMC are … more often the product of interesting and predictable interactions of several mutual influences than main effects of media. — Walther et al. (2001) “ ”

2/20/13Cheshire & Fiore — Computer-Mediated Communication26 “In general, I found Walther & Parks' discussion to be useful, especially the discussion of the hyperpersonal effects of online communication. I like that this perspective put pressure on the assumption that FtF communication is, a priori, the epitome of media richness.” — Kyle “I wish they had talked more about the tools that are prescribed for expression and interaction on various CMC platforms. For instance, when there is a toolbar containing several different emoticons, it naturally makes for more interesting social interaction between the users than a purely text-based interaction. I was wondering what other kinds of tools are available that enhance the ability to express oneself in online communication, and how much these tools affect users' willingness to use a site.” — Kayal

Let the dance begin! 2/8/1227Cheshire & Fiore — Computer-Mediated Communication

Abstract visual interfaces 2/20/13Cheshire & Fiore — Computer-Mediated Communication28

2/20/13Cheshire & Fiore — Computer-Mediated Communication29 Social translucence  Visibility: make social information apparent  Awareness: knowing based on what you see  Accountability: knowing that I know you know  Why? To recreate a “social physics.”  Why not “social transparency”?

2/20/13Cheshire & Fiore — Computer-Mediated Communication30

2/20/13Cheshire & Fiore — Computer-Mediated Communication31 “[T]ranslucence … stands in for the notion that, in the physical world, cues are differentially propagated through space — something which, as social creatures, we understand and make use of in governing our interactions. Thus, we know that those across the room may see we are talking, but will be unable to hear what we say; and we adjust our interactions to take advantage of this.” — Erickson et al.

2/20/13Cheshire & Fiore — Computer-Mediated Communication32

2/20/13Cheshire & Fiore — Computer-Mediated Communication33 Babble social proxy “provide cues about the presence and activity of those in the current conversation”

“Socially useful ambiguity”  Pretending to pay attention, e.g., clicking the Babble proxy to feign attention to the conversation  Plausible deniability: consider the fallibility of cell phones, /spam filtering, etc. — tech. limitations, not design decisions, but the social utility of these devices would change without them. 2/20/13Cheshire & Fiore — Computer-Mediated Communication34

Lecture proxy 2/20/13Cheshire & Fiore — Computer-Mediated Communication35

Auction proxy 2/20/13Cheshire & Fiore — Computer-Mediated Communication36

Waiting-in-line proxy 2/20/13Cheshire & Fiore — Computer-Mediated Communication37

2/20/13Cheshire & Fiore — Computer-Mediated Communication38 “They discuss how social ambiguity is preserved in their system by way of their visualization design which seems to suggest that it is possible to build an online system with absolutely no social ambiguity but I don't think that's a realistic idea.” — Laura “[A]s the article noted, someone could be pretending to participate. But what about active listeners? This form of participation is not capable within social mediating tools, unless video mediated, given the missing cues of watching a person's body language, eyes, etc. -- a clear social- technical gap.” — Tine

2/20/13Cheshire & Fiore — Computer-Mediated Communication39 Chat Circles 2

DEMO: 2/20/13Cheshire & Fiore — Computer-Mediated Communication40

The Chat Circles avatar  Vaguely humanoid form, but stylized, not realistic — no faces!  Words centered in/around the form — ties words to identity, “face”  2D location allows proximity  Size tied to length of utterance 2/20/13Cheshire & Fiore — Computer-Mediated Communication41

Temporality and spatiality  Utterances vanish after a few seconds  Hearing range: can see only nearby utterances  What is the real-world effect mimicked here? 2/20/13Cheshire & Fiore — Computer-Mediated Communication42

Chat Circles Movement  Rhythm of conversation: growing and shrinking circles set the pace  Proximity: friendliness, intimacy, or aggression  Expressivity: fidgeting, dancing, leading, following, playing 2/20/13Cheshire & Fiore — Computer-Mediated Communication43

Chat Circles Traces  Movement traces  Speech traces  Visual indicator of social history of the chat space 2/20/13Cheshire & Fiore — Computer-Mediated Communication44

Chat Circles History 2/20/13Cheshire & Fiore — Computer-Mediated Communication45

Visiphone 2/20/13Cheshire & Fiore — Computer-Mediated Communication46 A B A B

Faces 2/20/13Cheshire & Fiore — Computer-Mediated Communication47

2/20/13Cheshire & Fiore — Computer-Mediated Communication48 What are faces good for? Conveying, among other things:  Social presence  Individual identity  Social identity  Emotion  Gaze By means of:  Structure  Dynamics  Decorations Source: galante.com

2/20/13Cheshire & Fiore — Computer-Mediated Communication49 Characteristics of basic emotions 1. Distinctive universal signals 2. Distinctive physiology 3. Automatic appraisal 4. Distinctive universals in antecedent events 5. Distinctive appearance developmentally 6. Presence in other primates 7. Quick onset 8. Brief duration 9. Unbidden occurrence 10. Distinctive thoughts, memories images 11. Distinctive subjective experience Basic emotions Anger Disgust Fear Joy Sadness Surprise Contempt Ekman (1999) Ekman, Friesen, & Ellsworth (1972) (and many others)

2/20/13Cheshire & Fiore — Computer-Mediated Communication50 Action unitsFacial muscles

2/20/13Cheshire & Fiore — Computer-Mediated Communication51 Representing the face: “Being close may be worse.”

2/20/13Cheshire & Fiore — Computer-Mediated Communication52

2/20/13Cheshire & Fiore — Computer-Mediated Communication53

2/20/13Cheshire & Fiore — Computer-Mediated Communication54

The Uncanny Valley 2/20/13Cheshire & Fiore — Computer-Mediated Communication55

2/20/13Cheshire & Fiore — Computer-Mediated Communication56 “being there” vs. “beyond being there”

2/20/13Cheshire & Fiore — Computer-Mediated Communication57 Designing with faces and bodies We read meaning in lots of things, but especially human forms!  There is no such thing as neutral.

2/20/13Cheshire & Fiore — Computer-Mediated Communication58

2/20/13Cheshire & Fiore — Computer-Mediated Communication59

2/20/13Cheshire & Fiore — Computer-Mediated Communication60 Chernoff faces

2/20/13Cheshire & Fiore — Computer-Mediated Communication61 The New York Times and Prof. Steve C. Wang

2/20/13Cheshire & Fiore — Computer-Mediated Communication62

2/20/13Cheshire & Fiore — Computer-Mediated Communication63 Indeed, the 2007 managerial statistics, as presented in an annual register published by the baseball analyst Bill James, are a relatively dull grid of digits. But the facial maps make comparisons much easier to grasp. The St. Louis Cardinals’ Tony La Russa, known as a constant tinkerer, had his National League-leading 150 different batting orders (in 162 games) translate into an elongated head and wider eyes. By contrast, the Philadelphia Phillies’ Charlie Manuel — who said this spring that he used far fewer lineups because he preferred to “get into a routine and stay with it” — had a much squatter face and dots for eyes. — The New York Times

Faces in interfaces 2/20/13Cheshire & Fiore — Computer-Mediated Communication64

Chit Chat Club 2/20/13Cheshire & Fiore — Computer-Mediated Communication65 (Karahalios and Dobson)

Chit Chat Club 2/20/13Cheshire & Fiore — Computer-Mediated Communication66

Second Life facial expressions 2/20/13Cheshire & Fiore — Computer-Mediated Communication67

Second Life expression plug-in 2/20/13Cheshire & Fiore — Computer-Mediated Communication68

Facial Expression Analysis 2/20/13Cheshire & Fiore — Computer-Mediated Communication69 (Mateos:

2/20/13Cheshire & Fiore — Computer-Mediated Communication70 Rather, the key is a balance between the information provided and the message that is sent. If minimal information is provided, a minimal message should be sent. The problem with many face-based interfaces is that they are sending too complex a message upon the receipt of too little data. The face is so highly expressive, and we are so adept at reading (and reading into) it, that any level of detail in its rendering is likely to provoke the interpretation of various social messages; if these messages are unintentional, the face is arguably hindering communication more than it is helping. — Donath

Eyes 2/20/13Cheshire & Fiore — Computer-Mediated Communication71

2/20/13Cheshire & Fiore — Computer-Mediated Communication72 Kobayashi & Kohshima 2001

2/20/13Cheshire & Fiore — Computer-Mediated Communication73 Kobayashi & Kohshima 2001

Video chat 2/20/13Cheshire & Fiore — Computer-Mediated Communication74

2/20/13Cheshire & Fiore — Computer-Mediated Communication75 The gaze angle problem, or… Source: Why so glum?

2/20/13Cheshire & Fiore — Computer-Mediated Communication76 Source:

2/20/13Cheshire & Fiore — Computer-Mediated Communication77 Yang & Zhang 2004

2/20/13Cheshire & Fiore — Computer-Mediated Communication78 Source: D. Nguyen

2/20/13Cheshire & Fiore — Computer-Mediated Communication79 Cameras Projectors MultiView Display Source: D. Nguyen

Lag, lip synch, social judgments  When audio precedes video by 5 video fields, viewers evaluate people on television more negatively (e.g. less interesting, more unpleasant, less influential, more agitated, less successful).  Audio-video asynchrony has no effect on viewer's memory for audio information.  Viewers can accurately tell when a television segment is in perfect synch, and when it is 5 fields out of synch. Viewers cannot accurately tell the same segments are 2.5 fields out of synch.  Even though detection is low when asynchrony is moderate (2.5 fields), viewer evaluations are still affected. 2/20/13Cheshire & Fiore — Computer-Mediated Communication80 (Reeves and Voelker 1993)

2/20/13Cheshire & Fiore — Computer-Mediated Communication81 “With all the modes at our disposal (picture, video, FaceTime), the emoticon is still the best transmitter of one's 'chosen' expression over text. Why use the emoticon, when you can send someone a picture of you smiling, or looking puzzled, or angry? The answer, it seems, is that we have collectively "understood" this medium of communication, and matched a :) to it. […]As a frequent user of GoogleHangouts and Skype conferencing, I find them oddly unsatisfying for reasons I can't fully explain.” — Jen “[T]he fact that there are attempts to bring gaze into the mediated world is bizarre and scary at the same time. If we can create a space on a virtual platform that allows for gaze to exist then inter-personal relationships will almost die because all communication can be simulated over a medium.” — Maurice

2/20/13Cheshire & Fiore — Computer-Mediated Communication82 For next Wednesday… Visualizations and Visual Interfaces  Monmonier, M. (1996) Chapters 3 and 10. In How to Lie with Maps. Chicago, Ill.: University Of Chicago Press.  Erickson, T. (2003) Designing visualizations of social activity: six claims. In Extended abstracts of ACM Computer-Human Interaction.  Donath, J. (2011) Visualizing Conversation.  Narayan, S., Cheshire, C. (2010) Not too long to read: The tldr Interface for Exploring and Navigating Large-Scale Discussion Spaces. In Proceedings of the 43rd Annual Hawaii International Conference on System Sciences. (HICSS). Computer Society Press. Remember to write your review!