Entrainment in SDS Julia Hirschberg CS 4706 11/18/2018

Entrainment/Adaptation/Accommodation/Alignment Hypothesis: over time, people tend to adapt their communicative behavior to that of their conversational partner Issues What are the dimensions of entrainment? How rapidly do people adapt? Does entrainment occur (on the human side) in human/computer conversations? 11/18/2018

Varieties of Entrainment… Lexical: S and H tend over time to adopt the same method of referring to items in a discourse A: It’s that thing that looks like a harpsichord. B: So the harpsichord-looking thing… .... B: The harpsichord… Phonological Word pronunciation: voice/voiceless /t/ in better Acoustic/Prosodic Speaking rate, pitch range, choice of contour Discourse/dialogue/social Marking of topic shift, turn-taking 11/18/2018

The Vocabulary Problem Furnas et al ’87: the probability that 2 subjects will producing the same name for a command for common computer operations varied from .07-.18 Remove a file: remove, delete, erase, kill, omit, destroy, lose, change, trash With 20 synonyms for a single command, the likelihood that 2 people will choose the same one was 80% With 25 commands, the likelihood that 2 people who choose the same term think it means the same command was 15% How can people possibly communicate? They collaborate on choice of referring expressions 11/18/2018

Early Studies of Priming Effects Hypothesis: Users will tend to use the vocabulary and syntax the system uses Evidence from data collections in the field Systems should take advantage of this proclivity to prime users to speak in ways that the system can recognize well 11/18/2018

User Responses to Vaxholm The answers to the question: “What weekday do you want to go?” (Vilken veckodag vill du åka?) 22% Friday (fredag) 11% I want to go on Friday (jag vill åka på fredag) 11% I want to go today (jag vill åka idag) 7% on Friday (på fredag) 6% I want to go a Friday (jag vill åka en fredag) - are there any hotels in Vaxholm? (finns det några hotell i Vaxholm) 11/18/2018

Verb Priming: How often do you go abroad on holiday? Hur ofta åker du utomlands på semestern? Hur ofta reser du utomlands på semestern? jag åker en gång om året kanske jag åker ganska sällan utomlands på semester jag åker nästan alltid utomlands under min semester jag åker ungefär 2 gånger per år utomlands på semester jag åker utomlands nästan varje år jag åker utomlands på semestern varje år jag åker utomlands ungefär en gång om året jag är nästan aldrig utomlands en eller två gånger om året en gång per semester kanske en gång per år ungefär en gång per år åtminståne en gång om året nästan aldrig jag reser en gång om året utomlands jag reser inte ofta utomlands på semester det blir mera i arbetet jag reser reser utomlands på semestern vartannat år jag reser utomlands en gång per semester jag reser utomlands på semester ungefär en gång per år jag brukar resa utomlands på semestern åtminståne en gång i året en gång per år kanske en gång vart annat år varje år vart tredje år ungefär nu för tiden inte så ofta varje år brukar jag åka utomlands 11/18/2018

Results no reuse no answer 4% 2% other 24% reuse 52% 18% ellipse 11/18/2018

Lexical Entrainment in Referring Expressions Choice of Referring Expressions: Informativeness vs. availability (basic level or not) vs. saliency vs. recency Gricean prediction People use descriptions that minimally but effectively distinguish among items in the discourse Garrod & Anderson ’87 Output/Input Principle Conversational partners formulate their current utterance according to the model used to interpret their partner’s most recent utterance Clark, Brennan, et al’s Conceptual Pacts People make Conceptual Pacts wrt appropriate referring expressions made with particular conversational partners They are loath to abandon these even when shorter expressions possible 11/18/2018

Entrainment in Spontaneous Speech S13: the orange M&M looking kind of scared and then a one on the bottom left and a nine on the bottom right S12: alright I have the exact same thing I just had it's an M&M looking scared that's orange S13: yeah the scared M&M guy yeah S12: framed mirror and the scared M&M on the lower right S13: and it's to the right of the scared M&M guy S13: yeah and the iron should be on the same line as the frightened M&M kind of like an L S12: to the left of the scared M&M to the right of the onion and above the iron Neither S12 nor S13 carried this term on to their second session. Neither repeats this description in their second session. 11/18/2018

Extraterrestrial vs Alien I s11: okay in the middle of the card I have an extraterrestrial figure s11: okay middle of the card I have the extraterrestrial … s10: I've got the blue lion with the extraterrestrial on the lower right s11: okay I have the extraterrestrial now and then I have the eye at the bottom right corner s10: my extraterrestrial's gone 11/18/2018

Extraterrestrial vs. Alien II S03: okay I have a blue lion and then the extraterrestrial at the lower right corner S11: mm I'll pass I have the alien with an eye in the lower right corner S03: um I have just the alien so I guess I'll match that ------------------------------------------------------------------------------ S10: yes now I've got that extraterrestrial with the yellow lion and the money … S12: oh now I have the blue lion in the center with our little alien buddy in the right hand corner S10: with the alien buddy so I'm gonna match him with the single blue lion okay I've got our alien with the eye in the corner 11/18/2018

Entrainment and High-Frequency Words How does entrainment to another’s use of HFWs (the N most common words in a corpus) affect the perceived naturalness and task success in dialogue? SwitchBoard 2430 conversations, 6m mean duration, 240h 250 very natural; 250 ‘problematic’ 100 HFWs over corpus  25 most predictive Um, how, okay, go, all, very, as, things, … HFWs predict naturalness w/ 63.75% accuracy 11/18/2018

9h8m, 12 conversations, subset: 48 tasks Scores logged for each game Columbia Games Corpus 9h8m, 12 conversations, subset: 48 tasks Scores logged for each game Labeled for turn-taking and other behaviors Results: 25 HFWs over corpus and game significantly positively correlated with game score Common use of ‘affirmative cue words’ also correlates with game score positively But entrainment in HFWs negatively correlated w/smooth turntaking and response latency, tho positively correlated with overlapping speech 11/18/2018

Timing and Voice Quality Guitar & Marchinkoski ’01: How early do we start to adapt to others’ speech? Do children adapt their speaking rate to their mother’s speech? Study: 6 mothers spoke with their own (normally speaking) 3-yr-olds (3M, 3F) Mothers’ rates significantly reduced (B) or not (A) in A-B-A-B design Results: 5/6 children reduced their rates when their mothers spoke more slowly Guitar & marchinkoski ’01: 11/18/2018

Utter a single sentence before and after the conversation Sherblom & La Riviere ’87: How are speech timing and voice quality affected by a non-familiar conversational partner? Study: 65 pairs of undergraduates asked to discuss a ‘problem situation’ together Utter a single sentence before and after the conversation Sentences compared for speaking rate, utterance length and vocal jitter Results: Substantial influence of partner on all 3 measures Interpersonal uncertainty and differences in arousal influenced degree of adaptation Other investigations of effect of gender: only when partner was male was their a significant effect of gender on adaptation 11/18/2018

Amplitude and Response Latency Coulston et al ’02: Do humans adapt to the behavior of non-human partners? Do children speak more loudly to a loud animated character? Study: 24 7-10-yr olds interacted with an extroverted, loud animated character and with an introverted, soft character (TTS voices) Multiple tasks using different amplitude ranges Human/TTS amplitudes and latencies compared Results: 79-94% of children adapted their amplitude, bi-directionally Also adapted their response latencies (mean 18.4%), bidirectionally 11/18/2018

Social Status and Entrainment Azuma ’97: Do speakers adapt to the style of other social classes? Study: Emperor Hirohito visits the countryside Corpus-based study of speech style of Japanese Emperor Hirohito during chihoo jyunkoo (`visits to countryside‘), 1946-54 Published transcripts of speeches Findings: Emperor Hirohito converged his speech style to that of listeners lower in social status Choice of verb-forms, pronouns no longer those of person with highest authority Perceived as like those of a (low-status) mother Before, used pronouns used only by the highest authority (only one recorded instance of such a speech tho) 11/18/2018

Socio-Cultural Influences and Entrainment Co-teachers adapt teaching styles (Roth ’05) Social context High school in NE with predominantly African-American student body Cristobal: Cuban-African-American teacher Chris: new Italian-American teacher Adaptation of Chris to Cristobal Catch phrases (e.g. right!, really really hot) and their production: pitch and intensity contours Pitch ‘matching’ across speakers Mimesis vs entrainment Right rises sharply Really really downstepped and then higher pitch on modifier 11/18/2018

Conclusions for SDS Systems can make use of user tendency to entrain to system vocabulary Should systems also entrain to their users? CMU’s Let’s Go system adapts confirmation prompts to non-native speech, finding the closest match to user input in its own vocabulary 11/18/2018

Personality and Computer Systems Early-pc-era reports that significant others were jealous of the time their partners spent with their computers. Reeves & Nass, The Media Equation How People Treat Computers, Television, and New Media Like Real People and Places, 1996 Evolution explains the anthropomorphization of the pc Humans evolved over millions of years without media Proper response to any stimulus was critical to survival Human psychology and physiological responses well developed before media invented Ergo, our bodies and minds react to media, immediately and fundamentally, as if they were real Clifford Nass and Byron Reeves, then both at Stanford in Communications Dept 11/18/2018

People See ‘Personality’ Everywhere Humans assess personality of another (human or otherwise) quickly, with minimal clues Perceived computer personality strongly affects how we evaluate the computer and information it provides Experiments: Created “dominant” and “submissive” computer interfaces and asked subjects to use to solve hypothetical problems Max (dominant) used assertive language, showed higher confidence in the information displayed (via a numeric scale), always presented its own analysis of the problem first Linus (submissive) phrased information more tentatively, rated its own information at lower confidence levels, and allowed human to discuss problem first Each used alternately by people whose personalities previously identified as being either dominant or submissive 11/18/2018

User Reactions Users described Max and Linus in human terms: aggressive, assertive, authoritative vs. shy, timid, submissive Users correctly identified machines more like themselves Users rated machines more like themselves as better computers even though content received exactly the same. Users rated their own performance better when machine’s personality matched theirs People more frank when rating a computer if questionnaire presented on another machine Subjects thought highly of computers that praised them, even if praise clearly undeserved 11/18/2018

Personality in SDS Mairesse & Walker ’07 PERSONAGE (PERSONAlity GEnerator) ‘Big 5’ personality trait model: extroversion, neuroticism, agreeableness, conscientiousness, openness to experience Attempts to generate “extroverted” language based on traits associated with extroversion in psychology literature Demo: find your personality type Francois Mairesse and Marilyn Walker PERSONAGE: Personality Generation for Dialogue 11/18/2018

11/18/2018

Conclusions for SDS Systems can be designed to convey different personalities Can they recognize users’ personalities and entrain to them? Should they? 11/18/2018