1 Back Channel Communication Antoine Raux Dialogs on Dialogs 02/25/2005

2 Outline From Back Channel to backchannels Function of the Back Channel Characteristics of the Back Channel The Back Channel in Spoken Dialogue Systems

3 From back channel… 70s: Conversation Analysts attempt to describe systematic rules for turn-taking management –Goal: minimize gaps and overlaps between speakers BUT many overlaps in natural speech –E.g.: “mm-hmm”, “okay”, “yeah”… “Back channel” (Yngve 1970): Parallel channel for communication (Duncan 1972) –“Back channel communication does not constitute a turn or a claim for a turn” –But it “may participate in a variety of communication functions, including the regulation of speaking turns.”

4 …to backchannels “Backchannel”: listener-produced signal such as “mm-hmm”, “yeah”… (“To backchannel”: to produce such signals) Does not imply the will to take the turn Implies some form of acknowledgment (in general)

5 Front ChannelBack Channel FunctionPropositional Transactional Conversation managmt Social Conversation managmt Social ProtocolTurn-taking Floor sharing ? (controlled by FC?) No floor to share Lexical contentAnythingvocalizations, short words, phrases (“That’s true”) Front vs Back Channel

6 Front-channel cues to back- channel signals Koiso et al (1998) Analyze the relationship between different syntactic and prosodic features and the occurrence of backchannels

7 Koiso et al (Methodology) Data: 8 dialogs from Japanese Map Task corpus: –replica of the Edinburgh MT –Face-to-face and speech only (no difference) Features –Syntactic: POS –Duration of last mora (normal/long/short) –F0 pattern of last mora (flat-fall, rise…) –Peak F0 (low/high) –Energy pattern (late-decr, decr, no-decr) –Peak energy (low/high)

8 Koiso et al (Results) Frequency of feature values BC > no-BC POS=verb-phrase, post-position, conjunction F0 pat=flat-fall or rise-fall Energy pat=late-decr Peak energy=high no-BC > BC POS=adv, conjunction, interjection, filler Dur=short F0 pat=fall or flat Energy pat=non-decr Peak energy=low

9 Koiso et al (Results) Decision Tree analysis Compare the loss in performance by not using each feature –POS: single best feature –Prosodic features altogether: as good as POS

10 Koiso et al (Discussion) Some POS strongly inhibit BC Individual prosodic features are not good indicators of BC occurrence BC occurrence is conditioned by both POS and prosody (as a whole) What about other languages? What about BC overlapping with speech?

11 BC cues in English and Japanese Ward and Tsukahara (2000) Tests one hypothesis (“BC are triggered by low pitch cues”) for two languages

12 The Low Pitch Cue Both in American English and Japanese, it appears that “after a region of low pitch lasting 110 ms the listener tends to produce back-channel feedback”. Goal of this paper: quantitatively test this on naturally occurring conversations

13 Ward and Tsukahara (Methodology) Data: –English: 8 conversations, 12 speakers (first author participates in 5 conversations!) –Japanese: 18 conversations, 24 speakers Prediction: –Every 10ms decide BC/no-BC by applying a hand coded rule with 5 parameters tuned to the data

14 Ward and Tsukahara (Results) Each predicted BC was considered correct if it fell within 500ms of an actual BC Low pitch region rule is better than chance both in English and Japanese

15 Ward and Tsukahara (Results) Issues: –Evaluation (tolerance window size, speakers produce BCs with different frequencies…) –No actual comparison between languages –Are low pitch regions and BCs simply correlated to other phenomena (syntactic completion, disfluencies…) or is there a direct cause/consequence relationship?

16 Effects of Native Language and Gender on BC Feke (2003) Conversation Analysis study of BC in native-English and native-Spanish, same- and mixed-gender dialogs

17 Definition of BC BC: responses of the participant that is “clearly not holding the floor”… Very loose compared to previous papers: –e.g. “How did you find Quechua?” is a BC Distinguishes In-Between BC and Overlap BC

18 Feke (Methodology) Recorded 8 non-scripted conversations between 8 different speakers (2 native languages x 2 genders x 2 subjects) Manually coded In-Between BCs and Overlap BCs

19 Feke (Results) No differences observed across cultures Participants of both genders tend to use more BC when conversing with someone of the opposite gender Difference seems bigger for females than for males

20 Feke (Discussion) Interesting/surprising result from the ethnological/sociological point of view Very few data points, no significance analysis Only looked at number of BCs Consequences on SDS? (e.g. using gender information in BC prediction, selecting the gender of an agent…)

21 BC in Practical Systems… Takeuchi et al (2003) Method to determine the timing of turn transitions and aizuchi (≈BC) on Japanese Human-Human corpus

22 Takeuchi (Approach) Similar to Koiso et al, but only using automatically extracted features Every 100 ms decide between: –Take turn –Aizuchi (BC) –Leave turn (wait)

23 Takeuchi (Approach) Decision Tree using –Syntax (POS, content/function words) –Utterance duration –Pause duration/pause since last content wd –Content word duration –F0 –Power

24 Takeuchi (Results) Precision/Recall of frame classification: –Around 80% on the training set –Less then 50% on a test set Subjective evaluation: –Artificially insert BC at predicted time –Timing was judged “good” in 70-80% –On real utterances: 72% (!)

25 Takeuchi (Discussion) Found that syntactic information did not help (contradicts Koiso?) Underscores the difficulty of evaluating turn-taking/backchanneling systems

26 Conclusion Hard to account for simultaneous turns in conversation Back Channel framework offers one explanation But most work remains very specific Missing a good theory of conversation…