1. Turn-Taking, Grounding and Diarization
Julia Hirschberg
LSA07 353
11/19/2018

2. Rules Governing Turn Construction
1. For any turn, at a transition-relevant place:
If “current speaker selects next” then the person selected is obligated to take the next turn.
If not, self-selection may be instituted.
Or, the current speaker may continue unless someone self-selects.
2. If 1c occurs, then rules a-c reapply at the next transition-relevant place
11/19/2018

3. The System Accounts for:
Speaker change
One party talking majority of the time
Occcurrences of more than one speaker at a time are common but brief
11/19/2018

4. Types of Overlap
Accounted for by rule 1b (self-selection): Competing for next turn.
Lil: Bertha’s lost, on our scale, about fourteen pounds.
Damora: Oh[::no::,
Jean: [Twelve pounds I think wasn’t it =
Daisy: [Can you believe it?
Lil: [Twelve pounds on the Weight Watcher’s scale.
11/19/2018

5. Types of Overlap (cont…)
Assume you know how the speaker will finish.
A: Well if you knew my argument why did you bother to a:[sk.
B: [Because I’d like to defend my argument.
The speaker added optional elements that can go after completion.
A: Uh you been down here before [havenche,
B: [Yeh.
11/19/2018

6. Because of unit types (single-word turns, single phrase turns, etc.)
Turn Order/Size
Turn Order
Not fixed yet it’s not random
Bias: Speaker before the current speaker is selected as the next speaker
Turn Size
Not fixed
Why?
Because of unit types (single-word turns, single phrase turns, etc.)
Because of rule 1C (current speaker can continue) the speaker can produce more than one unit type
11/19/2018

7. More on Turns
The length and content of conversations are not specified in advance
Anyone can be the next speaker
Number of parties can vary.
With 4 you can have multiple conversations
Turns begin at “possible completion points”
Repairs exist for errors:
Who me?
Excuse me?
11/19/2018

8. Continuous or Discontinuous
Talk can be continuous or discontinuous
Continuous = minimum gap or overlap
Discontinuous = a current speaker has stopped and no speaker starts, and non-talk is a lapse.
J: Oh I could drive if you want me to.
C: Well no I’ll drive (I don’ m//in’).
J: hhh.
(1.0)
J: I meant to offah.
(16.0)
J: Those shoes look nice when you keep on putting stuff on ‘em.
C: Yeah I ‘ave to get another can cuz cuz it ran out….
11/19/2018

9. Turn Allocation Techniques: Select Next Speaker
1. “Type of Sequence” parts. (Adjacency pairs).
Examples:
Complaint/denial
Ken: Hey yuh took my chair by the way an’t I don’t think that was very nice.
Al: I didn’ take yer chair, it’s my chair.
Compliment/rejection
A: I’m glad I have you for a friend.
B: That’s because you don’t have any others.
11/19/2018

10. Turn Allocation Techniques: Select Next Speaker (cont…)
2. Repeat parts of prior utterance with a question intonation or one word question.
Ben: They gotta – a garage sale
Lori: Where.
Ben: On Third Avenue.
11/19/2018

11. Turn Allocation Techniques: Select Next Speaker (cont…)
3. Tag question
You know? Don’t you agree?
4. Introduce social identities
Two couples speaking. An invitation is made by a speaker to go to the movies. The next speaker should be a member of the other couple.
11/19/2018

12. Turn Allocation Techniques: Self-Selection
1. Starting First
2. “Second Starters”
R: Hey::, the place looks different
F: Yeah::hh.
K: Ya have to see all ou[r new-*
D: [It does?*
R: Oh yeah
11/19/2018

13. Consequences of the Model
Motivation for listening to all utterances in a conversation
To see if you are the next speaker
If you want to speak, make sure no one else was selected to speak
When a new speaker is selected, the speaker has to perform the second part of the adjacency pair.
Shows he understood the prior turn’s talk as the first part
11/19/2018

14. Today Turn-taking behaviors in human-human conversation
Task/circumstance dependencies
Conversational Analysis
Linguistic/cultural differences
How do we take and give up turns?
Diarization: Automatic Turn Segmentation
11/19/2018

15. Turn-taking Behavior Dialogue characterized by turn-taking
How do speakers know what to say and when to say it?
Conversational partners expect certain patterns of behavior in normal conversation
Pat: You got an A? That’s great!
Chris: Yeah, I’m really smart you know.
Chris: Well, I was just lucky I happened to read the chapter on dialogue systems right before the test. Otherwise I never would have squeaked through.
General patterns in ordinary conversation
Deviation is significant
11/19/2018

16. Children learn turn taking within first 2 years (Stern ’74)
General individual differences
Shy people pause longer and speak less and less often (Pilkonis ’77)
Schizophrenics, neurotics, depressed people less skilled in turn-taking
11/19/2018

17. Expectations of What to Say May Depend on Task at Hand
Telephone
Openings
Pat: Hello?
Chris: Hi, Pat. It’s Chris.
Pat: Hi!
Closings (6-turn)
Chris: Well, I just wanted to see how you were doing
Pat: Thanks for calling. We'll have to have lunch sometime
Chris: I'd like to
Pat: Okay
Chris: Okay
Pat: See you
Chris: Yeah, see you
11/19/2018

18. Email Service encounters Meetings News broadcasts
Pat: “Hi, can we switch lunch to 12:30? I’m running late.”
Chris: “Sure. 12:30.”
Pat: “Great. See you.”
Service encounters
Clerk: Good morning. Is there something I can help you with?
Pat: Hi. Yeah. I wonder if you could show me….
Meetings
Boss: Today I want to focus on next year’s goal statements. Chris, could you report please….
Chris: …
Boss: Pat, now let’s hear from you…
Pat: …
News broadcasts
Anchor: …Chris Smith reports from Rome now on the upcoming conclave. Chris?
Reporter: Thanks, Pat….. And now back to Pat Jones in New York.
11/19/2018

19. Today Turn-taking behaviors in human-human conversation
Task/circumstance dependencies
Conversational Analysis
Linguistic/cultural differences
How do we take and give up turns?
Diarization: Automatic Turn Segmentation
11/19/2018

20. Conversational Analysis (Sacks et al ’74)
Can we characterize expectations of ‘what to say’ more generally?
‘Rules’ of turn-taking
If, during this turn the current speaker has selected A as the next speaker, then A must speak next
If the current speaker does not select the next speaker, any other speaker may take the next turn
If no one else takes the next turn, the current speaker may take the next turn
Rules Apply at Transition Relevant Places (TRPs) where something allows speaker changes to occur
11/19/2018

21. Where Can Speaker Shifts Occur
Adjacency pairs
Question/answer
Greeting/greeting
Compliment/downplayer
Dispreferred responses
Silence
‘No’ to a simple request without explanation
Changing the topic abruptly without transition
Important for Spoken Dialogue Systems
11/19/2018

22. Today Turn-taking behaviors in human-human conversation
Task/circumstance dependencies
Conversational Analysis
Linguistic/cultural differences
How do we take and give up turns?
Diarization: Automatic Turn Segmentation
11/19/2018

23. Cultural Differences in Turn-Taking
Chinese telephone conversations
Openings (Zhu ’04)
Mandarin vs. British
Identification differences
British self-report
Chinese callees ask the caller
Closings (Sun ’05)
39 female-female telephone conversations
Closings initiated through matter-of-fact statement of intention to end conversation
Verbalized thanking occurs except in mother/daughter closings – not the standard English model
Finnish business calls (Halmari ’93) vs. American
Americans get right to the point
Finns chat
11/19/2018

24. Today Turn-taking behaviors in human-human conversation
Task/circumstance dependencies
Conversational Analysis
Linguistic/cultural differences
How do we take and give up turns?
Diarization: Automatic Turn Segmentation
11/19/2018

25. Individual Differences: British Politicians (Beattie ’82)
Data: 25m televised interviews before 1979 British General election
Margaret Thatcher (Tory leader): the Iron Lady
Jim Callaghan (Prime Minister): Sunny Jim
Who interrupts?
Less intelligent, highly neurotic, extroverted
Men interrupt women
Interruptions may indicate
Desire for dominance
Desire for social approval
Conveyance of ‘joint enthusiasm’, heightened involvement
11/19/2018

26. Method: Identify spkr 2 attempts to take the turn
Smooth switches: no simultaneous speech, spkr 1’s utterance complete, turn to spkr 2
Simple interruptions: simultaneous speech, spkr 1 doesn’t complete utterance, turn to spkr 2
Overlap: simultaneous speech, spkr 1 completes utterance, turn to spkr 2
Butting-in: simultaneous speech but no change of turn, spkr 1 keeps the turn
Silent interruption: spkr 1’s utterance incomplete, no simultaneous speech, turn to spkr 2
11/19/2018

27. Analyze acoustic/prosodic and gestural information
Turn-yielding behavior
Pauses
Speaking rate slows
Drawl at end of clause
Drop in pitch or loudness
Completion of syntactic clause
Gesture of termination
Attempt suppression signals
Filled pauses
Gestures
11/19/2018

28. Results
Mrs. Thatcher interrupted almost twice as often as she interrupts interviewer (19/10)– unlike Callaghan (14/23)
Thatcher: Starts slow and gets faster, few FPs (4)
Callaghan: starts fast and gets slower, many FPs (22)
Public perception: Thatcher is domineering in interviews and Callaghan is a ‘nice guy’
But Thatcher does not dominate
Why is Thatcher interrupted?
Interruptions come at end of syntactic clause when drawl on stressed syllable in clause and falling intonation
11/19/2018

29. No suppression signals Why does she do this?
Speech training before election?
Why is she still perceived as domineering?
When interrupted she doesn’t cede the floor despite lengthy stretches of simultaneous speech
11/19/2018

30. Today Turn-taking behaviors in human-human conversation
Task/circumstance dependencies
Conversational Analysis
Linguistic/cultural differences
How do we take and give up turns?
Diarization: Automatic Turn Segmentation
11/19/2018

31. Diarization: Automatic Speaker Identification/Segmentation
Segment audio corpora (Broadcast News, meetings, telephone conversations) into speaker segments
Speaker segmentation
Speaker identification
Speech and music
Speaker segmentation (Diarization)
Initial segmentation
Segment clustering based on acoustic features
State-of-the-art: 8.47% error
11/19/2018

32. Speaker identification
Linguistic information to identify speaker types and speaker names (LIMSI ’04)
Templates (“<name> has this report from <location>”)
Results: 10.9% error on test set
But only 10% of segments contain relevant patterns
Estimate 25% error on broadcast news if segmentation and clustering is done to id all of each speaker’s segments
11/19/2018

33. <DOCNO> CNN19980104.1130.0000 </DOCNO>
<DOCTYPE> MISCELLANEOUS TEXT (automatic initial) </DOCTYPE>
<DATE_TIME> 01/04/ :30:00.00 </DATE_TIME>
<BODY>
<TEXT>
</TEXT>
</BODY>
<END_TIME> 01/04/ :30:34.71 </END_TIME>
</DOC>
<DOCNO> CNN </DOCNO>
<DOCTYPE> NEWS STORY </DOCTYPE>
<DATE_TIME> 01/04/ :30:34.71 </DATE_TIME>
in northern kentucky are forcing 3,000 people in two states to flee their
homes.
the fire started early this morning at the cargill company plant in
maysville near the ohio river.
authorities have been going door-to-door advising people in kentucky and
ohio to take shelter in area high schools.
the fire is in a building where several fertilizers and chemicals are
stored.
11/19/2018

34. officials say all they can do is let the fire burn itself out, because
spraying water on the flames would be too dangerous.
<TURN>
at the current time, our only way of getting it under control is to stay
away from it.
we've backed everyone off from the fire by about a mile and a quarter and
evacuated homes in that radius and the chief threat at this point is a very
small risk of a very large explosion caused by 400 tons of ammonia nitrate
stored in the building.
foir people have been taken to hospitals.
one firefighter was injured and treated on the scene.
</TEXT>
</BODY>
<END_TIME> 01/04/ :31:31.00 </END_TIME>
</DOC>
<DOC>
<DOCNO> CNN </DOCNO>
<DOCTYPE> NEWS STORY </DOCTYPE>
<DATE_TIME> 01/04/ :31:31.00 </DATE_TIME>
<BODY>
<TEXT>
authorities in brooklyn, new york, say an explosion at a tire company has
11/19/2018

35. caused at least three buildings to collapse.
it set off a four-alarm fire, which has been contained.
officials tell cnn one person was injured.
investigators have not determined the cause of the incident.
</TEXT>
</BODY>
<END_TIME> 01/04/ :31:48.11 </END_TIME>
</DOC>
<DOC>
<DOCNO> CNN </DOCNO>
<DOCTYPE> NEWS STORY </DOCTYPE>
<DATE_TIME> 01/04/ :31:48.11 </DATE_TIME>
<BODY>
<TEXT>
unexpected weather conditions are the rule across much of the united states
this weekend.
angela astore reports.
<TURN>
<ANNOTATION> Reporter: </ANNOTATION>
it was a nice day to play along the beach -- spend a few hours fishing --
or get in a game of golf -- not uncommon -- unless it's january in chicago.
record high temperatures were set yesterday from minnesota to
massachusetts.
warm air drawn northward from the gulf of mexico was behind the rise in the
mercury.
11/19/2018

36. it was a different scene in the northwest, where snow is the story.
but the winter weather didn't stop this man from getting in some warmer
pursuits.
and he wasn't bothered by the fact that he couldn't see where his golf
balls landed.
<TURN>
it's not really where it's going to land that's important at this point
while you are learning.
once you've learned, then it is.
we'll worry about that when the snow clears.
right now, it's probably better that i don't see where they land.
11/19/2018

37. Next Class Components of SDS: Automatic Speech Recognition
Text-to-Speech
Natural Language Understanding
11/19/2018