Initiative in Dialogue (draft) Mark Core

An Informal Definition OED.com: to take the initiative: to take the lead, make the first step, originate some action Sometimes referred to as “control” dialogue initiative identifies who is leading the dialogue at any given point

Why do people study initiative? Dialogue System Builders for some applications, system may need to reason about whether to take the initiative systems need to take and give away initiative smoothly like humans system acts differently when it does and does not have initiative Linguists open question: what factors influence when dialogue participants take initiative

Outline Defining and Annotating Initiative Discourse Structure Whittaker and Stenton Strayer and Heeman Linell et al. What is Known about Initiative Initiative and Current Dialogue Systems Going Beyond Initiative

Discourse Structure and Initiative “dialogue initiative... tracks the lead in determining the current discourse focus” (Chu-Carroll and Brown 1997, p. 263) topic-based segmentation – who initiated the topic. Grosz and Sidner (1986) dialogue moves – who produced an initiating move.

Grosz and Sidner (1986) Discourse grammar (syntax) Discourse => Discourse_Segment + Discourse_Segment => utterance + Discourse grammar (semantics) there is one unique ‘foundational’ purpose to the dialogue and each discourse segment meant to be recognized by listener the discourse segment purposes (DSPs) support the discourse purpose utterances support discourse support the DSP

Structural Relations dominance: discourse is hierarchical. one DSP may be intended to provide part of the satisfaction of another satisfaction-precedence: order of DSPs may be significant E: First you have to remove the flywheel.... loosen the two allen head setscrews... A: I can only find one screw. Where’s the other one?.... A: I can find them both now E: Use the wheelpuller. Do you know how to use it?... E: The wheel should slide off [p. 186] DS1 DS2 DS3

Attentional State focus spaces – contain salient entities (mentioned explicitly or implicitly) Entities could be properties, objects, relations, as well as the discourse segment purpose Let’s just look at objects

Focus Space Stack E: First you have to remove the flywheel.... loosen the two allen head setscrews... A: I can only find one screw. Where’s the other one?.... A: I can find them both now E: Use the wheelpuller. Do you know how to use it?... E: The wheel should slide off DS1 DS2 DS3 flywheel screw1 screw2 FOCUS SPACE STACK screw1 screw 2 wheel puller DS1 DS2 DS3

Dialogue Moves and Games Dialogue Moves also called speech acts or dialogue acts capture intention behind an utterance examples initiating: e.g., command, statement, question responding: acknowledgement, answer, accept, reject Dialogue Games start with an initiating move and end when the goal of that move is accomplished or abandoned games can be nested

[1] T: Do you still believe [your answer]? [2] S: I suppose not. [3] But doesn’t I=V/R? [4] T: Yes. That is true. [5] S: because V=IR [6] S: So why doesn‘t I = 75 / the resistence? [7] T: If P=IV, then you know that I = P/V. [8] I can only equal P/R if R=V. [9] Do you have a reason to believe that R=V? [10] S: no Dialogue Games Example

Complimentary Approaches Can group dialogue games based on topic In tutoring ‘how to measure current’ we can group dialogue games around the topics: preparing the circuit (turn off power and break circuit) connect leads across break (+ and – leads must be connected correctly)

Whittaker and Stenton (1988) Simpler procedure for initiative assignment Use four dialogue moves Assertions – “Declarative utterances used to state facts” Commands Questions Prompts – “Utterances which did not express propositional content, such as Yeah, Okay, Uh-huh” Initiative assignment Assertion – speaker has initiative unless responding to a question Command – speaker has initiative Question – speaker has initiative unless responding to a question or command Prompt – hearer has initiative

Walker and Whittaker (1990) “third person and one anaphors cross [these] boundaries extremely rarely” p discourse segment boundaries marked by initiative changes

Strayer and Heeman (2001) Compared initiative (annotated with Whittaker and Stenton’s rules) to dialogue structure (as defined by Grosz and Sidner) Whittaker and Stenton rules are more fine-grained than Grosz and Sidner segmentation Forward Acknowledgments: listener takes initiative by completing the speaker’s utterance Other-Contributions: listener and speaker building utterances together Usually initiative goes back to the original speaker after the interjection

Linell et al. (1988) each utterance is ranked based on how much “they can be regarded as governing or steering the ensuing dialogue and as being governed or commanded by the preceding dialogue” p. 419 highest rank (6): not a response in any way but requires a response from the listener lowest rank (2): invite no response and give no more information than required

Whittaker and Stenton (again) Whittaker and Stenton (1988) benefits relatively simple to annotate correlates well with discourse structure gives us insight into “taking” and “giving away” initiative extendable (consider additional dialogue moves) drawbacks does not account for answers giving more information than requested never considers clarification requests as taking initiative redundant if you know discourse structure

Outline Defining and Annotating Initiative Discourse Structure Whittaker and Stenton Strayer and Heeman Linell et al. What is Known about Initiative Initiative and Current Dialogue Systems Going Beyond Initiative

Walker and Whittaker (1990) Distinguish between types of control shifts (i.e., speaker 2 takes initiative after...) Abdication: speaker1 utters a prompt signaling initiative is released Repetition/Summary: speaker1 signals the end of their contribution with a repetition or summary Interruption: speaker1 has not invited speaker2 to take initiative Rationale: listener knows a relevant fact, listener detects a problem in the speaker’s proposal, or listener finds the utterance ambiguous

Walker and Whittaker (1990) FinanceSupportPump-PhonePump-Chat Turns/Seg Exp-Initiative60%51%91% Abdication38% 45%28% Summary23%27%7%6% Interrupt38%36%48%67%

Guinn (1996) used simulated conversations to argue that the most efficient problem-solving dialogues are those where the participant who knows the most about the current subtask takes initiative

Distribution of Expert Initiative DomainExpert Control Finance60% Computer Support51% Pump Repair91% TRAINS % Airline58.1% Maptask86.6% Switchboard59.9% Socratic Tutoring90% Didactic Tutoring79%

Initiative and Learning Gain Pearson’s r = , n=23, NS

Initiative Shifts Chu-Carroll and Brown (1997) Explicit cues Discourse cues silence, repetitions, prompts, questions, obligation fulfilled Analytical cues invalid action/belief, sub-optimality, ambiguity Giving more information than requested

Factors affecting initiative Individual or joint goals? Collaborative or competitive task? Distribution of knowledge/expertise Complexity of task Task-based roles Social roles Social projection (face) Group discussion v. one-on-one Mediation/modality

Outline Defining and Annotating Initiative Discourse Structure Whittaker and Stenton Strayer and Heeman Linell et al. What is Known about Initiative Initiative and Current Dialogue Systems Going Beyond Initiative

Horvitz (1999) reading system (graphical) – based on the content of the should the system (1) bring up the calendar, (2) ask the user first, (3) do nothing utility-based approach – penalties for interrupting user, bringing up calendar unnecessarily, and reward for bringing up the calendar when needed

Simple Speech-based Systems Here, initiative affects the system questions: system initiative: “What city do you want to leave from?” user initiative: “How may I help you?” initiative also affects how the system deals with extra information e.g., S: What city do you want to leave from? U: LA on November 17 th system initiative: ignores extra info mixed/user initiative: processes all information the major factor is speech recognition performance

Tutorial Dialogue Systems Tend to be system initiative although sometimes students can ask questions about definitions: “What is a CPU?” EDGE system (Cawsey 1989) if user question is related to a presentation goal, system will request that the user wait Duke Programming Tutor (Keim et al. 1997) topics picked on basis of believed student understanding importance of topic distance from current topic how many times topic has been discussed student interest

Outline Defining and Annotating Initiative Discourse Structure Whittaker and Stenton Strayer and Heeman Linell et al. What is Known about Initiative Initiative and Current Dialogue Systems Going Beyond Initiative

Tutorial Dialogue Systems (Core et al. 2003) study showed no relationship between initiative and learning Shah (1997) found more student initiative in students’ first sessions Graesser and Person (1994) in the first half of a course, found a negative correlation between number of student questions and exam scores in second half of course, found a positive correlation between exam scores and proportion of student questions that were deep-reasoning and knowledge-deficit questions

Task Initiative Who has initiative can be quite independent of what’s happening with the task (e.g., putting together the water pump, learning physics) (Chu-Carroll and Brown 97) & (Jordan and Di Eugenio 97) If the dialogue concerns a task, it is possible to lead the dialogue but not contribute to the task Chu-Carroll and Brown separate these ideas into dialogue initiative and task initiative

Chu-Carroll and Brown Example S: I want to take NLP to satisfy my seminar course requirement Who is teaching NLP? A1: Dr. Smith is teaching NLP A2: You can’t take NLP because you haven’t taken AI, which is a prerequisite for NLP. A3: You can’t take NLP because you haven’t taken AI, which is a prerequisite for NLP. You should take distributed programming to satisfy your requirement, and sign up as a listener for NLP.

Simple Task Initiative MIMIC (Chu-Carroll and Nickerson 2000) simple spoken dialogue system (for movie information) task initiative means system gives directions e.g., “Please say the name of the movie or theatre or town you would like information about” task initiative means system makes suggestions e.g., “Terminator is not playing at Terminator is playing at at Mixed-initiative MIMIC system outperformed system- initiative MIMIC (better user satisfaction and task efficiency)

Chu-Carroll and Nickerson (2000) Mixed-initiative MIMIC reasons about cues TakeOverTask (user gives more info than requested) NoNewInfo InvalidAction/InvalidActionResolved AmbiguousAction/AmbiguousActionResolved

Future Work “Although a system needs to reason about how helpful it needs to be, it is unclear whether this can be done through a single variable [task initiative] that is tied to dialogue initiative” (Strayer and Heeman p. 7 of pdf) How to apply this principle to more complex domains such as tutoring student model contains all facts about domain student has task initiative if he demonstrates knowledge of such a fact also if student recognizes their own error may want to model hints which to various degrees give away the answer