Dialogue in Intelligent Tutoring Systems Dialogs on Dialogs Reading Group CMU, November 2002

ITS? Goal: help a human learn to perform a task Task Model: models how an expert would perform the goal task Student Model: models how the student currently performs the task + prior knowledge on how students usual perform Teacher Model: usually none …

AutoTutor (U. of Memphis) Textual conversation with an animated agent tutor Originally for Computer Literacy, also for Newtonian Physics and Research Methods Goal: get (long) answers to general, concrete questions and elicit/correct student knowledge e.g.: Suppose a runner is running in a straight line at constant speed, and the runner throws a pumpkin straight up. Where will the pumpkin land? Explain.

Autotutor: Dialogue Selects dialogue moves from: –Positive/negative feedback –Prompts –Hints –… Students can ask Wh- and Yes/No-questions Based on a “ Dialogue Advancer Network ” : FSM that selects the next move according to student ’ s last utterance Latent Semantic Analysis to match student answers with expectations

Autotutor: Comments “ super form filling ” where the system knows the value of the slots beforehand. Mostly system-initiative, no memory (if the student asks a question, the system forgets what it is doing) Global strategy fixed (by system architecture)

ATLAS/ANDES (Pitt) ANDES: ITS for physics, no natural language ATLAS: “add-on” to ANDES, provides Knowledge Construction Dialogues for hints (main task/evaluation is left to ANDES) KCD: recursive FSM –Reactive planner to pick next KCD –Can insert subdialogues (clarification, rectification…) and go back to original topic

ATLAS/ANDES

BEETLE (U. of Edinburgh) Fully plan-based tutorial dialogue: –Top tier: global strategy/repair when failure –Middle tier: handles specific tasks according to the situation –Bottom tier: performs primitive dialogue actions Teaches basic electricity and electronics Very dialogue-oriented but not completely implemented yet

PACO: Pedagogical Agent for Collagen (USC, Mitsubishi, MITRE) Simulation-based training Domain-independent: adapts to any simulator (e.g. Gas Turbine Engine) Collaborative Discourse Theory-based: –Rules describe interactions between three agents: student, tutor, simulator –Discourse acts: both utterances and domain actions

Stanford’s CSLI System Also simulation-based (Shipboard Damage Control) Complex dialogue management: –tree based activity model (similar to CMU Communicator) built dynamically Separation between dialogue management and tutoring strategy: –Tutoring Module constructs the activity tree using recipes while the Dialogue Manager uses the tree to conduct the dialogue

Simulation-based Systems Comments Combine advanced dialogue architectures with ITS Mixed-initiative dialogue management Really multimodal (click-based simulator, speech-based tutorial dialogue)

CALL System (U. Le Mans) Actional approach to Language Learning: –User must perform a task (cooking) Communicative approach: –Interaction with a partner agent Tutor agent to give instructions/help Specificity of CALL: language is the domain taught, not only a means of teaching

CALL System Based on a theory of Human Computer Dialogue Partner agent: usual dialogue system Tutor agent: must monitor language and dialogue issues Evaluation in terms of efficacy/efficiency: similar to evaluation of dialogue systems (number of turns taken) but for the human!

What can dialogue bring to ITS? Human tutor-like instruction: –Qualitative, natural (cf science) –Helps the student construct knowledge instead of just “ telling ” him/her  deeper understanding (?) Realistic simulation of certain tasks when teaching communicative skills

What characterizes tutorial dialogue? Tutor has expectations about student’s utterances Student must be able to: –Exhibit knowledge –Ask questions (although this does not happen so often…) Open-ended: no specific goal (except teaching) ?

Are standard dialogue systems suitable for tutorial dialogues? Possibilities of state-of-the-art dialogue systems underexploited? –Strategies for repair/elicitation –Confidence measures (integrated with student model?) –Real mixed-initiative: its meaning in the context of tutoring

What can dialogue research learn from ITS? Task modeling Student/User modeling Multi-level planning (long-term strategies, mid-term tactics, short-term actions) more necessary in ITS than anywhere else (pedagogical goals)

Other issues… General lack of cooperation between language technology and ITS researchers? –Natural Language Understanding –Dialogues Both dialogue systems and ITS require heavy human work to create: –how can we derive a task model automatically? Anything else?

Any other comment…