Download presentation
Presentation is loading. Please wait.
1
Heterogeneous Multi-Robot Dialogues for Search Tasks Thomas K Harris, Satanjeev (Bano) Banerjee Alexander Rudnicky AAAI Spring Symposium 2005: Dialogical Robots
2
AAAI Spring ’05 Symposium: Dialogical Robots2 Communication among autonomous robots and humans → Embodied (not necessarily robotic, or even anthropomorphic) agents will become ubiquitous → NL dialogue will be a useful modality → Agent’s dialogue and task models will remain tightly coupled, and independent from other agents We will be talking to multiple agents without a (completely) shared dialog system
3
AAAI Spring ’05 Symposium: Dialogical Robots3 Gedankenexperiment The problem of many robots: sweeper, furniture mover, baby monitor. Task: clean up house; don’t wake baby up. Who do you talk to? What do you say? Who talks back?
4
AAAI Spring ’05 Symposium: Dialogical Robots4 Issues for polylogical systems What’s a reasonable architecture? –A single system controlling multiple robots –A single DM directing autonomous robots –Shared understanding/generation –Shared microphone and speakers –Robots self-contained: Nothing shared How to communicate with multiple agents? –Directions to the entire team? (“talk into the air”) –Mediation through a designated team leader? (foreman) –Instructions to each team member? (direct management) –To a proxy agent within/without the team? (personal assistant)
5
AAAI Spring ’05 Symposium: Dialogical Robots5 What do we have today? A platform that supports multiple (robot) participants –Sphinx II ASR; Festival TTS –RavenClaw dialogue manager –Galaxy-II message-passing architecture –Several “back-end” interfaces A basic corridor movement domain –Commands to move robots along in vectors and towards named locations –Mechanisms for describing robot status and location to human interlocutor
6
AAAI Spring ’05 Symposium: Dialogical Robots6 AGENT2 AGENT1 HUMAN INTERFACE Current Architecture ASR Multi-voice TTS DM1 DM2 Robot1 Robot2 Interpret1 Interpret2
7
AAAI Spring ’05 Symposium: Dialogical Robots7 AGENT2 AGENT1 HUMAN INTERFACE Implementation Sphinx IIPhoenixHelios Galaxy Ravenclaw Bashful Ravenclaw Clyde Bashful Robot Communication System RosettaSwift
8
AAAI Spring ’05 Symposium: Dialogical Robots8 Human-Robot Treasure Hunt One Human Two robots –Segway –Pioneer Treasures “hidden” in large space TASK: Retrieve all treasures
9
AAAI Spring ’05 Symposium: Dialogical Robots9 Scenario for Search We found it! We are at
10
AAAI Spring ’05 Symposium: Dialogical Robots10 Issues in H-R communication 1. How do people decompose the task into sub-tasks? 2. What language do people use to get the tasks performed by the robots? 3. Given a human command, what is the expected robot behavior? Explore using Wizard of Oz experiments
11
AAAI Spring ’05 Symposium: Dialogical Robots11 WOZ design Communication takes place by normal speech, walkie- talkie, and through the webcam. 1 experimenter, 2 robot actors, 1 participant. Experimenter places treasure, simulates robot treasure sensors. 1st robot actor is blind, but carries a webcam for the participant’s consumptions. 2nd robot actor can only move by crawling.
12
AAAI Spring ’05 Symposium: Dialogical Robots12 Annotation and analysis Data transcribed and annotated Utterances classified into functional categories 394 utterances 20 utterance categories 8 major categories Carnegie Mellon MockBrow annotation tool
13
AAAI Spring ’05 Symposium: Dialogical Robots13 Utterance/Task Breakdown Controlling team behaviors Controlling team behaviors Grounding Grounding Positive/negative feedback Informing robot of it’s state or the world Explanations of commands Orientation Grounding Navigation Navigation Simple Navigation commands Spatial Referential Navigation Object Referential Navigation Manipulation Manipulation Manipulating the environment Manipulating treasure Coverage Coverage Manipulating the webcam view Object coverage commands Generic coverage Asking about the robot’s abilities Asking about the robot’s abilities Filler Filler Real-time command modifications Real-time command modifications
14
AAAI Spring ’05 Symposium: Dialogical Robots14 Summary / Future Work Architecture for human-robot teams WoZ study of language requirements Different WoZ scenarios Implement language for robot team “Field” testing
15
AAAI Spring ’05 Symposium: Dialogical Robots15 Controlling team behaviors “you guys get together” “T- you go first and B- follow”
16
AAAI Spring ’05 Symposium: Dialogical Robots16 Grounding Positive/negative feedback “ok that’s better” Informing robot of state “so that’s up” “I don’t see anything there” Explanations of commands “so I can see which direction is up” Orientation Grounding “What you’re facing now with the camera – is that the vehicle that you just circumnavigated” “I can tell you’re going in the wrong direction, stop”
17
AAAI Spring ’05 Symposium: Dialogical Robots17 Navigation Simple Navigation commands “so um T- turn to you left” “T- I want you to turn right 90 degrees” “can you go in that general direction” “can you proceed in that direction” Spatial Referential Navigation “go to that open area” “continue around the periphery of that open area” “back out of that alley” “proceed in that direction until you find an opening to turn left” Object Referential Navigation “go over by T-” “can you go on the other side of that vehicle” “go over by the posters”
18
AAAI Spring ’05 Symposium: Dialogical Robots18 Manipulation Manipulating the environment “T- why don’t you move the trash can” Manipulating treasure “T- bring the coin to me” Manipulating the webcam view “ok B- look to your left” “B- can you look around with the camera a little”
19
AAAI Spring ’05 Symposium: Dialogical Robots19 Coverage Object coverage commands “ok so examine the shelf” “do you see something on that shelf in front of B-” “can you look over by that table over there” Generic Coverage “do you see anything that looks interesting”
20
AAAI Spring ’05 Symposium: Dialogical Robots20 Asking about the robot’s abilities “is that possible”
21
AAAI Spring ’05 Symposium: Dialogical Robots21 Filler “and now um” “ok um”
22
AAAI Spring ’05 Symposium: Dialogical Robots22 Real-time command modifications “keep going” “stop” “a little more” “change of plans” “other direction”
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.