Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender,

Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender, Ivana Kruijff-Korbayova, and Nick Hawes Kim, Jong In Oct,2, 2015 College of Interdisciplinary Program in Cognitive Science Seoul National University

Contents 8.4. Talking about What You Can See(Small Scale) What you see and what you mean What you All know, and What you are Saying Using What you see to Rank Alternative Interpretations Referring to What you see 8.5. Talking about places You can Visit(Large Scale) Talking about places Representing Places to Talk about Referring to Elsewhere Determining the Appropriate contrast set Understanding References to Elsewhere 8.6 Talking about things you can do 8.7. Conclusions 2

Overview 3

Talking about What you Can see Talking about What You Can See 1) A small-scale space or closed context 2) Perception – Symbol 3) Grounding – determines the meaning of the linguistic symbol 4) Old information(common ground) vs New information (Guide Learning) “The red mug is Big” 4

Translation of Spatial expressions. 6 Talking about What you Can see

Talking about What you Can see What you see and What you mean 1) Semantic – Ontologically Sorted 2) Delimitation, Quantification (Grounding) 3) Dealing with instances, update 4) Evaluation whether the new information is added Motivation Working memoryBinding Working memory Prompt to take action Intentional contents Operating modality Visuo-Spatial information Indexical content

Talking about What you Can see Using What you see to Rank Alternative Interpretations :assign a score to each possible semantic interpretation of a given spoken input 1) Linguistic feature -The acoustic feature (ASR score ) -The syntactic Level (Parsing) -The semantic Level (Logical form) 1) Contextual Feature -Situated context(the objects in the visual scene) -Dialogue context(previously referred entities in the dialogue history) 2) Current data set -195 individual utterance -Check exact match, partial match, error match -Depending on Grammar relaxation, Activated feature

Talking about What you Can see Using What you see to Rank Alternative Interpretations

Talking about What you Can see 1) Indexical Ambiguity 2) “Put the ball near the mug to the left of the box” Using What you see to figure out what is meant

Talking about What you Can see 1) Robot could refer to objects and the spatial relations 2) Q1. If the number of Object is too much? 3) Problem) Inter-object Spatial Relation 4) Define the set of objects –function as landmark – Referring to What you see(small scale)

Talking about Places you can visit Talking about Places(Large Scale) 1) How people tend to employ many different strategies to introduce new location to Robot? (“Space which cannot be perceived at once”) 2) How human presents a familiar indoor environment to a robot (Guided tour scenario) -Human guided the robot around and names and objects

Talking about Places you can visit Talking about Places(Large Scale) Strategies to introduce new locations. 1) Naming whole rooms (“this it the kitchen” –referring to the room itself) 2) Naming specific locations (“this is the kitchen” –referring to cooking area) 3) Naming specific locations by the objects (“this is the coffee machine”) ->Personalizing the representati on of the environment that robot Constructs)

15 Multi-Layer Spatial Map 1)The first layer of SLAM SLAM: Simultaneous localization and mapping 1) Abstract form. 2) Topological segmentation is represented by the coloring of the nodes 3) In order to determine category of an area, we take a majority vote approach of the classification 1) On the highest level of Abstraction 2) Spatial unit(e.g. room) ->human concept(e.g. kitchen)

Overview 16 Conceptual Map is represented as a Description logic ontology -conceptual taxonomy (hand-written commonsense ontololgy representing various aspects of indoor environ ment) -a storage of instances (T-Box and A-Box of a Description Logics reaso ning framework)

Talking about Places you can visit Referring to Elsewhere 1.“the kitchen around the corner” 2.“the red mug left of the china plate” 3.“Peter’s office” 4.“the large hall on the first floor” Communicate goal issue. 1) Robot are good at measuring exact distances, but humans are not 2) Infinite recursion 3) The robot might have a vast knowledge but have to separate uniquely the referent from all entities.

Talking about Places you can visit Issue1) Determining proper contrast Set 1) If contrast set is too much? 2) if contrast set is too little? Issue2) Robot viewpoints? 1) Exact measure or Topological Abstraction -The context for a dialogue situated in large-scale space can be determined on the basis of a topological representation like human Determining the Appropriate Contrast set

8.6 Talking about things you can do 1) Action Planning: the goal of action planning is to choose actions and ordering relations among these actions to achieve a suitable location and plans for executing the action - Action planning and Dialogue processing should interact. 2) Event nucleus -it models the action as an event with temporal and casual dimension 3) Indexicality, intentionality, and information structure “First take the mug, and put it in near the plate” Things you can do

Small Scale Guide Tour -Ambiguity, Semantic Grounding -Motivation working memory -Binding working memory Large Scale Guide Tour -Multi-Layered Spatial mapping -topological map, conceptual map Action -Action planning, Summary

Debate issue 1. How to solve ambiguity problem in Situated Dialogue. 2. Drone and Robot cooperation

APPENDIX( 질문 대비 )

28 Ambiguity

Topological Partitioning © 2015, SNU CSE Biointelligence Lab., http://bi.snu.ac.kr 29 Solution> Topological Partitioning -Large red stars indicate doorways and the different coloring of the nodes depicts the topological partitioning of the environment

30 Ambiguity

31 CCG grammar

Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender,

Similar presentations

Presentation on theme: "Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender,"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender,

Similar presentations

Presentation on theme: "Chapter 8. Situated Dialogue Processing for Human-Robot Interaction Greert-Jan M. Kruijff, Pierre Lison, Trevor Benjamin, Henril Jacobsson, Hendrik Zender,"— Presentation transcript:

Similar presentations

About project

Feedback