1. Learning Prepositions for Spatial Relationships in BOLT Soar Workshop 2012 James Kirk, John Laird 6/21/2012 1

2. Outline Prepositions in BOLT Compositional Spatial Relationships Learning Process in Architecture with SVS Projection Conclusions 2

3. Prepositions/Spatial Relationships in BOLT “What is to the right of the pantry?” “Put the triangle in the stove.” “Pick up the red object to the right of the pantry.” 3

4. Spatial Relationships as Compositions y-aligned z-aligned x-greater than 4 Z Y X “right of” “Intersecting” y-aligned z-aligned x-aligned “Above” z-greater than Other potential compositions: “In front of” y-less than z-aligned x-aligned

5. Learn new prepositions that map to spatial relations “The red triangle is right of the blue sphere” 5

6. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 1 Perception - Receive object information from input link - Create internal representations Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Human: “The red triangle is right-of the blue sphere” 6 Word – Category Mapping

7. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 2 Message Parsing - Parse the given sentence - Create usable message structures “ The red triangle is right-of the blue sphere ” Working Memory LGSoar Obj1 - bl3 - sp7 Obj2 -re23 - tr17 Obj3 - bl3 - sq8 relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere Human: “The red triangle is right-of the blue sphere” 7 Word – Category Mapping

8. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 3 Interaction Stack - Designate message purpose as preposition learning Working Memory LGSoar relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere Interaction Prep. Learning Obj1 - bl3 - sp7 Obj2 -re23 tr17 Obj3 - bl3 - sq8 Human: “The red triangle is right-of the blue sphere” 8 Word – Category Mapping

9. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning Working Memory Interaction relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere relation-message ^word right-of ^o1 red, triangle ^o2 blue, sphere Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning Prep Learning right-of red, triangle blue, sphere Human: “The red triangle is right-of the blue sphere” 9 Word – Category Mapping

10. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - Lookup the category of red from smem Working Memory Noun Learning red color Prep Learning Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of red, triangle blue, sphere Human: “The red triangle is right-of the blue sphere” Indexing 10

11. Noun/Adjective – Perceptual Symbol Mapping Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - Lookup the symbol for red + color from smem Working Memory red color Noun/Adjective – Perceptual Symbol Map re23 Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of red, triangle blue, sphere re23, triangle Human: “The red triangle is right-of the blue sphere” Indexing 11 Word – Category Mapping

12. Noun/Adjective – Perceptual Symbol Mapping Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - Update the descriptions with perceptual symbols Working Memory Noun/Adjective – Perceptual Symbol Map Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of red, triangle blue, sphere re23, triangle re23, tr17 bl3, sphere bl3, sp7 Human: “The red triangle is right-of the blue sphere” Indexing 12 Word – Category Mapping

13. Noun/Adjective – Perceptual Symbol Mapping Perception LGSoar Interaction Indexing Verb Learning Noun Learning Prep Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - Look for the objects matching the description Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of re23, tr17 bl3, sp7 Obj2 Obj1 Human: “The red triangle is right-of the blue sphere” Indexing 13 Word – Category Mapping

14. Obj2 Obj1 Perception LGSoar Interaction Indexing Verb Learning Noun Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - SVS updates model of objects in the world from the perception input. Human: “The red triangle is right-of the blue sphere” Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of Obj1Obj2 Obj3 Spatial Primitives Prep Learning 14 Word – Category Mapping

15. Obj2 Obj1 Perception LGSoar Interaction Indexing Verb Learning Noun Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - SVS is queried for primitive spatial relationships between Obj1 and Obj2 Human: “The red triangle is right-of the blue sphere” Prep Learning Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning right-of Obj1 Obj2 Obj3 Obj2 Obj1 ? Spatial Primitives 15 Word – Category Mapping

16. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning -The results of the query are stored in working memory. Human: “The red triangle is right-of the blue sphere” Prep Learning y -aligned z-aligned +x Working Memory right-of Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 Prep. Learning 16 Word – Category Mapping

17. Perception LGSoar Interaction Indexing Verb Learning Noun Learning Action Knowledge Procedural Memory Preposition – Spatial Rel Mapping Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - List of spatial primitives are abstracted into general spatial relationship and mapped to “right- of.” Human: “The red triangle is right-of the blue sphere” Prep Learning Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 SR11 y-aligned z-aligned +x right-of Prep. Learning 17 Word – Category Mapping

18. Preposition – Spatial Rel Mapping Preposition – Spatial Rel Mapping Perception LGSoar Interaction Indexing Verb Learning Noun Learning Action Knowledge Procedural Memory Verb – Operator Mapping Noun/Adjective – Perceptual Symbol Mapping Semantic Memory Primitive Verb – Operator Mapping Episodic Memory Agent’s Experiences Spatial Visual System Spatial Primitives Action Phase 4 Preposition Learning - Learned spatial relationship mapping to “right-of” is stored in semantic memory. Human: “The red triangle is right-of the blue sphere” Prep Learning Working Memory Obj1 - bl3 - sp7 Obj2 -re23 -tr17 Obj3 - bl3 - sq8 SR11 y-aligned z-aligned +x right-of Prep. Learning 18 Word – Category Mapping

19. Refining learned knowledge Is this triangle also to the right of the square? 19 Disjunctive primitive relationships in training are removed.

20. Learning is interactive Learning can be initiated by instructor or agent “What is behind the blue sphere?” -“I don’t know the preposition behind. Please teach me with examples.” “The green square is behind the red triangle.” “Finished teaching.” - “The blue square is behind the blue sphere.” 20

21. Spatial Projection “Put the object to the right of the pantry.” 21 Z Y X “right of”

22. Current Uses Describing the scene Specifying objects in the scene Tracking objects and relationships – “The goal is that the red triangle is inside the pantry” Projecting relationships back to a point in 3D space to perform actions 22

23. Evaluation Nuggets Learns very fast Works well for simple relationships and projection/tracking Coals Cannot learn complex relationships with many objects Cannot learn certain types of relationships such as diagonal Not robust to mistakes in training examples 23

24. Future Work Complex multi-object relationships – For example in a line or a circle More complex spatial relationships – Using distance/contacting etc. – Pattern matching (diagonal) – Will require more training examples Projection with multiple relationships – “put this inside the pantry and to the right of the blue object” 24

25. Questions? 25