1. cse@buffalo S.C. Shapiro Symbol Anchoring in a Grounded Layered Architecture with Integrated Reasoning Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo shapiro@cse.buffalo.edu

2. cse@buffalo S.C. Shapiro Based on Stuart C. Shapiro & Haythem O. Ismail, “Anchoring in a grounded layered architecture with integrated reasoning,” Robotics and Autonomous Systems 43, 2-3 (May 2003) 97-108.

3. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

4. cse@buffalo S.C. Shapiro Definition “Anchoring is the problem of connecting, inside an artificial system, symbols and sensor data that refer to the same physical objects in the external world.” [Silvia Coradeschi & Alessandro Saffiotti]

5. cse@buffalo S.C. Shapiro Embodied Cassie A computational cognitive agent –Embodied in hardware –or Software-Simulated –Based on SNePS and GLAIR.

6. cse@buffalo S.C. Shapiro Cassie, the FEVAHR (Foveal ExtraVehicular Activity Helper-Retriever)

7. cse@buffalo S.C. Shapiro FEVAHR/Cassie in the Lab

8. cse@buffalo S.C. Shapiro Crystal Cassie First Person Perspective Views

9. cse@buffalo S.C. Shapiro Patofil and Filopat from “The Trial, The Trail” A VR drama by Josephine Anstey et al.

10. cse@buffalo S.C. Shapiro DeliveryAgent Using Byron Weber Becker’s Java version of Rich Pattis’ Karel the Robot

11. cse@buffalo S.C. Shapiro Magellan Pro TM Mobile Robot from iRobot

12. cse@buffalo S.C. Shapiro SNePS Knowledge Representation and Reasoning –Intensional Representation –Propositions as Terms SNIP: SNePS Inference Package –Specialized connectives and quantifiers SNeBR: SNePS Belief Revision SNeRE: SNePS Rational Engine (Acting Language) Interface Languages –SNePSUL: Lisp-Like –SNePSLOG: Logic-Like –GATN for Fragments of English.

13. cse@buffalo S.C. Shapiro Interaction with Cassie English (Statement, Question, Command) (Current) Set of Beliefs [SNePS] (Updated) Set of Beliefs [SNePS] Actions [SNeRE] (New Belief) [SNePS] English sentence expressing new belief answering question reporting actions Answer [SNIP] GATN Parser GATN Generator Reasoning Clarification Dialogue Looking in World Reasoning

14. cse@buffalo S.C. Shapiro Entities, Terms, Symbols, Objects Cassie’s mental entity: a person named Stu SNePS term: B4 Object in world:

15. cse@buffalo S.C. Shapiro GLAIR Architecture Knowledge Level Perceptuo-Motor Level Sensory-Actuator Level NL Vision Sonar MotionProprioception Grounded Layered Architecture with Integrated Reasoning SNePS Mind BodyBody World IP Sockets

16. cse@buffalo S.C. Shapiro Alignment KL Body (PML/SAL) World SNePS term PML structure Object/PhenomenonAction

17. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

18. cse@buffalo S.C. Shapiro World Objects to Feature Tuples <Height, Width, Texture,.. > WorldPML/SAL

19. cse@buffalo S.C. Shapiro Feature Tuples to KL Terms <Height, Width, Texture,.. > PML/SALKL ProperName(B4, Stu) Alignment

20. cse@buffalo S.C. Shapiro Incomplete PML-Descriptions <Height, nil,.. > PML/SALKL Height(B4, B12)

21. cse@buffalo S.C. Shapiro Unifying PML-Descriptions PML/SAL KL B20 B30 B31 B6 Isa Prop

22. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

23. cse@buffalo S.C. Shapiro Deictic & Modality Registers for being situated in the world I You Now Vision. PML KL Terms denoting Cassie Addressee Current time Current state(s) of looking at x.

24. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

25. cse@buffalo S.C. Shapiro Primitive Actions Aligned with PML/SAL Functions PMLKL M2(B4) Find(B4) SAL Modality Registers World A KL symbol is a pivot coordinating different modalities.

26. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

27. cse@buffalo S.C. Shapiro Aligning NOW using MTF NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states Maximal Temporal Frame based on *NOW KL PML

28. cse@buffalo S.C. Shapiro Moving NOW with MTF NOW KL PML Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers.

29. cse@buffalo S.C. Shapiro Providing a Feel for Time NOW COUNTn hom 0 KL PML t1 t2 q ! beforeafter time duration !

30. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

31. cse@buffalo S.C. Shapiro Aligning Lexemes/NL “Stu” PML/SALKL ProperName(B4, Stu) Isa(B4, person) “person” ctgy npr ctgy n num sing Grammar NL World

32. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

33. cse@buffalo S.C. Shapiro Acting 1

34. cse@buffalo S.C. Shapiro Acting 2 I found a red robot. I am looking at a red robot. Follow a red robot.

35. cse@buffalo S.C. Shapiro Acting 3 I went to a red robot. I am near a red robot. I am following a red robot. I found a red robot. I am looking at a red robot. Follow a red robot.

36. cse@buffalo S.C. Shapiro Use of Indexicals 1 Come here. I found you. I am looking at you.

37. cse@buffalo S.C. Shapiro Come here. I came to you, Stu. I am near you. Use of Indexicals 2 I found you. I am looking at you.

38. cse@buffalo S.C. Shapiro Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. Use of Indexicals 3

39. cse@buffalo S.C. Shapiro Come here. I found you. I am looking at you. Use of Indexicals 4

40. cse@buffalo S.C. Shapiro Come here. I came to you. I am near you. I found you. I am looking at you. Use of Indexicals 5

41. cse@buffalo S.C. Shapiro Who am I? I talked to Stu and I am talking to you. Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? Use of Indexicals 6

42. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

43. cse@buffalo S.C. Shapiro Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. PML attentional structures anchor symbols by changing only when motivated.

44. cse@buffalo S.C. Shapiro Summary from Jackendoff Ray Jackendoff, Foundations of Language, Oxford University Press, 2002, Fig. 11.1, p. 348. (Independent work)

45. cse@buffalo S.C. Shapiro For More Information Personnel Manual Tutorial Bibliography ftp’able SNePS source code etc. http://www.cse.buffalo.edu/sneps/