1. cse@buffalo S.C. Shapiro Symbol-Anchoring in Cassie Stuart C. Shapiro and Haythem O. Ismail Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo {shapiro | hismail}@cse.buffalo.edushapiro | hismail}@cse.buffalo.edu

2. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

3. cse@buffalo S.C. Shapiro Embodied Cassie A computational cognitive agent –Embodied in hardware –or Software-Simulated –Based on SNePS and GLAIR.

4. cse@buffalo S.C. Shapiro Cassie, the FEVAHR

5. cse@buffalo S.C. Shapiro FEVAHR/Cassie in the Lab

6. cse@buffalo S.C. Shapiro SNePS Knowledge Representation and Reasoning –Intensional Representation –Propositions as Terms SNIP: SNePS Inference Package –Specialized connectives and quantifiers SNeBR: SNePS Belief Revision SNeRE: SNePS Rational Engine Interface Languages –SNePSUL: Lisp-Like –SNePSLOG: Logic-Like –GATN for Fragments of English.

7. cse@buffalo S.C. Shapiro Interaction with Cassie English (Statement, Question, Command) (Current) Set of Beliefs [SNePS] (Updated) Set of Beliefs [SNePS] Actions [SNeRE] (New Belief) [SNePS] English sentence expressing new belief answering question reporting actions Answer [SNIP] GATN Parser GATN Generator Reasoning Clarification Dialogue Looking in World Reasoning

8. cse@buffalo S.C. Shapiro Entities, Terms, Symbols, Objects Cassie’s mental entity: a person named Stu SNePS term: B4 Object in world:

9. cse@buffalo S.C. Shapiro GLAIR Architecture Knowledge Level Perceptuo-Motor Level Sensory-Actuator Level NL Vision Sonar MotionProprioception Grounded Layered Architecture with Integrated Reasoning SNePS

10. cse@buffalo S.C. Shapiro Alignment KL Body (PML/SAL) World SNePS term PML structure Object/PhenomenonAction

11. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

12. cse@buffalo S.C. Shapiro World Objects to Feature Vectors <Height, Width, Texture,.. > WorldPML/SAL

13. cse@buffalo S.C. Shapiro Feature Vectors to KL Terms <Height, Width, Texture,.. > PML/SALKL ProperName(B4, Stu) Alignment

14. cse@buffalo S.C. Shapiro Incomplete Feature Vectors <Height, nil,.. > PML/SALKL Height(B4, B12)

15. cse@buffalo S.C. Shapiro Unifying PML-Descriptions PML/SAL KL B20 B30 B31 B6 Isa Prop

16. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

17. cse@buffalo S.C. Shapiro Deictic & Modality Registers I You Now Vision. PML KL Terms denoting Cassie Addressee Current time Current state(s) of looking at x.

18. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

19. cse@buffalo S.C. Shapiro Primitive Actions Aligned with PML/SAL Functions PMLKL Find(B4) SAL Modality Registers World

20. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

21. cse@buffalo S.C. Shapiro Aligning NOW using MTF NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states Maximal Temporal Frame based on *NOW KL PML

22. cse@buffalo S.C. Shapiro Moving NOW with MTF NOW KL PML Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers.

23. cse@buffalo S.C. Shapiro Providing a Feel for Time NOW COUNTn hom 0 KL PML t1 t2 q ! beforeafter time duration !

24. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

25. cse@buffalo S.C. Shapiro Aligning Lexemes/NL “Stu” PML/SALKL ProperName(B4, Stu) Isa(B4, person) “person” ctgy npr ctgy n num sing Grammar NL World

26. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

27. cse@buffalo S.C. Shapiro Acting 1

28. cse@buffalo S.C. Shapiro Acting 2 I found a red robot. I am looking at a red robot. Follow a red robot.

29. cse@buffalo S.C. Shapiro Acting 3 I went to a red robot. I am near a red robot. I am following a red robot. I found a red robot. I am looking at a red robot. Follow a red robot.

30. cse@buffalo S.C. Shapiro Use of Indexicals 1 Come here. I found you. I am looking at you.

31. cse@buffalo S.C. Shapiro Come here. I came to you, Stu. I am near you. Use of Indexicals 2 I found you. I am looking at you.

32. cse@buffalo S.C. Shapiro Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. Use of Indexicals 3

33. cse@buffalo S.C. Shapiro Come here. I found you. I am looking at you. Use of Indexicals 4

34. cse@buffalo S.C. Shapiro Come here. I came to you. I am near you. I found you. I am looking at you. Use of Indexicals 5

35. cse@buffalo S.C. Shapiro Who am I? I talked to Stu and I am talking to you. Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? Use of Indexicals 6

36. cse@buffalo S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

37. cse@buffalo S.C. Shapiro Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. PML attentional structures anchor symbols by changing only when motivated.

38. cse@buffalo S.C. Shapiro For More Information Personnel Manual Tutorial Bibliography ftp’able SNePS source code etc. http://www.cse.buffalo.edu/sneps/