S.C. Shapiro Symbol-Anchoring in Cassie Stuart C. Shapiro and Haythem O. Ismail Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo {shapiro | |

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Embodied Cassie A computational cognitive agent –Embodied in hardware –or Software-Simulated –Based on SNePS and GLAIR.

S.C. Shapiro Cassie, the FEVAHR

S.C. Shapiro FEVAHR/Cassie in the Lab

S.C. Shapiro SNePS Knowledge Representation and Reasoning –Intensional Representation –Propositions as Terms SNIP: SNePS Inference Package –Specialized connectives and quantifiers SNeBR: SNePS Belief Revision SNeRE: SNePS Rational Engine Interface Languages –SNePSUL: Lisp-Like –SNePSLOG: Logic-Like –GATN for Fragments of English.

S.C. Shapiro Interaction with Cassie English (Statement, Question, Command) (Current) Set of Beliefs [SNePS] (Updated) Set of Beliefs [SNePS] Actions [SNeRE] (New Belief) [SNePS] English sentence expressing new belief answering question reporting actions Answer [SNIP] GATN Parser GATN Generator Reasoning Clarification Dialogue Looking in World Reasoning

S.C. Shapiro Entities, Terms, Symbols, Objects Cassie’s mental entity: a person named Stu SNePS term: B4 Object in world:

S.C. Shapiro GLAIR Architecture Knowledge Level Perceptuo-Motor Level Sensory-Actuator Level NL Vision Sonar MotionProprioception Grounded Layered Architecture with Integrated Reasoning SNePS

S.C. Shapiro Alignment KL Body (PML/SAL) World SNePS term PML structure Object/PhenomenonAction

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro World Objects to Feature Vectors <Height, Width, Texture,.. > WorldPML/SAL

S.C. Shapiro Feature Vectors to KL Terms <Height, Width, Texture,.. > PML/SALKL ProperName(B4, Stu) Alignment

S.C. Shapiro Incomplete Feature Vectors <Height, nil,.. > PML/SALKL Height(B4, B12)

S.C. Shapiro Unifying PML-Descriptions PML/SAL KL B20 B30 B31 B6 Isa Prop

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Deictic & Modality Registers I You Now Vision. PML KL Terms denoting Cassie Addressee Current time Current state(s) of looking at x.

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Primitive Actions Aligned with PML/SAL Functions PMLKL Find(B4) SAL Modality Registers World

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Aligning NOW using MTF NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states Maximal Temporal Frame based on *NOW KL PML

S.C. Shapiro Moving NOW with MTF NOW KL PML Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers.

S.C. Shapiro Providing a Feel for Time NOW COUNTn hom 0 KL PML t1 t2 q ! beforeafter time duration !

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Aligning Lexemes/NL “Stu” PML/SALKL ProperName(B4, Stu) Isa(B4, person) “person” ctgy npr ctgy n num sing Grammar NL World

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Acting 1

S.C. Shapiro Acting 2 I found a red robot. I am looking at a red robot. Follow a red robot.

S.C. Shapiro Acting 3 I went to a red robot. I am near a red robot. I am following a red robot. I found a red robot. I am looking at a red robot. Follow a red robot.

S.C. Shapiro Use of Indexicals 1 Come here. I found you. I am looking at you.

S.C. Shapiro Come here. I came to you, Stu. I am near you. Use of Indexicals 2 I found you. I am looking at you.

S.C. Shapiro Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. Use of Indexicals 3

S.C. Shapiro Come here. I found you. I am looking at you. Use of Indexicals 4

S.C. Shapiro Come here. I came to you. I am near you. I found you. I am looking at you. Use of Indexicals 5

S.C. Shapiro Who am I? I talked to Stu and I am talking to you. Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? Use of Indexicals 6

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. PML attentional structures anchor symbols by changing only when motivated.

S.C. Shapiro For More Information Personnel Manual Tutorial Bibliography ftp’able SNePS source code etc.