S.C. Shapiro Symbol Anchoring in a Grounded Layered Architecture with Integrated Reasoning Stuart C. Shapiro Department of Computer Science and Engineering and Center for Cognitive Science University at Buffalo

S.C. Shapiro Based on Stuart C. Shapiro & Haythem O. Ismail, “Anchoring in a grounded layered architecture with integrated reasoning,” Robotics and Autonomous Systems 43, 2-3 (May 2003)

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Definition “Anchoring is the problem of connecting, inside an artificial system, symbols and sensor data that refer to the same physical objects in the external world.” [Silvia Coradeschi & Alessandro Saffiotti]

S.C. Shapiro Embodied Cassie A computational cognitive agent –Embodied in hardware –or Software-Simulated –Based on SNePS and GLAIR.

S.C. Shapiro Cassie, the FEVAHR (Foveal ExtraVehicular Activity Helper-Retriever)

S.C. Shapiro FEVAHR/Cassie in the Lab

S.C. Shapiro Crystal Cassie First Person Perspective Views

S.C. Shapiro Patofil and Filopat from “The Trial, The Trail” A VR drama by Josephine Anstey et al.

S.C. Shapiro DeliveryAgent Using Byron Weber Becker’s Java version of Rich Pattis’ Karel the Robot

S.C. Shapiro Magellan Pro TM Mobile Robot from iRobot

S.C. Shapiro SNePS Knowledge Representation and Reasoning –Intensional Representation –Propositions as Terms SNIP: SNePS Inference Package –Specialized connectives and quantifiers SNeBR: SNePS Belief Revision SNeRE: SNePS Rational Engine (Acting Language) Interface Languages –SNePSUL: Lisp-Like –SNePSLOG: Logic-Like –GATN for Fragments of English.

S.C. Shapiro Interaction with Cassie English (Statement, Question, Command) (Current) Set of Beliefs [SNePS] (Updated) Set of Beliefs [SNePS] Actions [SNeRE] (New Belief) [SNePS] English sentence expressing new belief answering question reporting actions Answer [SNIP] GATN Parser GATN Generator Reasoning Clarification Dialogue Looking in World Reasoning

S.C. Shapiro Entities, Terms, Symbols, Objects Cassie’s mental entity: a person named Stu SNePS term: B4 Object in world:

S.C. Shapiro GLAIR Architecture Knowledge Level Perceptuo-Motor Level Sensory-Actuator Level NL Vision Sonar MotionProprioception Grounded Layered Architecture with Integrated Reasoning SNePS Mind BodyBody World IP Sockets

S.C. Shapiro Alignment KL Body (PML/SAL) World SNePS term PML structure Object/PhenomenonAction

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro World Objects to Feature Tuples <Height, Width, Texture,.. > WorldPML/SAL

S.C. Shapiro Feature Tuples to KL Terms <Height, Width, Texture,.. > PML/SALKL ProperName(B4, Stu) Alignment

S.C. Shapiro Incomplete PML-Descriptions <Height, nil,.. > PML/SALKL Height(B4, B12)

S.C. Shapiro Unifying PML-Descriptions PML/SAL KL B20 B30 B31 B6 Isa Prop

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Deictic & Modality Registers for being situated in the world I You Now Vision. PML KL Terms denoting Cassie Addressee Current time Current state(s) of looking at x.

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Primitive Actions Aligned with PML/SAL Functions PMLKL M2(B4) Find(B4) SAL Modality Registers World A KL symbol is a pivot coordinating different modalities.

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Aligning NOW using MTF NOW Semi-lattice of times, all of which contain *NOW, any of which could be meant by “now” Finite---only conceptualized times of conceptualized states Maximal Temporal Frame based on *NOW KL PML

S.C. Shapiro Moving NOW with MTF NOW KL PML Moves when Cassie acts, newly observes a state, or is informed of a new state. Always includes times of states in modality registers.

S.C. Shapiro Providing a Feel for Time NOW COUNTn hom 0 KL PML t1 t2 q ! beforeafter time duration !

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Aligning Lexemes/NL “Stu” PML/SALKL ProperName(B4, Stu) Isa(B4, person) “person” ctgy npr ctgy n num sing Grammar NL World

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Acting 1

S.C. Shapiro Acting 2 I found a red robot. I am looking at a red robot. Follow a red robot.

S.C. Shapiro Acting 3 I went to a red robot. I am near a red robot. I am following a red robot. I found a red robot. I am looking at a red robot. Follow a red robot.

S.C. Shapiro Use of Indexicals 1 Come here. I found you. I am looking at you.

S.C. Shapiro Come here. I came to you, Stu. I am near you. Use of Indexicals 2 I found you. I am looking at you.

S.C. Shapiro Who am I? Your name is ‘Stu’ and you are a person. Who have you talked to? I am talking to you. Talk to Bill. I am talking to you, Bill. Come here. Use of Indexicals 3

S.C. Shapiro Come here. I found you. I am looking at you. Use of Indexicals 4

S.C. Shapiro Come here. I came to you. I am near you. I found you. I am looking at you. Use of Indexicals 5

S.C. Shapiro Who am I? I talked to Stu and I am talking to you. Your name is ‘Bill’ and you are a person. Who are you? I am the FEVAHR and my name is ‘Cassie’. Who have you talked to? Use of Indexicals 6

S.C. Shapiro Outline Introduction Perceivable entities and properties Attentional Structures Actions Time Language Examples Summary

S.C. Shapiro Summary KL terms denote mental entities. KL terms aligned to PML structures. PML variables/registers contain KL terms. PML variables/registers/structures grounded in world via sensors & effectors. PML attentional structures anchor symbols by changing only when motivated.

S.C. Shapiro Summary from Jackendoff Ray Jackendoff, Foundations of Language, Oxford University Press, 2002, Fig. 11.1, p (Independent work)

S.C. Shapiro For More Information Personnel Manual Tutorial Bibliography ftp’able SNePS source code etc.