1. cse@buffalo The GLAIR Architecture for Cognitive Robotics Stuart C. Shapiro Department of Computer Science & Engineering and Center for Cognitive Science State University of New York at Buffalo shapiro@buffalo.edu

2. cse@buffalo GLAIR: Grounded Layered Architecture with Integrated Reasoning Focus of This Talk: Connecting Reasoning Mind with Acting/Sensing Body CogRob2010 S. C. Shapiro 2

3. cse@buffalo CogRob2010 S. C. Shapiro 3 Motivations Add acting and sensing to a reasoning agent. –First person reasoning; on-line acting & sensing. Layers –Motivated by mind/body connections/distinctions. –Let same mind be plugged into different bodies. Embodiment –Origin of beliefs in sensation & proprioception. –First-person privileged knowledge of own body. Situatedness –Has a sense of where it is in the world. Symbol grounding –In body-layer structures. –Symbol as pivot between various modalities.

4. cse@buffalo Outline  The Layers Symbol Grounding Putting it Together: Two examples CogRob2010 S. C. Shapiro 4

5. cse@buffalo Cog Rob 2010 S. C. Shapiro 5 KL PMLa PMLb PMLc SAL Mind Body Independent of lower-body implementation Hearing Vision Motion Speech WORLDWORLD I/P s o c k e t s GLAIR Architecture Dependent on lower-body implementation Proprioception

6. cse@buffalo Sensori-Actuator Layer Sensor and effector controllers CogRob2010 S. C. Shapiro 6

7. cse@buffalo Perceptuo-Motor Layer PMLa PMLb PMLc CogRob2010 S. C. Shapiro 7

8. cse@buffalo PMLc Abstracts sensors & effectors Body’s behavioral repertoire CogRob2010 S. C. Shapiro 8

9. cse@buffalo PMLb Translation & Communication –Between PMLa & PMLc Highest layer that knows body implementation CogRob2010 S. C. Shapiro 9

10. cse@buffalo PMLa Grounds KL symbols –Perceptual structures –Implementation of primitive actions Registers for Embodiment & Situatedness –Deictic Registers –Modality Registers CogRob2010 S. C. Shapiro 10

11. cse@buffalo The Knowledge Layer Implemented in SNePS Agent’s Beliefs Representations of conceived of entities Semantic Memory Episodic Memory Quantified & conditional beliefs Plans for non-primitive acts Plans to achieve goals Beliefs re. preconditions & effects of acts Policies: Conditions for performing acts Self-knowledge Meta-knowledge CogRob2010 S. C. Shapiro 11

12. cse@buffalo SNePS A KRR system Every non-atomic expression is simultaneously –An expression of SNePS logic –An assertional frame –A propositional graph Every SNePS expression is a term –Denoting a mental entity CogRob2010 S. C. Shapiro 12

13. cse@buffalo Ontology of Mental Entities Entity –Proposition Agent can believe it or its negation Includes quantified & conditional beliefs –Act Agent can perform it –Policy Condition-act rule agent can adopt –Thing Other entities: individuals, properties, times, etc. CogRob2010 S. C. Shapiro 13

14. cse@buffalo Outline The Layers  Symbol Grounding Putting it Together: Two examples CogRob2010 S. C. Shapiro 14

15. cse@buffalo S. C. Shapiro Entities, Terms, Symbols, Objects Agent’s mental entity: a person named Stu SNePS term: b4 Object in world: 15 CogRob2010

16. cse@buffalo S. C. Shapiro Alignment Mind (KL) Body (PML/SAL) World KL term PML structure Object/PhenomenonAction CogRob201016

17. cse@buffalo World Objects to Feature Tuples <Height, Width, Texture,.. > WorldPML/SAL 17 S. C. Shapiro CogRob2010

18. cse@buffalo Feature Tuples to KL Terms <Height, Width, Texture,.. > PML/SALKL ProperName(b4, Stu) Alignment CogRob201018 S. C. Shapiro

19. cse@buffalo Incomplete PML-Descriptions <Height, nil,.. > PML/SALKL Height(b4, b12) CogRob201019 S. C. Shapiro

20. cse@buffalo Unifying PML-Descriptions PML/SAL KL b20 b30 b31 b6 Isa Prop 20CogRob2010 S. C. Shapiro

21. cse@buffalo Deictic Registers For being situated in the world PML registers hold KL terms I term denoting agent YOU term denoting dialogue partner NOW term denoting current time CogRob2010 S. C. Shapiro 21

22. cse@buffalo Modality Registers For privileged first-person knowledge of what agent is doing Register for each modality, m, holds KL term denoting act that m is engaged in CogRob2010 S. C. Shapiro 22

23. cse@buffalo Outline The Layers Symbol Grounding  Putting it Together: Two examples CogRob2010 S. C. Shapiro 23

24. cse@buffalo PML/SAL KL b20 b30 b6 Isa Prop 24CogRob2010 S. C. Shapiro m2 Find a green robot. (find ) cassie m75 m76 robbie VISION Language-Mind-World-Mind WORLD

25. cse@buffalo Building Episodic Memory CogRob2010 S. C. Shapiro 25 KL PML e1 I a1 b1b1 m1! t1e2 a2 m2! t2 m3! NOW COUNT n hom 0 q m4! before after event time act agent duration ACT

26. cse@buffalo Summary GLAIR is an architecture: –To connect mind and body For symbol grounding –To provide first-person privileged knowledge of The body What it’s doing –To provide situatedness in the world Via deictic registers CogRob2010 S. C. Shapiro 26

27. cse@buffalo CSE 719 S. C. Shapiro 27 Collaborators Past and present members of SNeRG: The SNePS Research Group http://www.cse.buffalo.edu/sneps/