AGI-09 Scott Lathrop John Laird 1

2  Cognitive Architectures Amodal, symbolic representations & computations No general reasoning with perceptual-based representations (Barsalou 1999, 2008)  Cognitive Architectures Amodal, symbolic representations & computations No general reasoning with perceptual-based representations (Barsalou 1999, 2008)  Resulting in… Ad-hoc reasoning in tasks rich with spatial and visual properties “Bolted-on” task-dependent components  Resulting in… Ad-hoc reasoning in tasks rich with spatial and visual properties “Bolted-on” task-dependent components  What we desire… “Best of both worlds” multi-representational, task-independent approach Functionality Computational advantage Link perceptual-based thought and cognition  What we desire… “Best of both worlds” multi-representational, task-independent approach Functionality Computational advantage Link perceptual-based thought and cognition

3 RepresentationProcessingUsesExample Symbolic Amodal Symbolic manipulation General Reasoning/Control Qualitative Spatial & Visual Reasoning feature (letter-B, line) color(letter-B, blue) feature(box, corner) color(box, yellow) on(letter-B, box) Quantitative spatial Amodal/Perceptual-based (Spatial Imagery) Mathematical manipulation Dynamical Systems (motion models) Spatial Reasoning (Generic Shapes) Facilitate building visual depictive representation letter-B location orient 0 length 2 height 4 box location orient 20 length 6 height 4 width 2 Visual depictive Modality- specific/Perceptual-based (Visual Imagery) Mathematical manipulation Depictive manipulation Spatial Reasoning (Specific Shapes) Visual Feature Recognition

4 Enemy Scout-2 Enemy Scout-1 (Agent) Scout-2 (Teammate) Enemy Scout-3 Actual Situation Scout-2 (Teammate) Enemy Scout-1 Teammate’s View Observed Enemy Scout-2 Scout-1 (Agent) Agent’s View Observed Enemy Scout-2 Hypothesized Enemy Scout Hypothesized Enemy Scout “Key” Terrain Imagined Situation

5

6

SVI Quantitative spatial Scene-graph Viewpoint Object Map STM Refresher Stimulus-Based Refresh “Rendered” Image Visual-Spatial STM Object Set Obj A,Obj B,… Feature Set Spatial Set Feature Set Object Set Obj A1,Obj A2,… Spatial Set Visual depictive Set of 2D bitmaps Visual Buffer STM Shape Texture Spatial configuration Perceptual LTM “What” Inspectors“Where” Inspectors Soar Symbolic STM Goals Current State Process Productions Memory Symbols Visual Symbols Data Path Control Path Symbolic (Procedural) LTM IMAGERY OPERATORS Construct Generate Transform Inspect TASK OPERATORS ConstructorManipulator

 Pixel-level rewrites (Furnas, 2000) Production-like rules (LHS/RHS) Shared image (i.e. working memory) LHS and RHS are pixel patterns Conflict resolution scheme (sequencing) Processing may match other orientations Active manipulation of shapes  Contrast with other image processing Computer vision o Sentential manipulations o Filters (e.g. Gaussian) where each pixel is rewritten as a specific function of its neighbors’ values Cellular Automata o Finite State Machines o Next state of a cell based on current state and state of its neighbors o Rather than rewriting many local configurations at once = “don’t care” “2x2 reduce” “nibble up” “nibble left” 8

 Takes specific shape of no-go terrain and obstacles (yellow) into account  Attention window shift controlled by Soar  Pixel rewrite rules sent from Soar to SVI  Determine approximate path coverage by overlaying each scout’s view frustum  Simulate alternate view orientations to determine “best” path coverage * Steep Terrain All Other Obstacles Mark Obstacles * * * * Layer 0-1 * Distance Field Flood Layer 0-1 Layer 1-2 Mark Path Start- Layer 2 Layer 2-1 Layer 1-0 9

10

 General reasoning with symbolic and perceptual-based representations where the reasoning emerges from the combination of the representations  Soar Symbolic representations and computations Goals and high-level knowledge Controls imagery processing  Spatial and Visual Imagery (SVI) Quantitative spatial and visual depictive representations and computations Leverages perceptual mechanisms  Future Work - Push architecture closer to sensory input (Robotics) 11