Sparse distributed memory for ‘conscious’ software agents Cognitive Systems Research 2006 Sparse distributed memory for ‘conscious’ software agents Ashraf Anwar, Stan Franklin SCALB Tae-min, Jung 2018-11-15 Software Agent / Tea-min Jung

Outline Introduction Autonomous, cognitive, and ‘conscious’ agents Sparse distributed memory CMattie Autonomous, cognitive, and ‘conscious’ agents Sparse Distributed Memory SDM as a control structure for autonomous agents The SDM agent Modeling the world using SDM Including action in the world model Some psychological aspects of SDM Cued behavior Learning to act CMattie, a ‘conscious’ clerical agent SDM in CMattie Results and statistics Conclusions & Future research 2018-11-15 Software Agent / Tea-min Jung

Introduction Sparse distributed memory (SDM) Using Agent Domain Content addressable memory developed by Kanerva (1988a) Useful model of human associative memory Using Agent Domain Cognitive software agents Conscious software agents CMattie Clerical software agent Uses SDM for associative memory 2018-11-15 Software Agent / Tea-min Jung

Introduction : CMattie Feature Clerical Software Agent Human-like Agent CMattie’s ‘consciousness’ mechanism follows global workspace theory SDM Extended version of SDM Build, Store, Retrieve Percepts, Behaviors, Emotions 2018-11-15 Software Agent / Tea-min Jung

Autonomous, cognitive, and ‘conscious’ agents Autonomous Agent Sensing Situated in an environment Action Can affect environment A cognitive agent One of Autonomous agents More Cognitive capabilities Problem Solving, Planning, Learning, Perception, Emotions, Etc. A ‘conscious’ agent Extra functionality ‘consciousness’ Global workspace theory (Baars, 1998, 1997) Broadcast of Information Recruit processes Can Help in handling the current novel or problematic situation 2018-11-15 Software Agent / Tea-min Jung

Sparse Distributed Memory What is SDM? Kanerva, 1988 Sparse allocation of storage locations Use Vast binary address space 2n Semantic space where n is both the word length and the dimension of the address space The distributed nature of information storage and retrieval Associative Memory Auto-associative Used to retrieve itself The contents and the addresses belong to the same space and are used alternatively hetero-associative Stored in sequences The word length (dimension of the space) Represented by one or more bits in a Boolean vector or binary string of length n 2018-11-15 Software Agent / Tea-min Jung

Sparse Distributed Memory (cont’d) Content addressable Items in memory can be retrieved by using part of their contents as a cue Do not having to know their actual addresses in memory as in traditional computer memory. Boolean spaces The set of all Boolean vectors (points) of some fixed length, n, called the dimension of the space Hamming Distance Boolean geometry uses a metric distance between two points is the number of coordinates at which they differ Example : d((1,0,0,1,0), (1,0,1,1,1)) = 2 2018-11-15 Software Agent / Tea-min Jung

Sparse Distributed Memory (cont’d) 2018-11-15 Software Agent / Tea-min Jung

Sparse Distributed Memory (cont’d) 2018-11-15 Software Agent / Tea-min Jung

SDM as a control structure for autonomous agents The SDM agent Learning and Adaptation Autonomous agent interacts with its environment, and records its interaction Patterns Binary vectors Large Size Usage Address Datum Addressing the memory Need not be exact Address patterns use write address attract 2018-11-15 Software Agent / Tea-min Jung

Modeling the world using SDM Subjective experience Merge internal and external ‘pictures’ Internal Scan our surrounding for overall cues and fill in much of the detail from the internal model External Sensory Input 2018-11-15 Software Agent / Tea-min Jung

Modeling the world using SDM 2018-11-15 Software Agent / Tea-min Jung

Modeling the world using SDM The world model Update (i) The pattern held in the focus at time t is used to address the memory, activating a set of memory locations. (ii) The response read from those locations is the memory prediction of the sensory input at time t+1. (iii) If the prediction agrees with the sensory input, there is no need to adjust the memory, and the read pattern simply becomes the contents of the focus at time t+1. (iv) If the prediction disagrees with the sensory input, a third correct pattern is computed from them, e.g. an average, and it becomes the content of the focus at time t11. However, before it is used to address the memory at time t11, it is written in the locations from which the faulty output was just read (the locations selected at time t). 2018-11-15 Software Agent / Tea-min Jung

Modeling the world using SDM Leaning Correction patterns are written into memory over time The memory builds a better model of the world Result Better understanding capacity to store information i.e. experience. 2018-11-15 Software Agent / Tea-min Jung

Including action in the world model Animals perform actions means learning to reproduce sequences of patterns that drive the muscles Agent the agent’s own actions can be included in the world model by storing motor sequences in memory in addition to sensory sequences 2018-11-15 Software Agent / Tea-min Jung

Some psychological aspects of SDM : Cued behavior Assume Focus 80% for sensory input 20% for effectors Stimulus sequence <A, B, C> Response sequence <X, Y, Z> Generated Focus <Aw, BX, CY, dZ> <AW, BX, CY, DZ> Guess<A, B, K, L> K and L are quite different from C and D BX will retrieve CY, and action Y will be executed Senses will report K instead of C Focus will be HY instead of CY (H is some combination of C and K) DZ will not be retrieved (failure of action Z) 2018-11-15 Software Agent / Tea-min Jung

Cued behavior (cont’d) This failure can be explained as follows. (i) An environment monitoring system ceases to act when the proper cues are no longer present. (ii) A system that monitors its own action effects, stops acting when the effects no longer confirm the system expectations. Result The system will initiate the ‘thought’ in the focus and then block off the present The memory will then retrieve into the focus the likely consequences of the contemplated actions 2018-11-15 Software Agent / Tea-min Jung

Some psychological aspects of SDM : Learning to act Success How well it predicts the world Action correction is much harder than perceptual correction Learning to act The System stores actions in a way that increase the likelihood of finding good states and of avoiding bad ones 2018-11-15 Software Agent / Tea-min Jung

CMattie, a ‘conscious’ clerical agent A ‘conscious’ software agent To Manage seminar announcements in the Mathematical Sciences Department at the University of Memphis Aims To discuss the use of SDM in CMattie Work Send weekly seminar announcements via email to a mailing list that they maintain Receive and understand email messages in natural language from seminar organizers and attendees Handle their information and requests, and compose and email the weekly department seminar announcements to their lists 2018-11-15 Software Agent / Tea-min Jung

CMattie, a ‘conscious’ clerical agent (cont’d) Perception Understanding email messages in natural language Codelets perception action, emotions, metacognition, etc. Small pieces of code Running independently Simple specific task to accomplish 2018-11-15 Software Agent / Tea-min Jung

CMattie, a ‘conscious’ clerical agent (cont’d) Memory Produce suggested actions and emotional states SDM Long term associative memory CBM (Case Based Memory) Episodic memory PRs Perception Registers (PRs). When an email message comes in Selected 2018-11-15 Software Agent / Tea-min Jung

SDM in CMattie Function of SDM in CMattie Step 1 Step 2 Step 3 Suggesting actions Emotional status : Incoming Percepts (Values in the perception registers) Previous experience Response to the observed values in the perception registers. Auto-associative SDM Relies on the dominance, in size, of the percepts over the behaviors and emotions Step 2 Providing for defaults for missing information in perception registers, based on previous history and previous associations built over time. Step 3 Removal of noise and correction of errors in percepts, based on previous history and previous associations built over time. 2018-11-15 Software Agent / Tea-min Jung

SDM in CMattie (cont’d) The output from SDM has three distinct parts. (i) One part fills a set of registers identical to the perception registers. Defaults for missing fields are taken from here. (ii) A second contains the suggested behavior in response to the percept received. (iii) The third influences CMattie emotional status based again on the percept received. 2018-11-15 Software Agent / Tea-min Jung

Enhancements to original SDM in CMattie Writing important or emotionally involved percepts into memory multiple times emphasize the specific percept by giving it more repetitions internally in the memory It also resembles a human going over emotional situations again and again in his or her mind. Using error detection and correction Allow for original information retrieval was considered and tested The Problem with this approach is that special encoding for various words need to be considered to allow for enough hamming distance between various words Word encoding 2018-11-15 Software Agent / Tea-min Jung

Tuning of SDM parameters in CMattie Address space dimension (word length): 736 Number of hard locations: 3000 Critical distance for divergence: 155 Access radius for read / write: 331 Convergence distance: 147 Maximum number of reading iterations: 10 2018-11-15 Software Agent / Tea-min Jung

Enhancements to original SDM in CMattie Use of hashing to encode various field values into uniform Fixed length hash keys. Example : All various length speaker names are hashed using fixed length Avoid having to deal with arbitrary long names. Assigning weight or precedence to certain fields in the PRs Adding extra word length for them Semantic space Making it more capable of retrieving missing fields when used as a partial cue Implementation-dependent 2018-11-15 Software Agent / Tea-min Jung

Run Trace Example Step1 Step 2 Step 3 Email message is a request to add a name to the mailing list Step1 The incoming email message (sensory input) is analyzed and understood by the perception module Moved into the appropriate perception registers (PRs). PRs ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘addToListMessage’, ‘aanwar1@memphis.edu’, ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ Step 2 The contents of the PRs are passed to the input foci of SDM and CBM Step 3 SDM and CBM go into reading cycles. Note that the behavior and emotion fields are EMPTY’ when reading 2018-11-15 Software Agent / Tea-min Jung

Run Trace (cont’d) Step 4 Step 5 Step 6 Upon convergence, the output from SDM and CBM is returned to the appropriate registers in their respective foci Suggestions for the behavior and its activation Let us assume that the SDM output behavior is ‘Update Seminar List Recipients’ with high enough activation e.g. 0.9, to make it execute Step 5 ‘Consciousness’ plays its role and the content of the focus including some output from memory, gets broadcast This makes the information available to the behavior net as well as the emotion module Step 6 The emotion module extracts the emotion part of the broadcast (originally from the memory output) 2018-11-15 Software Agent / Tea-min Jung

Run Trace (cont’d) Step 7 Step 8 Step 9 Step 10 The behavior codelets extract the behavior part of the broadcast (originally from the memory output) Step 8 The emotion module determines the new emotional status for the agent and puts it into the input focus overwriting the ‘EMPTY’ placed by perception Step 9 The behavior net determines the new behavior activation for the agent and puts it into the input focus overwriting the ‘EMPTY’ placed by perception Step 10 SDM and CBM go into a write cycle to update the agent memory with the new input percept along with the corresponding emotion and behaviors 2018-11-15 Software Agent / Tea-min Jung

Results and statistics 2018-11-15 Software Agent / Tea-min Jung

Conclusions SDM has proven to be a successful tool for associative memory in ‘conscious’ software agents 2018-11-15 Software Agent / Tea-min Jung

Future research Associations based upon relatively small cues, as we human do. For SDM to converge Sufficiently large portion of a previously written word must be presented to the memory as an address. Future Work We are currently working on a new version of SDM that addresses this problem If successful, this new technique should solve one of the major open problems for the use of SDM as associative memory 2018-11-15 Software Agent / Tea-min Jung