1. Sparse distributed memory for ‘conscious’ software agents
Cognitive Systems Research 2006
Sparse distributed memory for ‘conscious’ software agents
Ashraf Anwar, Stan Franklin
SCALB
Tae-min, Jung
Software Agent / Tea-min Jung

2. Outline Introduction Autonomous, cognitive, and ‘conscious’ agents
Sparse distributed memory
CMattie
Autonomous, cognitive, and ‘conscious’ agents
Sparse Distributed Memory
SDM as a control structure for autonomous agents
The SDM agent
Modeling the world using SDM
Including action in the world model
Some psychological aspects of SDM
Cued behavior
Learning to act
CMattie, a ‘conscious’ clerical agent
SDM in CMattie
Results and statistics
Conclusions & Future research
Software Agent / Tea-min Jung

3. Introduction Sparse distributed memory (SDM) Using Agent Domain
Content addressable memory developed by Kanerva (1988a)
Useful model of human associative memory
Using Agent Domain
Cognitive software agents
Conscious software agents
CMattie
Clerical software agent
Uses SDM for associative memory
Software Agent / Tea-min Jung

4. Introduction : CMattie
Feature
Clerical Software Agent
Human-like Agent
CMattie’s ‘consciousness’ mechanism follows global workspace theory
SDM
Extended version of SDM
Build, Store, Retrieve
Percepts, Behaviors, Emotions
Software Agent / Tea-min Jung

5. Autonomous, cognitive, and ‘conscious’ agents
Autonomous Agent
Sensing
Situated in an environment
Action
Can affect environment
A cognitive agent
One of Autonomous agents
More Cognitive capabilities
Problem Solving, Planning, Learning, Perception, Emotions, Etc.
A ‘conscious’ agent
Extra functionality ‘consciousness’
Global workspace theory (Baars, 1998, 1997)
Broadcast of Information
Recruit processes
Can Help in handling the current novel or problematic situation
Software Agent / Tea-min Jung

6. Sparse Distributed Memory
What is SDM?
Kanerva, 1988
Sparse allocation of storage locations
Use Vast binary address space
2n Semantic space where n is both the word length and the dimension of the address space
The distributed nature of information storage and retrieval
Associative Memory
Auto-associative
Used to retrieve itself
The contents and the addresses belong to the same space and are used alternatively
hetero-associative
Stored in sequences
The word length (dimension of the space)
Represented by one or more bits in a Boolean vector or binary string of length n
Software Agent / Tea-min Jung

7. Sparse Distributed Memory (cont’d)
Content addressable
Items in memory can be retrieved by using part of their contents as a cue
Do not having to know their actual addresses in memory as in traditional computer memory.
Boolean spaces
The set of all Boolean vectors (points) of some fixed length, n, called the dimension of the space
Hamming Distance
Boolean geometry uses a metric
distance between two points is the number of coordinates at which they differ
Example : d((1,0,0,1,0), (1,0,1,1,1)) = 2
Software Agent / Tea-min Jung

8. Sparse Distributed Memory (cont’d)
Software Agent / Tea-min Jung

9. Sparse Distributed Memory (cont’d)
Software Agent / Tea-min Jung

10. SDM as a control structure for autonomous agents
The SDM agent
Learning and Adaptation
Autonomous agent interacts with its environment, and records its interaction
Patterns
Binary vectors
Large Size
Usage
Address
Datum
Addressing the memory
Need not be exact
Address patterns use write address attract
Software Agent / Tea-min Jung

11. Modeling the world using SDM
Subjective experience
Merge internal and external ‘pictures’
Internal
Scan our surrounding for overall cues and fill in much of the detail from the internal model
External
Sensory Input
Software Agent / Tea-min Jung

12. Modeling the world using SDM
Software Agent / Tea-min Jung

13. Modeling the world using SDM
The world model Update
(i) The pattern held in the focus at time t is used to address the memory, activating a set of memory locations.
(ii) The response read from those locations is the memory prediction of the sensory input at time t+1.
(iii) If the prediction agrees with the sensory input, there is no need to adjust the memory, and the read pattern simply becomes the contents of the focus at time t+1.
(iv) If the prediction disagrees with the sensory input, a third correct pattern is computed from them, e.g. an average, and it becomes the content of the focus at time t11. However, before it is used to address the memory at time t11, it is written in the locations from which the faulty output was just read (the locations selected at time t).
Software Agent / Tea-min Jung

14. Modeling the world using SDM
Leaning
Correction patterns are written into memory over time
The memory builds a better model of the world
Result
Better understanding capacity to store information
i.e. experience.
Software Agent / Tea-min Jung

15. Including action in the world model
Animals
perform actions means learning to reproduce sequences of patterns that drive the muscles
Agent
the agent’s own actions can be included in the world model by storing motor sequences in memory in addition to sensory sequences
Software Agent / Tea-min Jung

16. Some psychological aspects of SDM : Cued behavior
Assume
Focus
80% for sensory input
20% for effectors
Stimulus sequence <A, B, C>
Response sequence <X, Y, Z>
Generated Focus
<Aw, BX, CY, dZ>
<AW, BX, CY, DZ>
Guess<A, B, K, L>
K and L are quite different from C and D
BX will retrieve CY, and action Y will be executed
Senses will report K instead of C
Focus will be HY instead of CY (H is some combination of C and K)
DZ will not be retrieved (failure of action Z)
Software Agent / Tea-min Jung

17. Cued behavior (cont’d)
This failure can be explained as follows.
(i) An environment monitoring system ceases to act when the proper cues are no longer present.
(ii) A system that monitors its own action effects, stops acting when the effects no longer confirm the system expectations.
Result
The system will initiate the ‘thought’ in the focus and then block off the present
The memory will then retrieve into the focus the likely consequences of the contemplated actions
Software Agent / Tea-min Jung

18. Some psychological aspects of SDM : Learning to act
Success
How well it predicts the world
Action correction is much harder than perceptual correction
Learning to act
The System stores actions in a way that increase the likelihood of finding good states and of avoiding bad ones
Software Agent / Tea-min Jung

19. CMattie, a ‘conscious’ clerical agent
A ‘conscious’ software agent
To Manage seminar announcements in the Mathematical Sciences Department at the University of Memphis
Aims
To discuss the use of SDM in CMattie
Work
Send weekly seminar announcements via to a mailing list that they maintain
Receive and understand messages in natural language from seminar organizers and attendees
Handle their information and requests, and compose and the weekly department seminar announcements to their lists
Software Agent / Tea-min Jung

20. CMattie, a ‘conscious’ clerical agent (cont’d)
Perception
Understanding messages in natural language
Codelets
perception action, emotions, metacognition, etc.
Small pieces of code
Running independently
Simple specific task to accomplish
Software Agent / Tea-min Jung

21. CMattie, a ‘conscious’ clerical agent (cont’d)
Memory
Produce suggested actions and emotional states
SDM
Long term associative memory
CBM (Case Based Memory)
Episodic memory
PRs
Perception Registers (PRs).
When an message comes in Selected
Software Agent / Tea-min Jung

22. SDM in CMattie Function of SDM in CMattie Step 1 Step 2 Step 3
Suggesting actions
Emotional status : Incoming Percepts (Values in the perception registers)
Previous experience
Response to the observed values in the perception registers.
Auto-associative SDM
Relies on the dominance, in size, of the percepts over the behaviors and emotions
Step 2
Providing for defaults for missing information in perception registers, based on previous history and previous associations built over time.
Step 3
Removal of noise and correction of errors in percepts, based on previous history and previous associations built over time.
Software Agent / Tea-min Jung

23. SDM in CMattie (cont’d)
The output from SDM has three distinct parts.
(i)
One part fills a set of registers identical to the perception registers. Defaults for missing fields are taken from here.
(ii)
A second contains the suggested behavior in response to the percept received.
(iii)
The third influences CMattie emotional status based again on the percept received.
Software Agent / Tea-min Jung

24. Enhancements to original SDM in CMattie
Writing important or emotionally involved percepts into memory multiple times
emphasize the specific percept by giving it more repetitions internally in the memory
It also resembles a human going over emotional situations again and again in his or her mind.
Using error detection and correction
Allow for original information retrieval was considered and tested
The Problem with this approach is that special encoding for various words need to be considered to allow for enough hamming distance between various words
Word encoding
Software Agent / Tea-min Jung

25. Tuning of SDM parameters in CMattie
Address space dimension (word length): 736
Number of hard locations: 3000
Critical distance for divergence: 155
Access radius for read / write: 331
Convergence distance: 147
Maximum number of reading iterations: 10
Software Agent / Tea-min Jung

26. Enhancements to original SDM in CMattie
Use of hashing to encode various field values into uniform
Fixed length hash keys.
Example : All various length speaker names are hashed using fixed length
Avoid having to deal with arbitrary long names.
Assigning weight or precedence to certain fields in the PRs
Adding extra word length for them
Semantic space
Making it more capable of retrieving missing fields when used as a partial cue
Implementation-dependent
Software Agent / Tea-min Jung

27. Run Trace Example Step1 Step 2 Step 3
message is a request to add a name to the mailing list
Step1
The incoming message (sensory input) is analyzed and understood by the perception module
Moved into the appropriate perception registers (PRs).
PRs
‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘EMPTY’, ‘addToListMessage’, ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’ ‘EMPTY’
Step 2
The contents of the PRs are passed to the input foci of SDM and CBM
Step 3
SDM and CBM go into reading cycles. Note that the behavior and emotion fields are EMPTY’ when reading
Software Agent / Tea-min Jung

28. Run Trace (cont’d) Step 4 Step 5 Step 6
Upon convergence, the output from SDM and CBM is returned to the appropriate registers in their respective foci
Suggestions for the behavior and its activation
Let us assume that the SDM output behavior is ‘Update Seminar List Recipients’ with high enough activation e.g. 0.9, to make it execute
Step 5
‘Consciousness’ plays its role and the content of the focus including some output from memory, gets broadcast
This makes the information available to the behavior net as well as the emotion module
Step 6
The emotion module extracts the emotion part of the broadcast (originally from the memory output)
Software Agent / Tea-min Jung

29. Run Trace (cont’d) Step 7 Step 8 Step 9 Step 10
The behavior codelets extract the behavior part of the broadcast (originally from the memory output)
Step 8
The emotion module determines the new emotional status for the agent and puts it into the input focus overwriting the ‘EMPTY’ placed by perception
Step 9
The behavior net determines the new behavior activation for the agent and puts it into the input focus overwriting the ‘EMPTY’ placed by perception
Step 10
SDM and CBM go into a write cycle to update the agent memory with the new input percept along with the corresponding emotion and behaviors
Software Agent / Tea-min Jung

30. Results and statistics
Software Agent / Tea-min Jung

31. Conclusions
SDM has proven to be a successful tool for associative memory in ‘conscious’ software agents
Software Agent / Tea-min Jung

32. Future research
Associations based upon relatively small cues, as we human do.
For SDM to converge
Sufficiently large portion of a previously written word must be presented to the memory as an address.
Future Work
We are currently working on a new version of SDM that addresses this problem
If successful, this new technique should solve one of the major open problems for the use of SDM as associative memory
Software Agent / Tea-min Jung