1. Towards Biological Limbic System Models as Basic Deliberative Architectures
Derek Harter, Dept of Computer Science and Information Systems, Texas A&M University – Commerce, TX
Robert Kozma, Dept of Computer Science, University of Memphis, TN
Walter J. Freeman, Dept of Molecular and Cell Biology, University of California at Berkeley, CA
Computational Neurodynamics
Laboratory
Experiments and Results
Introduction
Conclusions
Cognitive Map Formation Simulation (Harter & Kozma 2004a)
Appetitive / Aversive Experiment (Harter & Kozma 2004b)
Our work shows that aperiodic dynamics can be used in autonomous agents to form perceptual categories and other long-term memories.
The intrinsic aperiodic dynamics of the K-III are shaped through experience in our autonomous agents by learning mechanisms, such as hebbian modification and habituation.
Different chaotic attractors come to “represent” the meanings of salient stimuli to the organism.
Intrinsic aperiodic dynamics appear to play an important role in neurological mechanisms of biological organisms:
One advantage is that it may be much quicker to shape attractor basins from baseline aperioidc background activity. Therefore memory formation is fast and occurs with only one or a few exposures to stimuli.
Aperiodic dynamics aid in the recognition process as well. Chaotic activity is not noise, it can quickly converge on formed attractors.
The evolution of the primordial limbic system marked the formation of long-term memory mechanisms in biological brains.
Introduced the first forms of deliberative behavior, where past experiences and memory are brought to bear on current problems (for example in order to successfully navigate a learned environment).
Yet another level of plasticity in behavior formation, such behavior goes beyond learning to chain sensory/motor stimuli together, to learning conditioned responses based on the ontogenetic experience of the organism with the environment.
Intelligent behavior is characterized by the flexible and creative pursuit of endogenously defined goals.
One characterization of intentional behavior is that it is an act of observation through time and space, by which information is sought for the guidance of future action.
Intent comprises the endogenous initiation, construction, and direction of behavior into the world.
What we think of as deliberative behavior begins with this process, where ontogenetic learning and memory combine with endogenous needs to drive behavior in a directed and rational manner.
The K-IV architecture proposes that:
Intrinsic aperiodic dynamics are important to the formation and recognition of objects/events/categories/memories in neuronal spatio-temporal mechanisms.
Intentional and therefore deliberative actions were first made possible by primordial long-term hippocampal memory systems.
A bottom-up understanding of the dynamics of neuronal populations and their contribution to mechanisms such as memory and perception may lead to a better understanding of the neurological basis of intentional and deliberative behavior.
While bottom-up approaches to studying cognition have proved insightful in many ways, top-down approaches are still better at explaining deliberative cognitive processes. Deliberative actions are those that go beyond simple sensory-motor loops and seem to require some type of internal model, map or logical reasoning. Examples of deliberative actions include planning a route to navigate to a goal or performing a chain of logical inference to determine a likely course of action.
Bottom-up approaches such as Walter’s tortoise (1951) and Braitenberg’s vehicles (1984) are excellent models of how simple sensory-motor loops can combine to produce complex intentional behavior. Such behaviors are still mainly of the tropic type (e.g. phototropic, chemotropic), which rely on detecting and following some type of perceptual gradient in the environment. Recently, models such as Brook’s (1990) subsumption architecture have shown us how collections of behavior patterns can combine in relatively flexible chains, in an emergent manner, to produce more complex behaviors. Tropic behaviors are present in even the simplest of single celled organisms, while the complex collection, chaining and combining of such sensory-motor behavior patterns first appear with insects and simple vertebrates like fish.
Deliberative actions appear to require the development of more long-term memory mechanisms that allow for the storage of past experiences and for these experiences to be brought to bear on the current situation (Freeman 2001). Evolutionarily, the development of the limbic system in simple vertebrates, such as amphibians, marks the first appearance of primitive hippocampal structures (Kozma, Freeman, Erdi, 2003). The hippocampus plays the role of forming and recalling longer-term representations of experiences. It is known to participate in the formation of episodic memory, logical reasoning and cognitive maps (Arbib, Érdi & Szentágothai, 1997). Building deliberative systems in a bottom-up whole-system approach would therefore potentially benefit from a more complete understanding of the biological limbic system and how its dynamics contribute to the formation and execution of deliberative behavior.
Task: learn edible / poisonous distinction using KA-III
Environment: 6 food sources, 3 edible 3 poisonous
Results: categories formed in KA-III PC layer, trigger avoidance behavior before reaching poisonous food source.
Task: form representation of environmental locations using a KA-III
Environment: 8 locations, distance and direction information to 4 landmarks
Results: 8 distinct aperiodic attractors form in layers of the KA-III
Right middle, contour maps of 500ms of spatio-temporal activity in the CA1 layer
Cluster analysis, right bottom, confirms attractors formed that correspond to the 8 locations.
Environment Key
Edible food
source 1
Poisonous
food source 1 1 2 3 1 2 3
Architecture used in the cognitive map formation simulation. The CA1,CA2 and CA3 layers form a KA-III. CA1 and CA3 are an 8x8 array of KA-II units.
Limbic System Architecture
K-Models
The K-IV architecture is a model of what biologists believe is the simplest neural system capable of basic intentional and deliberative actions, the limbic system (Kozma, Freeman & Érdi, 2003; Harter & Kozma, submitted). The purpose of the K-IV is to model a complete autonomous organism, in a bottom-up manner, to understand better the neurodynamical mechanisms involved in intentional and deliberative behavior. The K-IV uses a neural population model (called K-sets) to describe the activity of large populations of neurons (as opposed to single unit or more abstract ANN models). It is a highly-recurrent multi-layer model of the important neurological structures of the basic limbic system.
We have developed pieces of the K-IV for use as control mechanisms in autonomous vehicles for exploration and navigation problems for NASA. We use both continuous and discrete versions of the K-set neural population models to develop autonomous agent simulations (Harter & Kozma, submitted).
3.5 billion years: single-cell entities
550 million years: fish & vertebrates
430 million years: insects
370 million years: reptiles
330 million years: dinosaurs
250 million years: mammals
120 million years: primates
18 million years: great apes
2.5 million years: man
5000 years: writing
Basic Limbic System
Primitive Hippocampus
Long-term memory
Beyond stimulus/response
Episodic Memory
Cognitive Maps
Intentional Behavior & Deliberative Actions
Table: Characterization of the hierarchy of K-sets
Type
Structure
Inhrt Dynamics
Exs. in brain*
K-0
Single Unit
Nonlinear I/O function
All higher level K sets are composed of K0 units
K-I
Populations of excitatory or inhibitory units
Fixed point convergence to zero or nonzero value
PG, DG, BG, BS
K-II
Interacting populations of excitatory and inhibitory units
Periodic, limit cycle oscillations; frequency in the gamma band
OB, AON, PC, CA1, CA3, CA2, HT, BG, BS, Amygdala
K-III
Several interacting KII and KI sets
Aperiodic, chaotic oscillations
Cortex, Hippocamp, Midline Forebrain
K-IV
Interacting KIII sets
Spatio-temporal dynamics with global phase transitions (itinerancy)
Hemisphere cooperation of cortical, HF and MF by the Amygdala
* Notations: PG – periglomerular; OB - olfactory bulb; AON - anterior olfactory nucleus; PC- prepyriform cortex; HF - hippocampal formation; DG - dentate gyrus; CA1, CA2, CA3 - curnu ammonis sections of the hippocampus; MF - midline forebrain; BG - basal ganglia; HT - hypothalamus; DB - diagonal band; SP – septum
K-0 Continuous ODE
Neural Population Model:
KA-0 Discrete Difference
Equation:
Transfer Function:
The basic hypothesis captured by the K-IV model, shown to the left, is that intrinsic aperiodic dynamics, like those observed in some perceptual processes, may play important roles in the hippocampal memories and value systems of biological organisms.
The biological limbic system, and thus the K-IV model, is composed of four basic areas. Perceptual areas which are mainly involved in determining the identity and type of perceptual stimuli. The hippocampus which forms more plastic long-term memories and can be thought of as involved in orienting the organism within its spatio-temporal environment. The midline forebrain which receives internal sensations and is implicated in the regulation of the organisms needs, drives and goals. And motor areas which regulate actions of the organism out into the environment.
In the K-IV model, the “What”, “Where” and “Why” systems are modeled by K-III hierarchical groups, capable of producing intrinsic aperiodic basil activity. The following properties are also important to the K-IV model:
References
K models are population models that capture the dynamics of populations of neurons.
Continuous models first developed by Freeman (see Freeman 1991)
Discrete model described in (Harter & Kozma 2004)
Hypothesis: Intrinsic aperiodic dynamics are important to perceptual and memory processes.
K-II produce oscillatory behavior, damped oscillation shown bottom left.
K-III produce intrinsic aperiodic dynamics, shown in bottom middle and right.
K-III E1 E2 I1 I2
Layer 1 d
receptors
Layer 2
Layer 3
Arbib, M.A., Érdi, P., and Szentágothai, J. (1997). Neural Organization: Structure, Function and Dynamics. The MIT Press, Cambridge, MA.
Braitenberg, V. (1984). Vehicles: Experiments in Synthetic Psychology. The MIT Press, Cambridge, MA.
Brooks, R.A. (1995). Intelligence without reason. in Building Agents out of Autonomous Behavior Systems, L. Steels, ed
Freeman, W.J. (2001). The neurodynamics of intentionality in animal brains may provide a basis for constructing devices that are capable of intelligent behavior. NIST Workshop on Metrics for Intelligence: Development of Criteria for Machine Intelligence, National Institute of Standards and Technology, Gaithersburg, MD.
Freeman, W.J. (1991). The physiology of perception. Scientific American, 264(2),
Harter, D., and Kozma, R. (submitted). Chaotic neurodynamics for autonomous agents. IEEE Transactions on Neural Networks.
Harter, D. and Kozma, R. (2004a). Navigation and cognitive map formation using aperiodic neurodynamics. From Animals to Animats 8: The Eighth International Conference on the Simulation of Adaptive Behavior, Los Angeles, CA.
Harter, D. and Kozma, R. (2004b). Aperiodic dynamics for appetitive/aversive behavior in autonomous agents. Proceedings of the 2004 IEEE International Conference on Robotics and Automation (ICRA), New Orleans, LA.
Kozma, R., Freeman, W.J. and Érdi, P. (2003). The KIV model – nonlinear spatio-temporal dynamics of the primordial vertebrate forebrain. Neurocomputing, 52-54:
Walter, G. (1951). A machine that learns. Scientific American, August
K-Ie
K-II E1 E2 E1 E2 I1 I2 + -
K-Ii I1 I2
Far from equilibrium, thermodynamic systems
Mechanisms of self-organization
competition
cooperation
autocatalytic loops
hierarchy & mesh
Aperiodic dynamics
Expectation or reafference
Embodiment
environment/organism coupling
Small worlds type divergent-convergent projecting connections
Results of measured lyapunov exponent of a KA-III when varying the projecting weight between the 3 layers from 0 to 100% of some initial configuration. Above 3 figures shows dynamics at 0, 0.25 and 1.0 scaling factors.
Supported by:
NCC
EIA
(Harter & Kozma, submitted
Kozma, Freeman & Erdi, 2003)