1. Aperiodic Dynamics for Appetitive/Aversive Behavior in Autonomous Agents
Derek Harter and Robert Kozma
University of Memphis
Computational Neurodynamics Lab
ICRA’04
April 26, 2004

2. Discrete difference eq.
ANN
Discrete KA K
Discrete difference eq.
Continuous ODE
Spiking
Population
Feed-Forward
Highly Recurrent
Back-Propogation
Hebbian

3. Table 1: Characterization of the hierarchy of K-sets
KA – Set Hierarchy
KA-0 Unit Difference Equation:
Table 1: Characterization of the hierarchy of K-sets
Type
Structure
Inhrt Dynamics
Exs. in brain*
KA-0
Single Unit
Nonlinear I/O function
All higher level K sets are composed of K0 units
KA-I
Populations of excitatory or inhibitory units
Fixed point convergence to zero or nonzero value
PG, DG, BG, BS
KA-II
Interacting populations of excitatory and inhibitory units
Periodic, limit cycle oscillations; frequency in the gamma band
OB, AON, PC, CA1, CA3, CA2, HT, BG, BS, Amygdala
KA-III
Several interacting KII and KI sets
Aperiodic, chaotic oscillations
Cortex, Hippocamp, Midline Forebrain
KA-IV
Interacting KIII sets
Spatio-temporal dynamics with global phase transitions (itinerancy)
Hemisphere cooperation of cortical, HF and MF by the Amygdala
* Notations: PG – periglomerular; OB - olfactory bulb; AON - anterior olfactory nucleus; PC- prepyriform cortex; HF - hippocampal formation; DG - dentate gyrus; CA1, CA2, CA3 - curnu ammonis sections of the hippocampus; MF - midline forebrain; BG - basal ganglia; HT - hypothalamus; DB - diagonal band; SP - septum
Transfer function:
KA-III E1 E2 I1 I2
Layer 1 d
receptors
Layer 2
Layer 3
KA-Ie
KA-Ii
Named in honor of Katchalsky, a famous and influential neuroscientist)
Maximal slope of transfer function is displaced to the excitatory side of the resting level. An excitatory input to the population raises the activity level of its neurons, and also increases their sensitivity to input from each other, so that they interact more strongly.
The asymmetry of the sigmoid function means that the OB is destabilized by input. E1 E2 I1 I2
KA-II E1 E2 I1 I2 + -

4. KA-III =0.058 =0.-044 =0.0109 =0.152 E1 E2 I1 I2 Layer 1 d
receptors
Layer 2
Layer 3

5. KA, K-Set and Rat Power Spectra
KA-III model – Layer 1

6. Appetitive/Aversive Behavior using a KA-III

7. 3 Environment Key 1 1 1 Agent Morphology 1 3 2 2 Edible food source
Poisonous
food source
Agent Morphology 1 3
Distance sensor
Light sensor
Front
Wheel &
Motor 2 2

8. 3 Environment Key 1 1 1 Agent Morphology 1 3 2 2 Edible food source
Poisonous
food source
Agent Morphology 1 3
Distance sensor
Light sensor
Front
Wheel &
Motor 2 2

9. Architecture of Appetitive/Aversive Experiment
Mapp
Mave
DS0
DS1
DS2
DS3
DS4
DS5
LS7
LS0
LS1
LS2
LS3
LS4
LS5
LS6
Left
Obs No
Obs
Right
Obs
Left
Grad
Right
Grad + + + + + + + - - -
Turn
Left
Move
Fwd
Turn
Right
Left
App
Right
App
Left
Ave
Right
Ave - - - + - + - - + + - - + + - + +
Front 2 3 1 4 5 LM RM
Wheel &
Motor 7 6
Distance sensor
Light sensor

10. Architecture of Appetitive/Aversive Experiment
Smell (Light)
(KA-0 10)
KA-III OB
(8x8 KA-II)
AON
(2x2 KA-II) V PC
(8x8 KA-II)
Mapp
Mave
Valence
Hebbian Modification
Tasteapp
Tasteave

11. (Loc 1) (Loc 2) (Loc 3) (Loc 4) (Loc 5) (Loc 6) (Loc 7) (Loc 8)a c d b
(Loc 1) (Loc 2) (Loc 3) (Loc 4) (Loc 5) (Loc 6) (Loc 7) (Loc 8)
(Test d) (Test c) (Test b) (Test a)

12. Agent Path No Learn (left) and Learn (Right)
Environment Key 1
Edible food
source 1
Poisonous
food source 1 3 1 3 1 1 3 3 2 2 2 2

13. No Learning
Learning
Edible
Poisonous 59 82 98 40

14. Conclusion
The KA simplification represents a new and very useful tool for exploring mesoscopic level neurodynamic models in autonomous agent simulations.
Efficiency
Comparability
Analyzability
Level of Abstraction
The KA-III forms aperiodic dynamics with the same temporal and spectral characteristics of biological populations.
It has been demonstrated that KA can be used as control mechanisms in autonomous agents.
We have also demonstrated that KA can form aperiodic attractors in autonomous agents that represent environmental meanings and are useful in guiding behavior.