Chaos in Neural Network Theme presentation Cui, Shuoyang 03/08/2005

Artificial Neural Network (ANN) is Neural Networks are a different paradigm for computing: neural network machines are based on the processing/memory abstraction of human information processing. neural networks are based on the parallel architecture of animal brains.

Neural networks versus conventional computers Conventional computers use an algorithmic approach. the problem solved are that we already understand and know how to solve. The network is composed of a large number of highly interconnected processing elements(neurones) working in parallel to solve a specific problem They cannot be programmed to perform a specific task Neural networks and conventional algorithmic computers complement each other The disadvantage is that the network finds out how to solve the problem by itself, its operation can be unpredictable. An important application of neural networks is pattern recognition

Neural networks are a form of multiprocessor computer system, with simple processing elements a high degree of interconnection simple scalar messages adaptive interaction between elements

Architecture of neural networks Feed-forward networks allowing signals to travel one way only Feedback networks have signals travelling in both directions by introducing loops in the network

Chaos and random system Chaos is statistically indistinguishable from randomness, and yet it is deterministic and not random at all. Chaotic system will produce the same results if given the same inputs, it is unpredictable in the sense that you can not predict in what way the system's behavior will change for any change in the input to that system. a random system will produce different results when given the same inputs.

Chaos has periodic, predictable behavior and totally random behavior It is random-appearing, and yet has a large degree of underlying order. Skarda & Freeman(1987) found chaotic activity in the brain Freeman(1991) decided that chaos "may be the chief property that makes the brain different from an artificial-intelligence machine" For brain of human: researchers believe chaotic background behavior is necessary for the brain to engage in continual learning. Background of Chaos

Chaotic neural networks Chaotic neural networks offer greatly increase memory capacity.Each memory is encoded by an Unstable Periodic Orbit (UPO) on the chaotic attractor. A chaotic attractor is a set of states in a system's state space with very special properties: the set is an attracting set. So that the system, starting with its initial condition in the appropriate basin, eventually ends up in the set. Tnd most important, once the system is on the attractor nearby states diverge from each other exponentially fast. Small amounts of noise are amplified.

A Novel Chaotic Neural Network Architecture Nigel Crook and Tjeerd olde Scheper

The delayed feedback method is considered to be best suited to the control of chaos in neural networks. the delay feedback method does not rely on a priori knowledge the delayed feedback method does not specify which UPO is to be stabilized delays in signal transmission are inherent in all biological neuronal networks. The feedback control method amounts to delayed inhibition

Pyragas’s delayed feedback method P(y,x) and Q(y,x) govern the chaotic dynamics of the system; output variable, y(t); input signal, F(t);  a delay time,  strength of the feedback K.

variable y repeats a value specified by the delay at the earlier time. F(t) system back to a state (periodic system coming) periodic system F(t) will become very small this method UPOs of different periodicities can be controlled

Each unit in the Chaotic Layer is governed by the following discrete time equations: Modified by K. Aihara, T. Takabe, and I. Tsuda in Chaotic Neural Networks yi(t) : the internal state of unit i at time t, (0 < <1) a and w : parameters of the Aihara model M : the number of units in the Chaotic Layer N : the number of units in each inhibitory cluster. wij : a weight of the connections between units in the chaotic layer ija : time delay xi(t) : the output activation of chaotic unit i at time t. kij : the weights of connections from the inhibitory units to the chaotic units zj(t) : the activation of chaotic unit j at time t f(y) is given by:

Inputs : xi(t), the activation of the chaotic unit i at time t. & xi(t -Dji), which is the activation of chaotic unit i at time t -Dji, where Dji is a randomly selected time delay. Each inhibitory unit one output + two inputs Each inhibitory unit has a different randomised time delay connection with the associated chaotic unit.

Compete in the inhibitory units within a cluster L : the number of input units Ik(t) : the activation of the kth input unit at time t. winner :the inhibitory unit with the smallest value for h(t) activation value is And then activation values of other units in that inhibitory cluster 0

Experiment 1 input unit 3 units in each inhibitory cluster 4 chaotic units The network was then iterated for a further 200 times steps. The input patterns used were: Input sequence (a) 1.0, 0.5 The activations of the input units at each time step are governed entirely by the input sequences consisting of discrete values in the range 0 to 1.

For sequence (a) I(1) = 1.0,I(2) =0.5, I(3) =1.0, I(4) =0.5, etc. The activations of 2 units from the Chaotic Layer stabilized into periodic after 200 times each of the chaotic units is stabilised to orbits with different periods

The activations of the units in 2 of the inhibitory clusters

References Nigel Crook and Tjeerd olde Scheper, A Novel Chaotic Neural Network Architecture W.J. Freeman and J.M. Barrie, Chaotic oscillations and the genesis ofmeaning in cerebral cortex. Temporal Coding in the Brain T. Shinbrot, C. Grebogi, E. Ott, and J.A. Yorke, Using Small Perturbations to Control Chaos Nature, M.R. Guevara, L. Glass, M.C. Mackey, and A. Shrier,Chaos in Neurobiology A. Babloyantz and C. Lourenco, Brain Chaos and Computation C. Lourenco and A. Babloyantz, Control of spatiotemporal chaos in neural networks