1 Properties of the Duration of Transient Oscillations in a Ring Neural Network Yo Horikawa and Hiroyuki Kitajima Kagawa University Japan.

Slides:



Advertisements
Similar presentations
Unijunction Transistor
Advertisements

MATHEMATICAL MODELS OF SPIKE CODES AS DETERMINISTIC DYNAMICAL SYSTEMS Eleonora Catsigeras Universidad de la República Uruguay Neural Coding Montevideo.
Properties of State Variables
Generated Waypoint Efficiency: The efficiency considered here is defined as follows: As can be seen from the graph, for the obstruction radius values (200,
Part II: Population Models BOOK: Spiking Neuron Models, W. Gerstner and W. Kistler Cambridge University Press, 2002 Chapters 6-9 Laboratory of Computational.
CH12: Neural Synchrony James Sulzer Background Stability and steady states of neural firing, phase plane analysis (CH6) Firing Dynamics (CH7)
The back-propagation training algorithm
Lecture #8 Circuits with Capacitors
4. Review of Basic Probability and Statistics
Calcium Oscillation in the Pollen Tube Growth Presented by: XIA,Fan
EE42/100 Lecture 9 Topics: More on First-Order Circuits Water model and potential plot for RC circuits A bit on Second-Order Circuits.
Lecture 9, Slide 1EECS40, Fall 2004Prof. White Lecture #9 OUTLINE –Transient response of 1 st -order circuits –Application: modeling of digital logic gate.
Series RLC Network. Objective of Lecture Derive the equations that relate the voltages across a resistor, an inductor, and a capacitor in series as: the.
F. Cheung, A. Samarian, W. Tsang, B. James School of Physics, University of Sydney, NSW 2006, Australia.
Hossein Sameti Department of Computer Engineering Sharif University of Technology.
Moment Generating Functions 1/33. Contents Review of Continuous Distribution Functions 2/33.
Capacitive Transients,
Large-amplitude oscillations in a Townsend discharge in low- current limit Vladimir Khudik, Alex Shvydky (Plasma Dynamics Corp., MI) Abstract We have developed.
Moment Generating Functions
Basic signals Why use complex exponentials? – Because they are useful building blocks which can be used to represent large and useful classes of signals.
Chapter 5 Trajectory Planning 5.1 INTRODUCTION In this chapters …….  Path and trajectory planning means the way that a robot is moved from one location.
Chapter 5 Trajectory Planning 5.1 INTRODUCTION In this chapters …….  Path and trajectory planning means the way that a robot is moved from one location.
Artificial Neural Network Supervised Learning دكترمحسن كاهاني
Advanced EM - Master in Physics The (GENERAL) solution of Maxwell’s equations Then for very small r, outside the charge region but near it,
Dynamical network motifs: building blocks of complex dynamics in biological networks Valentin Zhigulin Department of Physics, Caltech, and Institute for.
Module 1: Statistical Issues in Micro simulation Paul Sousa.
2.4 Fundamental Concepts of Integral Calculus (Calc II Review)
1 CHAPTER 6 HEAT TRANSFER IN CHANNEL FLOW 6.1 Introduction (1) Laminar vs. turbulent flow transition Reynolds number is where  D tube diameter  u mean.
Monte Carlo Methods So far we have discussed Monte Carlo methods based on a uniform distribution of random numbers on the interval [0,1] p(x) = 1 0  x.
Multiple attractors and transient synchrony in a model for an insect's antennal lobe Joint work with B. Smith, W. Just and S. Ahn.
Differential Equations Copyright © Cengage Learning. All rights reserved.
1 What is small scale fading? Small scale fading is used to describe the rapid fluctuation of the amplitude, phases, or multipath delays of a radio signal.
Time-dependent Schrodinger Equation Numerical solution of the time-independent equation is straightforward constant energy solutions do not require us.
Study on synchronization of coupled oscillators using the Fokker-Planck equation H.Sakaguchi Kyushu University, Japan Fokker-Planck equation: important.
Synchronization in complex network topologies
Physics 361 Principles of Modern Physics Lecture 11.
Chapter 2 Mathematical Background Professor Shi-Shang Jang Chemical Engineering Department National Tsing-Hua University Taiwan March,
B. Stochastic Neural Networks
Week 3 Capacitance & Inductance Transitions Try your hand at graphing these functions Graph: y = e x Graph: y = e -x Graph: y = 1 - e -x.
Novel Cascaded Chaotic Masking for Secure Communications
Sara A. Solla Northwestern University
Long time correlation due to high-dimensional chaos in globally coupled tent map system Tsuyoshi Chawanya Department of Pure and Applied Mathematics, Graduate.
Designing High-Capacity Neural Networks for Storing, Retrieving and Forgetting Patterns in Real-Time Dmitry O. Gorodnichy IMMS, Cybernetics Center of Ukrainian.
Signals and Systems Analysis NET 351 Instructor: Dr. Amer El-Khairy د. عامر الخيري.
Ch 9. Rhythms and Synchrony 9.7 Adaptive Cooperative Systems, Martin Beckerman, Summarized by M.-O. Heo Biointelligence Laboratory, Seoul National.
1 Noise-Induced Increases in the Duration of Transient Oscillations in Ring Neural Networks and Correlations in the Periods of Ring Oscillators Yo Horikawa.
Chapter 5 Elementary Stochastic Analysis Prof. Ali Movaghar.
Wednesday, Aug. 27, 2003 PHYS , Fall 2003 Dr. Jaehoon Yu 1 PHYS 1443 – Section 003 Lecture #2 Wednesday, Aug. 27, 2003 Dr. Jaehoon Yu 1.Dimensional.
Arthur Straube PATTERNS IN CHAOTICALLY MIXING FLUID FLOWS Department of Physics, University of Potsdam, Germany COLLABORATION: A. Pikovsky, M. Abel URL:
Source-Free Series RLC Circuits.
Spatiotemporal Networks in Addressable Excitable Media International Workshop on Bio-Inspired Complex Networks in Science and Technology Max Planck Institute.
1 Exponential Transient Oscillations and Standing Pulses in Rings of Coupled Symmetric Bistable Maps Yo Horikawa Kagawa University Japan.
1 10. Harmonic oscillator Simple harmonic motion Harmonic oscillator is an example of periodic motion, where the displacement of a particle from.
Mean Field Methods for Computer and Communication Systems Jean-Yves Le Boudec EPFL Network Science Workshop Hong Kong July
Cmpe 588- Modeling of Internet Emergence of Scale-Free Network with Chaotic Units Pulin Gong, Cees van Leeuwen by Oya Ünlü Instructor: Haluk Bingöl.
1 Experiments on Transient Oscillations in a Circuit of Diffusively Coupled Inverting Amplifiers Yo Horikawa Kagawa University, Japan.
Discrete-time Markov chain (DTMC) State space distribution
The Cournot duopoly Kopel Model
Logarithmic Functions and Their Graphs
Mechatronics Engineering
Our favorite simple stochastic process.
Collins Assisi, Mark Stopfer, Maxim Bazhenov  Neuron 
OCNC Statistical Approach to Neural Learning and Population Coding ---- Introduction to Mathematical.
2.4 Fundamental Concepts of Integral Calculus (Calc II Review)
A Dynamic System Analysis of Simultaneous Recurrent Neural Network
Statistical Methods for Data Analysis Random number generators
Collins Assisi, Mark Stopfer, Maxim Bazhenov  Neuron 
Lecture 6: Time Domain Analysis and State Space Representation
Volume 6, Issue 4, Pages e3 (April 2018)
C H A P T E R 22 Transients. C H A P T E R 22 Transients.
Presentation transcript:

1 Properties of the Duration of Transient Oscillations in a Ring Neural Network Yo Horikawa and Hiroyuki Kitajima Kagawa University Japan

2 1. Background Ring network of sigmoidal neuron models with inhibitory unidirectional coupling When the coupling gain: |g|> g th (N) ≥ 1, ・ Number of neurons: odd (N = 2M + 1) → Stable oscillation ・ Number of neurons: even (N = 2M) → Stable steady states (e.g. x 2k ≈ 1, x 2k-1 ≈ -1 (0 ≤ k ≤ m) τdx 1 /dt = -x 1 + f(x N ) τdx n /dt = -x n + f(x n - 1 ) (2 ≤ n ≤ N) (A1) x n : state of neuron n N: number of neurons f(x) = tanh(gx): output of neuron g: coupling gain (g < 0) τ: time constant 4 5 N

3 ・ Number of neurons is odd (N = 2M + 1). → Ring oscillator: Ring of inverters (NOT gates) Stable oscillations of rectangular waves 4 5 2M t τdx 1 /dt = -x 1 + f(x N ) τdx n /dt = -x n + f(x n - 1 ) (2 ≤ n ≤ N) (A1) x n : state of neuron n N: number of neurons f(x) = tanh(gx): output of neuron g: coupling gain (g < 0) τ: time constant xn(t)xn(t)

4 ・ Number of neurons is even (N = 2M). → Network is bistable. However, transient states from random initial values can be extremely long. Transient oscillations lasted more than a month in computer simulation with a workstation for N = M2M x = (+ - ・・・ + -) OR x = (- + ・・・ - +) Simulation N = 40, g = -10.0, τ= 1.0, x n (0) ~ N(0, ) (1 ≤ n ≤ 40) Circuit experiment n = 40, C = 10μF, R = 10kΩ (time constant: CR = 0.1s) Oscillation lasts about 45 minutes.

5 The transient oscillations are traveling waves of the boundaries of separated blocks. Scenario to the steady state The neurons are separated into two blocks in which the signs of the states of the neurons change alternately. ( ・・・ ・・・ -+-+). Two boundaries (inconsistencies) of the two blocks move in the direction of the coupling by changing the signs of the states of the neurons. ( ・・・ ・・・ -+-+) →( ・・・ ・・・ -+-+) →( ・・・ ・・・ -+-+) The velocities of the boundaries differ only slightly and continue to move for a long time. ( ・・・ ・・・ -+-+) →( ・・・ ・・・ -+-+) They finally merge and the network reaches the steady state.

6 We consider excitatory coupling instead of inhibitory coupling for simplicity. Neurons are separated in two blocks of the same signs. ( ) ( ) ( ) ( ) ( ) ( ) ( ) Demonstration τdx 1 /dt = -x 1 + f(x N ) τdx n /dt = -x n + f(x n - 1 ) (2 ≤ n ≤ N) (A2) x n : state of neuron n f(x) = tanh(gx): output of neuron g: coupling gain (g > 1) t

7 2. Velocity of the boundaries Random initial values of x n (0) (1 ≤ n ≤ N) (after short time) → Propagating two blocks ( ) Difference between the velocities of two blocks: v 0 – v 1 < 0 → Changes in the length of a smaller block: l(t) with dl(t)/dt = v 0 – v 1 → Duration T of the oscillation with the condition l(T) = 0 v1v1 v0v0 l(t)l(t) l(t): the length of a smaller block The number of neurons in the block

8 Propagation time of the boundary per neuron: Δt → Velocity of the block per neuron: v = 1/ Δt Sigmoidal function: f(x) = tanh(gx) → Sign function: sgn(x) τdx n /dt = -x n - 1 (x n -1 < 0) = -x n + 1 (x n -1 > 0) (A3) τdx n /dt = -x n + 1τdx n /dt = -x n - 1 x(t 0 ) ≈ - 1 n n

9 τdx n /dt = -x n + 1 (x n -1 > 0) x n (t 1 ) = exp(-(t 1 - t 0 )/τ)(x n (t 0 ) - 1) + 1 ≈ -2exp(-(t 1 - t 0 )/τ) + 1 τdx n /dt = -x n - 1 (x n -1 < 0) x n (t 1 +Δt 1 ) = exp(-Δt 1 /τ)(x n (t 1 ) + 1) - 1 = 0 Δt 1 =τlog{2 - 2exp[-(t 1 - t 0 )/τ]} =τlog2 + log{1 - exp[-(t 1 - t 0 )/τ]} v 1 = 1/Δt 1 = 1/[τlog{2 - 2exp[-(t 1 - t 0 )/τ]}] ≈ 1/[τlog{2 - 2exp[log2 ・ l(t)]}] v 0 = 1/Δt 0 = 1/[τlog{2 - 2exp[-(t 0 – t -1 )/τ]}] ≈ 1/[τlog{2 - 2exp[log2 ・ (N - l(t))]}] dl(t)/dt = v 0 - v 1 = 1/Δt 0 - 1/Δt 1 ≈ 1/[τlog{2 - 2exp[log2 ・ (N - l(t))]}] - 1/[τlog{2 - 2exp[log2 ・ l(t)]}] Δt ≈ τlog2 (t 1 - t 0 ) → ∞

10 dl/dt ≈ 1/[τlog{2 - 2exp[log2 ・ (N - l)]}] - 1/[τlog{2 - 2exp[log2 ・ l]}] = 1/{τ[log2 + log( (N - l) )]} - 1/{τ[log2 + log( l )]} ≈ 1/[τ(log (N - l) )] - 1/[τ(log l )] ≈ 1/τ·[1/log (N - l) /(log2) 2 - (1/log l /(log2) 2 )] = 1/[τ(log2) 2 ]·(2 -(N - l) - 2 -l ) = k(exp(-c(N - l)) - exp(-cl)) (k = 1/[τ(log2) 2 ], c = log2) (A4) The velocity of the boundary depends on an exponential of distance to the forward boundary. The difference between the velocites of two boundaries decreases exponentially with the length l and N - l of the blocks. Changes in the length of the blocks is exponentially small when the block lengths are large. n = 1 N

11 3. Duration of the transient oscillations dl/dt = k(exp(-c(N - l)) - exp(-cl)) (k = 1/[τ(log2) 2 ], c = log2) This equation is solved as exp(cl(t)) = exp(cN/2)tanh(-exp(-cN/2)ckt + arctanh(exp(cl 0 - cN/2))) (l 0 = l(0) (0 ≤ l 0 ≤ N/2)) (A5) The two blocks merge, i.e. a smaller block disappear. → The oscillation ceases. The duration T of the transient oscillations is given with l(T) = 0. T = 1/(ck)·exp(cN/2){arctanh[exp(c(l 0 - N/2))] – arctanh[exp(-cN/2)]} (A6) T 10 T 15 T 18 T 22

12 A simple form of the duration T(l 0 ) when the number N of neurons is large is obtained by letting N be infinity (N → ∞). dl/dt = -kexp(-cl) l(t) = 1/c·log(exp(cl 0 ) - ckt) T = (exp(cl 0 ) - 1))/ck = τlog2·(2 l 0 - 1) (l(T) = 0) (k = 1/[τ(log2) 2 ], c = log2, l 0 = l(0) (0 ≤ l 0 ≤ N/2)) (A7) The duration of the transient oscillations increases exponentially with the length of a smaller block, i.e. the number of neurons in the block. Fig. A1. Duration T of the transient oscillations with the initial block length l 0 (1 ≤ l 0 ≤ 19) and N = 40.

13 4. Distribution of the duration under random initial conditions Random initial states x n (0) (1 ≤ n ≤ N) of neurons ~ i. i. d. → The initial length of a smaller block is distributed uniformly in (0, N/2). l 0 ~ U(0, N/2). The probability density function h(T) of the duration T of the transient oscillations is derived with h(T) = |dT(l 0 ; N)/dl 0 | -1 /(N/2) = 2kexp(-cN/2)cosech[2(exp(-cN/2)ckT + arctanh(exp(-cN/2)))]∙2/N = 2/[τ(log2) 2 ]∙2 -N/2 cosech[2(2 -N/2 T/[τlog2] + arctanh(2 -N/2 ))]∙2/N (A8)

14 The distribution of the duration T under random initial condition. h(T) = 2/[τ(log2) 2 ]∙2 -N/2 cosech[2(2 -N/2 T/[τlog2] + arctanh(2 -N/2 ))]∙2/N (A8) A cut-off point: T c = τlog2·2 N/2 ・ T < T c h(T) ≈ k/(ckT+1)∙2/N = 1/[τ(log2) 2 (T/(τlog2) + 1)]∙2/N (0 ≤ T ≤ 1/ck·(exp(cN/2) - 1) = τlog2·(2 N/2 - 1)) (A9) The duration T is thus distributed in the form of 1/T. ・ T > T c h(T) ≈ λexp(-λT) (λ≈ 2exp(-cN/2)ck = 1/[2 N/2-1 ∙τ(log2)]) (A10) The cut-off point T c increases exponentially with the number of neurons. Proportion of the duration of the oscillations over the cut-off is inversely proportional to the number N of neurons. Pr{T < T c } ∝ 1/N → The inverse power-law distribution dominates as N increases.

15 Fig. A2. Probability density function h of the duration T of the transient oscillations in the network with the numbers N of neurons 10 (a), 20 (b), 40 (c). h ∝ 1/T

16 The mean m, the variance σ 2 and the coefficient of variation CV of the duration T is derived with Eq. (A9). m(T(N)) = 2(exp(cN/2) - 1 – cN/2)/(c 2 kN) = 2τ(2 N/ log2/2∙N)/N σ 2 (T(N)) = (exp(cN) - 4exp(cN/2) cN)/(c 3 k 2 N) - {m(T(N)} 2 CV(T(N)) = σ(T(N))/m(T(N)) ≈ (cN) 1/2 /2 = (log2) 1/2 /2∙N 1/2 (N » 1) (A11) Fig. A3. Mean m(T) of the duration of the transient oscillations vs Number N of neurons. The mean duration m(T) increases exponentially with the number of neurons.

17 5. Conclusion Properties of the transient oscillations in the ring networks of neurons with inhibitory unidirectional coupling were studied. ・ Kinematical model of the traveling waves in the networks The transient oscillations are the propagating boundaries of the blocks of the neurons at which the signs of the state of neurons are inconsistent. The kinematical equation of the propagating boundaries was derived. The difference between the velocities of the two propagating boundaries is exponentially small with the length of the blocks. ・ Exponential increases in the duration of the oscillations with the number of neurons The duration of the transient oscillations increases exponentially with the number of neurons. ・ Power-law distributions of the duration of the oscillations The distribution of the duration of the oscillations is in the form of the inverse power-law below the cut-off.

18 Transient states increasing exponentially with system size ・ Transient spatio-temporal chaos in coupled map lattices [A1] and reaction-diffusion equations [A2] ・ The length and number of cycles and transient time to them in asymmetric neural networks [A3] ・ Transient irregular firings in diluted inhibitory networks of pulse-coupled neurons [A4] ・ Transient well-controlled sequences in continuous-time Hopfield networks with Liapunov functions [A5]. These systems never reach their asymptotically stable states in a practical time when the system size is sufficiently large. Their functions, e.g. information processing in the nervous systems, may proceed in the transient states. The transient states thus play more important roles than the asymptotic states in actual systems. [8] K. Kaneko, Supertransients, spatiotemporal intermittency and stability of fully developed spatiotemporal chaos, Phys. Lett. A 149 (1990) pp [9] A. Wacker, S. Bose and E. Schöll, Transient spatio-temporal chaos in a reaction-diffusion model, Europhysics Letters 31 (1995) pp [10] U. Bastolla1 and G. Parisi, Relaxation, closing probabilities and transition from oscillatory to chaotic attractors in asymmetric neural networks, J. Phys. A 31 (1998) pp [11] R. Zillmera, R. Livib, A. Politic and A. Torcini, Desynchronized stable states in diluted neural networks, Neurocomputing 70 (2007) pp [12] J. Šíma and P. Orponen, Exponential transients in continuous-time Liapunov systems, Theoretical Computer Science 306 (2003) pp