1 سمينار درس مباحث پيشرفته در شبكه هاي عصبي شبکه های عصبی ايمپالس و کوپلاژ پالس Impulse and pulse-coupled Neural Net پيمان گيفانی خرداد 84

2 information processing by the brain. DURING last few years we have witnessed a shift of the emphasis in the artificial neural network community toward spiking neural networks. Motivated by biological discoveries, many studies consider pulse-coupled neural networks with spike-timing as an essential component in In any study of network dynamics, there are two crucial issues which are 1) what model describes spiking dynamics of each neuron and 2) how the neurons are connected.

3 20 of the most prominent features of biological spiking neurons 20 of the most prominent features of biological spiking neurons. The goal of this section is to illustrate the richness and complexity of spiking behavior of individual neurons in response to simple pulses of dc current. What happens when only tens (let alone billions) of such neurons are coupled together is beyond our comprehension.

4

5 Which Model to Use for Cortical Spiking Neurons? To understand how the brain works, we need to combine experimental studies of animal and human nervous systems with numerical simulation of large-scale brain models. As we develop such large-scale brain models consisting of spiking neurons, we must find compromises between two seemingly mutually exclusive requirements: The model for a single neuron must be: 1) computationally simple, yet 2) capable of producing rich firing patterns exhibited by real biological neurons. Using biophysically accurate Hodgkin–Huxley-type models is computationally prohibitive, since we can simulate only a handful of neurons in real time. In contrast, using an integrate-and-fire model is computationally effective, but the model is unrealistically simple and incapable of producing rich spiking and bursting dynamics exhibited by cortical neurons.

6

7 Spike-timing Dynamics of Neuronal Groups Spiking Model by Izhikevich (2003) A neuronal network inspired by the anatomy of the cerebral cortex was simulated to study the self-organization of spiking neurons into neuronal groups. The network consisted of reentrantly interconnected neurons exhibiting known types of cortical firing patterns, receptor kinetics, short-term plasticity and long-term spike-timing dependent plasticity (STDP), as well as a distribution of axonal conduction delays. The dynamics of the network allowed us to study the fine temporal structure of emerging firing patterns with millisecond resolution. We found that the interplay between STDP and conduction delays gave rise to the spontaneous formation of neuronal groups — sets of strongly connected neurons capable of firing time-locked, although not necessarily synchronous, spikes. Despite the noise present in the model, such groups repeatedly generated patterns of activity with millisecond spike-timing precision. Exploration of the model allowed us to characterize various group properties, including spatial distribution, size, growth, rate of birth, lifespan, and persistence in the presence of synaptic turnover.

8

9 Synaptic Dynamics –Short-term Depression and Facilitation –Synaptic Conductance –Long-term Synaptic Plasticity –Spike-timing in Neuronal Groups

10 Time-locked spiking patterns

11

12 Distribution of groups on the simulated ‘cortical’ surface. Neurons within each group are marked by the same color.

13 self-reconstruction & Self-organization when synaptic turnover Even synaptic turnover, which changed all synapses in a random order during a 24 h period, did not affect the stability of spike-timing neuronal groups (yellow bins in Fig. 11). Repetitive activation of neuronal groups combined with STDP resulted in a self-reconstructing behavior of the model.

14 Parallel Processing Concurrent Processing Unprecedented Memory Capacity Competition and Cooperation between groups

15 Rhythmic Activity Rhythmic activity in the delta frequency range around 4 Hz. This is one of the four fundamental types of brain waves, sometimes called “deep sleep waves”, because it occurs during dreamless states of sleep, infancy, and in some brain disorders. As the synaptic connections evolve according to STDP, the delta oscillations disappear, and spiking activity of the neurons becomes more Poissonian and uncorrelated. After a while, gamma frequency rhythms in the range Hz appear. This kind of oscillations, implicated in cognitive tasks in humans and other animals, play an important role in the activation of polychronous groups.

16 Cognitive Computations Rate to Spike-Timing Conversion Neurons in the model use spike-timing code to interact and form groups. However, the external input from sensory organs, such retinal cells, hair cells in cochlear, etc., arrives as the rate code, i.e., encoded into the mean firing frequency of spiking. How can the network convert rates to precise spike timings? Open circles - excitatory neurons, black circles - inhibitory neurons. inhibitory postsynaptic potential (IPSP). Notice that synchronized inhibitory activity occurs during gamma frequency oscillations. Thus, the network constantly converts rate code to spike-timing code (and back) via gamma rhythm. The functional implications of such a non-stop conversion are not clear.

17 Representations of Memories and Experience hypothesize that polychronous groups could represent memories and experience. In the simulation above, no coherent external input to the system was present. As a result, random groups emerge; that is, the network generates random memories not related to any previous experience. Persistent stimulation of the network with two spatio-temporal patterns result in emergence of polychronous groups that represent the patterns. the groups activate whenever the patterns are present.

18 Consciousness When no stimulation is present, there is a spontaneous activation of polychronous groups. If the size of the network exceeds certain threshold, a random activation of a few groups corresponding to a previously seen stimulus may activate other groups corresponding to the same stimulus so that the total number of activated groups is comparable to the number of activated groups that occurs when the stimulus is present. Not only such an event excludes all the other groups not related to the stimulus from being activated, but from the network point of view, it would be indistinguishable from the event when the stimulus is actually present. One can say that the network “thinks” about the stimulus. A sequence of spontaneous activations corresponding to one stimulus, then another, and so on, may be related to the stream of thought and primary consciousness.

19 NEURAL EXCITABILITY, SPIKING AND BURSTING The brain types of cells: neurons, neuroglia, and Schwann cells. neurons are believed to be the key elements in signal processing. neurons in the human brain each can have more than synaptic connections with other neurons. Neurons are slow, unreliable analog units, yet working together they carry out highly sophisticated computations in cognition and control. Action potentials play a crucial role among the many mechanisms for communication between neurons.

20 Synchronization and locking are ubiquitous in nature Anti-phase SynchronizationIn-phase Synchronization No LockingOut-of-phase Synchronization

21 Neural Excitability Excitability is the most fundamental property of neurons allowing communication via action potentials or spikes. From mathematical point of view a system is excitable when small perturbations near a rest state can cause large excursions for the solution before it returns to the rest. Systems are excitable because they are near bifurcations from rest to oscillatory dynamics. The type of bifurcation determines excitable properties and hence neuro- computational features of the brain cells. Revealing these features is the most important goal of mathematical neuroscience. The neuron produse spikes periodically when there is a large amplitude limit cycle attractor, which may coexist with the quiescent state.

22 Most of the bifurcations discussed here can be illustrated using a two- dimensional (planar) system of the form Much insight into the behavior of such systems can be gained by considering their nullclines. the sets determined by the conditions f(x, y) = 0 or g(x, y) = 0. When nullclines are called fast and slow, respectively. Since the language of nullclines is universal in many areas of applied mathematics

23

24

25 An excitable system at an Andronov/Hopf bifurcation possesses an important information processing capability: Its response to a pair (or a sequence) of stimuli depends on the timing between the stimuli relative to the period of the small amplitude damped oscillation at the equilibrium.

26 Selective communication and multiplexing The same doublet may or may not elicit response in a postsynaptic neuron depending on its eigenfrequency. This provides a powerful mechanism for selective communication between such neurons. In particular, such neurons can multiplex send many messages via a single transmission line.

27 Phase of the sub-threshold oscillation

28 Fast sub threshold oscillation If a neuron exhibits fast subthreshold oscillation of its membrane potential, then its response to a brief strong input may depend on the amplitude and timing of the input. If the input is weak, so that it never evokes an action potential, but can modulate the subthreshold oscillation, by changing its phase, so that the neuron would react dierently to a future strong pulse. From the FM interaction theory it follows that the phase of subthreshold oscillation can be affected only by those neurons with a certain resonant frequency. By changing the frequency of the subthreshold limit cycle, the neuron can control the set of the presynaptic neurons that can modulate its dynamics. The entire brain can rewire and regulate itself dynamically without changing the synaptic hardware.

29 Bursters When neuron activity alternates between a quiescent state and repetitive spiking, the neuron activity is said to be bursting. It is usually caused by a slow voltage- or calcium-dependent process that can modulate fast spiking activity. There are two important bifurcations associated with bursting: –Bifurcation of a quiescent state that leads to repetitive spiking. –Bifurcation of a spiking attractor that leads to quiescence. These bifurcations determine the type of burster and hence its neuro-computational features. An example of "fold/homoclinic" (square-wave) bursting. When a slow variable changes, the quiescent state disappears via fold bifurcation and the periodic spiking attractor disappears via saddle homoclinic orbit bifurcation

30 Bursting So far we have considered spiking mechanisms assuming that all parameters of the neuron are fixed. From now on we drop this assumption and consider neural systems of the form Fast spiking Slow modulation where u represents slowly changing parameters in the system.

31

32

33

34

35 Synchronization of bursters

36 Coupling

37 Applications Pattern Recognition Via Synchronization in Phase-Locked Loop Neural Networks –a novel architecture of an oscillatory neural network that consists of phase-locked loop (PLL) circuits. It stores and retrieves complex oscillatory patterns as synchronized states with appropriate phase relations between neurons.

38

39 Synchronization of MEMS Resonators and Mechanical Neurocomputing Combination two well-known and established concepts: microelectromechanical systems (MEMS) and neurocomputing. First, we consider MEMS oscillators having low amplitude activity and we derive a simple mathematical model that describes nonlinear phase-locking dynamics in them. Then, we investigate a theoretical possibility of using MEMS oscillators to build an oscillatory neurocomputer having autocorrelative associative memory. The neurocomputer stores and retrieves complex oscillatory patterns in the form of synchronized states with appropriate phase relations between the oscillators. Thus, we show that MEMS alone can be used to build a sophisticated information processing system (U.S. provisional patent 60/178,654).

40 PCNNW and Noise Automatic edge and target extraction based on Pulsed- Coupled NN wavelet theory (PCNNW)

41 با سپاس از تحمل شما