Presentation is loading. Please wait.

Presentation is loading. Please wait.

OCNC---2004 Statistical Approach to Neural Learning and Population Coding ---- Introduction to Mathematical.

Published byJeffry Wilcox Modified over 5 years ago

Similar presentations

Presentation on theme: "OCNC---2004 Statistical Approach to Neural Learning and Population Coding ---- Introduction to Mathematical."— Presentation transcript:

1 OCNC Statistical Approach to Neural Learning and Population Coding Introduction to Mathematical Neuroscience Shun-ichi Amari Laboratory for Mathematical Neuroscience RIKEN Brain Science Institute

2 BRAIN biological science information science
Computational neuroscience Neurocomputing Mathematical Neuroscience

3 II. Population Coding I. Mathematical Neuroscience ---- modern topics
----classical theories II. Population Coding ---- modern topics III. Bayesian Inference ---- its merits and critique

4 Dynamics of Neuro-Ensembles Dynamics of Neuro-Fields
1. Mathematical Neurons Dynamics of Neuro-Ensembles Dynamics of Neuro-Fields Learning and Self-Organization 5. Self-Organization of Neuro-Fields

5 I Mathematical Neurons
Simple model

6 output function u

7 spiking neuron integration-and-fire neuron rate coding

8 synchrony : spatial correlations firing probability

9 rate coding ensemble coding

10 1-layer network

11 Ensemble of networks macroscopic state macroscopic law

12

13 stability = =

14

15

16 Associative memory m pairs

17 Randomly generated Random matrix

18

19

20 II Dynamics of Neuro-Ensembles
spiking neurons : stochastic point process synchronization Ensemble coding : macrodynamics

21 Simple examples Bistable S S Multi-stable

22 oscillation Amari (1971); Wilson-Cowan (1972)

23 competitive model (winner-take-all)
・・・ (winner-share-some)

24 multistable associative memory decision process
(Anderson, Amari, Nakano, Kohonen Hopfield) decision process (Hopfield) travelling salesman problem

25 General Theory Transient Attractors stable state limit cycle
chaos (strange attractors)

26 Chaotic behavior random stable states chaos Chaotic memory search

27 Associative memory (content-addressable memory)
dynamics random attractor

28 Theory 1 =

29 =

30 Theory 2 …..

31 Macroscopic state Amari & Maginu, 1998

32 Dynamics of recalling processes
Direction cosine Correct pattern 1 time simulations

33 Direction cosine 1 theory time

34 simulation Threshold of recalling Spurious memory

35 Dynamics of temporal sequence (Amari, 1972)
non-monotonic output function Morita model

36 Nonmonotonic model non-monotonic

37 memory capacity : sparse
exact : no spurious memories chaotic oscillation inhibitory connection

38 Biology hippocumpus, Rolls et al Chaotic associative memory
Tonegawa et al CA3 Chaotic associative memory Aihara et al Chaotic search

39 Associative Memory Dynamics of a Chaotic Neural Networks
Each neuron model shows chaotic dynamics Synaptic weights are determined by an auto-correlation matrix of the stored patterns Stored Patterns t=0 t=1 t=2 t=3 t=4

40 t=5 t=6 t=7 t=8 t=9 t=10 t=11 t=12 t=13 t=14 t=15 t=16 t=17 t=18 t=19

41 t=20 t=21 t=22 t=23 t=24 t=25 t=26 t=27 t=28 t=29 t

42 III Field Dynamics of Neural Excitation
timing local excitations: travelling wave: oscillatory: memory decision Amari, Biol. Cybern,1978

43 Dynamics of Neural Fields

44

45

46 unstable stable

47 excitatory and inhibitory fields
traveling wave oscillation

48 Neural Learning (Hebbian)
classic theory ……… Information source I

49 Amari, Biol,Cybern,1978 Hebbian correlation generalized inverse
principal component analyzer Perceptron ….

50 Neural learning (STDP)
Spike-time dependent plasticity …. ……. emergence of synchrony LTP LTD

51 Learning Potential ………

52 1. Hebbian … 2. correlation associative memory …

53 3. generalized inverse least square

54 4. principal component analyzer
Amari (1978), Oja (1980) 5. perceptron

55 Theory of Learning Networks
Amari, IEEE Trans.C,1967 PDP: backprop; natural gradient

56 Learning algorithm

57 outer world

58 Self-organization …. …….

59 Proof RF of a neuron :

60

61 Special case Theorem: each : receptive field size of receptive field

62 1.resolution 2.topological property Self-Organizing Nerve Field
signal space neural field 1.resolution 2.topological property

63 higher-dimensional 2- dimensional
Topology Signal space Neural field higher-dimensional 2- dimensional position×orientation

64 orientation Signal space Neural field position

65 Self-organizing nerve field

66

67 dynamical stability patch structure
variational equation stability : Takeuchi&Amari, Biol.Cybern, 35, 63-72, 1979

68 Topological properties
emergence of block structure

69 Bayesian vs Fisherian. Any confrontation. --- histrical New framework
Bayesian vs Fisherian? Any confrontation? histrical New framework? Amari, New in neuroscience?

70 Bayesian framework mle vs map information and decision asymptotically equivalent regularization theory predictive distribution

71 Priors: uniform, Jeffreys, smooth Hierarchical (empirical) Bayes decision of prior

72 Singular statistical model Singular model and prior

Download ppt "OCNC---2004 Statistical Approach to Neural Learning and Population Coding ---- Introduction to Mathematical."

Similar presentations

Chapter3 Pattern Association & Associative Memory

Chapter3 Pattern Association & Associative Memory
On Bubbles and Drifts: Continuous attractor networks in brain models

On Bubbles and Drifts: Continuous attractor networks in brain models
Learning in Neural and Belief Networks - Feed Forward Neural Network 2001 년 3 월 28 일 20013329 안순길.

Learning in Neural and Belief Networks - Feed Forward Neural Network 2001 년 3 월 28 일 안순길.
Introduction to Neural Networks Computing

Introduction to Neural Networks Computing
2806 Neural Computation Self-Organizing Maps Lecture 9 2005 Ari Visa.

2806 Neural Computation Self-Organizing Maps Lecture Ari Visa.
Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.

Un Supervised Learning & Self Organizing Maps. Un Supervised Competitive Learning In Hebbian networks, all neurons can fire at the same time Competitive.
WINNERLESS COMPETITION PRINCIPLE IN NEUROSCIENCE Mikhail Rabinovich INLS University of California, San Diego ’

WINNERLESS COMPETITION PRINCIPLE IN NEUROSCIENCE Mikhail Rabinovich INLS University of California, San Diego ’
5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.

5/16/2015Intelligent Systems and Soft Computing1 Introduction Introduction Hebbian learning Hebbian learning Generalised Hebbian learning algorithm Generalised.
Artificial neural networks:

Artificial neural networks:
Learning crossmodal spatial transformations through STDP Gerhard Neumann Seminar B, SS 06.

Learning crossmodal spatial transformations through STDP Gerhard Neumann Seminar B, SS 06.
Artificial Spiking Neural Networks

Artificial Spiking Neural Networks
Image Segmentation by Complex-Valued Units Cornelius Weber and Stefan Wermter Hybrid Intelligent Systems School of Computing and Technology University.

Image Segmentation by Complex-Valued Units Cornelius Weber and Stefan Wermter Hybrid Intelligent Systems School of Computing and Technology University.
Introduction: Neurons and the Problem of Neural Coding Laboratory of Computational Neuroscience, LCN, CH 1015 Lausanne Swiss Federal Institute of Technology.

Introduction: Neurons and the Problem of Neural Coding Laboratory of Computational Neuroscience, LCN, CH 1015 Lausanne Swiss Federal Institute of Technology.
Part II: Population Models BOOK: Spiking Neuron Models, W. Gerstner and W. Kistler Cambridge University Press, 2002 Chapters 6-9 Laboratory of Computational.

Part II: Population Models BOOK: Spiking Neuron Models, W. Gerstner and W. Kistler Cambridge University Press, 2002 Chapters 6-9 Laboratory of Computational.
Biological Modeling of Neural Networks: Week 11 – Continuum models: Cortical fields and perception Wulfram Gerstner EPFL, Lausanne, Switzerland 11.1 Transients.

Biological Modeling of Neural Networks: Week 11 – Continuum models: Cortical fields and perception Wulfram Gerstner EPFL, Lausanne, Switzerland 11.1 Transients.
Introduction to the mathematical modeling of neuronal networks Amitabha Bose Jawaharlal Nehru University & New Jersey Institute of Technology IISER, Pune.

Introduction to the mathematical modeling of neuronal networks Amitabha Bose Jawaharlal Nehru University & New Jersey Institute of Technology IISER, Pune.
Brain-like design of sensory-motor programs for robots G. Palm, Uni-Ulm.

Brain-like design of sensory-motor programs for robots G. Palm, Uni-Ulm.
1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.

1 Chapter 11 Neural Networks. 2 Chapter 11 Contents (1) l Biological Neurons l Artificial Neurons l Perceptrons l Multilayer Neural Networks l Backpropagation.
Un Supervised Learning & Self Organizing Maps Learning From Examples 1 3 4 6 5 2 1 9 16 36 25 4.

Un Supervised Learning & Self Organizing Maps Learning From Examples
Basic Models in Neuroscience Oren Shriki 2010 Associative Memory 1.

Basic Models in Neuroscience Oren Shriki 2010 Associative Memory 1.

Similar presentations

Ads by Google