1. Three classic ideas in neural networks
Sebastian Seung
HHMI and MIT

2. Three classic ideas Network structure-function relationships
Hebbian synaptic plasticity
Reward-dependent synaptic plasticity

3. Network motifs

4. Hebbian synaptic plasticity
test
induction
test A B
synaptic strength

5. Reward-dependent synaptic plasticity
Dopamine system

6. Zebra finch song Repetitions of a motif (0.5 to 1 sec)
A motif is composed of 3-7 syllables.

7. Birdsong Structure-function relationships Hebbian synaptic plasticity
Synaptic chain model of sequence generation
Hebbian synaptic plasticity
Learning sequences
Reward-dependent synaptic plasticity
Learning motor commands

8. Synaptic chain model of HVC
with Dezhe Jin and Fethi Ramazanoglu

9. HVC is crucial for song production
“high vocal center”
old name: “hyperstriatum ventrale pars caudale”

10. Two classes of HVC neurons
Projection neurons (HVCRA)
Project from HVC to RA
Excitatory
Interneurons (HVCIN)
Make synapses onto other HVC neurons.
Inhibitory
Ignore X-projecting neurons

11. Michale Fee

12. Spiking of projection neurons is temporally selective
Hahnloser et al., Nature (2002)
neuron 1
neuron 2
neuron 3
song-related activity is found in roughly half of 20,000 HVCRA neurons.

13. Spiking of interneurons is temporally unselective

14. The arrow of time
Hypothesis: There is a directionality to the network of excitatory synapses between projection neurons
Li and Greenside (2006)
Jin, Ramazanoglu, Seung (2007)
Jin (unpublished)

15. Chain-like network structures

16. Synaptic chain models Amari (1972): “Type II net” Abeles (1982)
Kleinfeld (1986)
Kanter and Sompolinsky (1986) …

17. The knee-jerk reflex Reflex behavior:
Rapid, involuntary, stereotyped response to a specific stimulus.

18. The reflex arc Chain of cause and effect Stimulus-response behaviors.
Temporal sequence generation.

19. Excitation and inhibition
Mooney and Prather (2005).

20. Chain with recurrent inhibition
70 groups of 30 projection neurons
300 interneurons
projection neuron to interneuron synapse 5% prob
interneuron to projection neuron synapse 10% prob

21. Numerical simulation of spiking activity

22. HVC spiking is temporally precise

23. Intrinsic cellular properties
duration is set by cellular properties.
Eliminates runaway instability
Less tuning is required to get multiple spikes per burst

24. Estes (1972)

26. Self-organization of HVC sequences by Hebbian synaptic plasticity
Dezhe Jin and Joseph Jun

27. Spike-timing dependent plasticity
Pre
Post
Strengthen
Pre
Post
Weaken
Pre
(Markram et al, 1997; Bi & Poo, 1998)
Post

28. STDP can reinforce chains
forward connections strengthened
backward connections weakened

29. Can STDP create chains? Low spontaneous activity
Intermittent transient input activates a fixed set of neurons.
A very short chain forms.
External input

30. Problem: STDP fails to generate long chains.
Levy, Horn, Meilijson, Ruppin (2001)

31. Solution: limit the fan-out of neurons

32. Structural vs. functional plasticity
Changes in the strength of existing synapses
Structural plasticity
Creation and elimination of synapses.
Changes in dendritic and axonal arbors.

33. Structural and functional plasticity are intertwined
Axonal branches with strong synapses are stabilized.
Those with weak synapses retract.
Meyer and Smith (2006)
Ruthazer, Li and Cline (2006)

34. Model of axon remodeling
Silent synapse
Weak synapse
Super synapse
Synaptic strength
Threshold 1
Threshold 2
If the number of supersynapses reaches Ns
All other synapses are “withdrawn.”

35. Self-organized synaptic chain
Saturated neuron
trials (2s each)
443 neuron (out of 1000) organized into 67 groups
Unsaturated neuron
Connection to next group
Connection beyond next group
Back connection

36. Learning motor commands
with Ila Fiete and Michale Fee

37. Learning phase I: sensory
father
son
Template acquisition
(days 20-45)

38. Learning phase II: sensorimotor
auditory feedback
learning to reproduce the stored template (days )

40. Song areas in the avian brain

41. RA activity

42. Sparse representation
HVC generates the sequence
later stages perform the motor map
similarity to hidden Markov model

43. Juvenile song is variable
This is a spectrogram of three consecutive song bouts of a juvenile zebra finch. The vertical axis is frequency, and the horizontal axis is time.
You can see that the songs look very different.

44. LMAN inactivation Olveczky, Andalman, Fee (in review)
The anterior forebrain pathway of birds is analogous to the basal ganglia in mammals.
The output of this pathway is nucleus LMAN.
Look what happens if LMAN is pharmacologically inactivated.
Olveczky, Andalman, Fee (in review)

45. Stereotyped song
You can see that the consecutive song bouts look almost identical now.
The novice still does not sing the right song, but he sings the same song every time.
By statistical measures, the song of this juvenile is now almost as stereotyped as that of an adult.

46. Hypothesis: LMAN is an experimenter
Doya and Sejnowski (2000)

47. A triune brain?
Evaluator
Experimenter
Performer

48. Evaluator?
A juvenile bird learns by comparing its own song to its memory of tutor song.
The brain area that performs this comparison is unknown.
The output of the comparison might be a scalar evaluation signal.

49. Trial and error learning
Success and failure can be evaluated.
Little insight into how to improve.
Slow climb up a gradient

50. How is performance evaluated?
Questions
How is performance evaluated?
Evaluator
Experimenter
How is the evaluation broadcast?
How are the experiments conducted?
Performer
How are synapses modified by learning?

51. Theory of empiric synapses
Synapses specialized for experimentation
Dynamic perturbation of conductances
Results of experiments are used to modify plastic synapses.

52. Practice makes perfect

54. Rules for synaptic modification
Stochastic gradient ascent
Fiete and Seung

56. Perturbative gradient estimation
eligibility
eligibility trace

57. Stochastic gradient learning
In the direction of the gradient, on average.

58. Stochastic gradient ascent
Instructions are in the correct direction only on average
Weak correlations require long times to detect
Learning in large networks is potentially slow.

59. a 1 2 1 2
HVC 3 3 RA
LMAN 4 4
motor b c d E e

60. Simulated song learning
tutor
before learning
after learning

61. Learning time independent of hidden layer size
error
independent of hidden layer size
linear in number of outputs
iterations/(No+2) b
error
tutor
200
800
iterations

62. A proposed experiment
HVC
plastic
evaluation RA
LMAN
empiric

63. Theories of gradient learning
Extrinsic experimentation
Empiric synapses
Intrinsic experimentation
Hedonistic synapses
Unreliable synaptic transmission
Hedonistic neurons (AR-I)
Irregular spiking