Plasticity and learning Dayan and Abbot Chapter 8

Introduction Learning occurs through synaptic plasticity Hebb (1949): If neuron A often contributes to the firing of neuron B, then the synapse from A to B should be strengthened –Stimulus response (Pavlov) –Converse: If neuron A does not contribute to the firing of B, the synapse is weakened –Hippocampus, neocortex, cerebellum

LTP and LTD at Shaffer collateral inputs to CA1 region of rat hippocampal slice. High stimulation yields LTP. Low stimulation yields LTD. NB: no stimulation yields no LTD

Function of learning Unsupervised learning (ch 10) –Feature selection, receptive fields, density estimation Supervised learning (ch 7) –Input-output mapping, feedback as teacher signal Reinforcement learning (ch 9) –Feedback in terms of reward, similar to control theory Hebbian learning (ch 8) –Biologically plausible + normalization –Covariance rule for (un)supervised learning –Occular dominance, maps

Rate model with fast time scale Neural activity as continuous rate, not spike train V is output neuron, u is vector input neurons, w is vector of weights If tau_r small wrt learning time:

Basic Hebb rule V and u are functions of time. Makes dynamics hard to solve. Alternative is to assume v,u from distribution p(v,u) and assume p time independent. Using v=w. u we get

Basis Hebb rule Hebb rule is unstable, because norm always increases Continuous differential equation can be simulated using Euler scheme

Covariance rule Basic Hebb rule describes only LTP since u, v positive LTD occurs when pre-synaptic activity co-occurs with low post synaptic activity Alternatively,

Covariance rule When Either rule produces Covariance rule is unstable

BCM rule Bienenstock, Munro, Cooper (1982): Requires both pre and post synaptic activity for learning For fixed threshold, the BMC rule is also unstable

BMC rule BMC rule can be made stable by threshold dynamics Tau_theta is smaller than tau_w BMC rule implements competition between synapses –Strenghtening one synapse, increases the threshold, makes strengthening of other synapses more difficult Such competition can also be implemented by normalization

Synaptic normalization Limit the sum of weights or sum of squared weights Impose this constraint rigidly, or dynamically Two examples: –Rigid scheme for sum of weights constraint (subtractive norm.) –Dynamic scheme for sum of squared weights (multipl. Norm.)

Subtractive normalization Subtractive normalization ensures that sum w does not change Not clear how to implement this rule biophysically (non- locality). We must add a constraint that weights are non-negative.

Multiplicative normalization Oja rule (1982) The rule implements the constraint dynamically:

Unsupervised learning Adapting the network for a set of tasks –Neural selectivity, receptive field –Cortical map Process depends partly on neural activity and partly not (axon growth) Ocular dominance –Adult neurons favor one eye over the other (layer 4 input from LGN) –Neurons are clustered in bands or stripes

Single post-synaptic neuron We analyze Eq. 8.5

Single post-synaptic neuron Solution in terms of eigenvalues of Q Eigen values are positive, so solution explodes. Asymptotically –e1 is the principle eigen direction –Neuron projects input onto this direction:

Single post-synaptic neuron Example with two weights. Weights grow indefinite, one positive one negative. Choice depends on initial conditions. Limit to [0, 1] yields different solutions depending in init value

Single post-synaptic neuron Subtractive normalization. Averaging over inputs: Analysis in terms of eigenvectors: –In ocular dominance e1=n/sqrt(n). W in direction of e1 has rhs equal to zero. Ie this component of w is unaltered –In other directions normalizing term is zero –W asymptotically dominated by second eigenvector

Hebbian development of ocular dominance Subtractive normalization may solve this, since e1=n weight grows proportional to e2=(1,-1)

Single post-synaptic neuron Using the Oja rule Show that each eigenvector of Q is solution. One can show that only principal eigenvector is stable.

Single post-synaptic neuron A: Behavior of –Unnormalized Hebbian learning –Multiplicative normalization (Oja rule) gives w propto e1. This is similar to PCA. B: Shifting mean of u may yield different solution C: Covariance based learning corrects for mean Saturation constraints may alter this conclusion

Hebbian development of ocular dominance Model layer 4 cell with input from two LGN cells, each associated with different eye.

Hebbian development of orientation selectivity Cortical receptive fields from LGN. ON-center (white) and OFF-center (black) cells excite cortical neuron. Spectral analysis also applicable to non-linear systems Dominant eigenvector uniform. Non-uniform receptive fields result from sub-dominant eigenvector

Multiple postsynaptic neurons

Hebbian development of ocular dominance stripes A: model with right and left eye inputs drive array of cortical neurons B: ocular dominance maps. Top: light and dark areas in top and bottom cortical layer show ocular dominance in cat primary cortex. Bottom: model of 512 neurons with Hebbian learning

Hebbian development of ocular dominance stripes Use 8.31 with W=(w+,w-) the n*2 matrix. See book. Subtractive normalization dw+/dt=0 Ocular dominance pattern given by largest eig. vector of K

Hebbian development of ocular dominance stripes Suppose K translation invariant –Periodic boundary conditions simulating a patch of cortex ignoring boundary effects –Eigenvectors are –Eigenvalues are Fourrier components –Solution of learning is spatially periodic (viz. fig 8.7)

Competitive Hebb rule A more abstract model to model non-linear effects –Large range inhibition yields soft-max –Soft-max is locally averaged Can also produce specialization without subtractive normalization

Competitive Hebb rule One dimensional ocular dominance map for entire visual field (rather than one input as before) –a labels cortex, b labels retina, periodic boundaries (torus) –Retinotopic receptive fields develop –A: weights after learning. B: –C:

Feature based models Multi dimensional input (retinal location, ocular dominance, orientation preference,....) Replace input neurons by input features, W_ab is selectivity of neuron a to feature b –Feature u1 is location on retina in coordinates –Feature u2 is ocularity (how much is the stimulus prefering left over right eye), a single number The couplingto neuron a describe the preferred stimulus Activity of output a is

Feature based models Output is soft-max Combined with lateral averaging –Self-organizing map (SOM) –Elastic net

Feature based models optical imaging shows ocularity and orientation selectivity in macaque primary visual cortex. Dark lines are ocular dominance boundaries, light lines are iso-orientation contours. Pin wheel singularities, linear zones

Feature based models Elastic net output SOM, competitive Hebbian rules can produce similar output

Anti-hebbian modification Another way to make different outputs specialize is by adaptive anti-Hebbian modification Consider Oja rule: Each output a will be identical Anti-Hebbian modification is shown at synapses from parallel fibers to Purkinje cells in cerebellum. Combination yields different eigenvectors as outputs

Timing based rules Left: in vitro cortical slice. Right: in vivo xenopus tadpoles LTP when pre-synaptic spike precedes post-synaptic spike LTD when pre-synaptic spike follows post-synaptic spike

Timing based rules Simulating spike-time plasticity requires spiking neurons Approximate description with firing rates H(t) positive/negative for t positive/negative

Timing based plasticity and prediction Consider array of neurons labeled by a with receptive fields f_a(s) (dashed and solid curves) Timing based learning rule. Stimulus s moves from left to right.

Timing based plasticity and prediction If a left of b, then link a to b is strengthened and link b to a is weakened. Receptive field of neuron a is asymmetrically deformed (A solid bold line) Prediction: next presentation of s(t) will activate a earlier. In agreement with shift of place field mean when rats run around track (B).

Supervised Hebbian learning Weight decay: Asymptotic solution is

Classification and the Perceptron If output values are +/- 1 the model implements a classifier, called the Perceptron: –The weight vector defines a separating hyper plane: = gamma. –The perceptron can solve problems that are ‘linearly separable’