1. Several strategies for simple cells to learn orientation and direction selectivity Michael Eisele & Kenneth D. Miller Columbia University

2. ON OFF illustration by de Angelis et al. 99 Orientation and Direction Selectivity Orientation Selectivity (OS) Direction Selectivity (DS)

3. space no OSOS orientation- selective? space time space time DSno DS direction- selective? orientation- selective? Lampl et al 01 Priebe & Ferster 05

4. Selected models Simple Hebbian learning rule produces OS (Miller 94), but not DS (Wimbauer et al 97) for unstructured input. Nonlinear Hebbian learning rules produce DS, but only for structured input (Feidler et al 97, Blais et al 00). More general principles (sparse coding, ICA, blind source separation) can explain occurence of OS ( Olshausen & Field 96; Bell & Sejnowski 97 ) and DS ( van Hateren & Ruderman 98 ), if applied to input from natural scenes.

5. Some OS and DS develops early (kittens at time of eye opening; Albus & Wolf 84) awake ferret P27 (before eye opening) Chiu & Weliky 01 Early spontaneous activity Ferret P30-32 correlations decay over a few 100 ms and several mm cortex (Fiser et al 04)

6. Fiser et al 04

7. Find rule that robustly produces DS, using only unstructured input. Identify underlying principle. Goal Blind source separation mixing

8. sourcesunmixed sources mixing unmixing Blind source separation (BSS) sensors

9. sourcessensors random mixing Blind source separation of random, spontaneous activity unmixing more even mixing ?

10. Motivation for blind source mixing (BSM) DS responses to all positions ⇒ no response to some positions no DS responses to all positions Hebbian learning

11. Combining BSM and Hebbian learning

12. Δw = η ⋅ (x ⋅ y + ε ⋅ x ⋅ y 3 ) - λ ⋅ w w = weight Δw = weight-change η = learning rate x = input y = output λ = multiplicative constraint linear Hebbian ε>0: blind source separation ε<0: blind source mixing based on bottom-up approach to blind-source separation; see “Independent Component Analysis” Hyvärinen, Karhunen, Oja 2001 Combined learning rule

13. spatial correlations: Mexican hat distribution of input amplitudes: long tails upper weight limits: none temporal input filters: diverse Important factors 4 week old kittens Cai et al 97

14. single neuron learning rate-coded only feedforward input arbor function linear neuron model Simplifications

15. Whitened input ⇒ BSM can perfectly mix sources. Gradient principle ⇒ convergence A few analytical results

16. preferred orientations of 100 receptive fields other choice of initial weights: Dependence on initial conditions rotation ON ⇔ OFF ε = −0.25

17. ε = 0 (Hebb) ε = −0.15 ε = −0.5 ε = −0.15 ε = −0.2 ε = −0.5 Robustness against parameter changes Δw = η ⋅ (x ⋅ y + ε ⋅ x ⋅ y 3 ) - λ ⋅ w OS and DS develop robustly under BSM + Hebb

18. Limitations special initial conditionsinput = drifting gratings input amplitudes = subgaussian distribution large negative ε: BSM dominates

19. response amplitude number of responses Comparision of response distributions

20. Other strategies: BSS with structured inputBSM with subgaussian input Hebb with hard upper w-limitHebb with soft upper w-limit hybrid with unstructured inputhybrid with structured input hybrid = BSS and Hebb with upper weight limit Any rule that produces OS and DS for structured and unstructured input?

21. Linear Hebbian rule + upper weight limit Miller 94

22. Blind source mixing (BSM) is designed to produce an output that responds evenly to many sources. BSM and and Hebbian learning can be combined to a simple synaptic learning rule. This rule robustly produces OS and DS while the input is unstructured. Conclusions BSS + ➧ BSM + Hebb + OS, DS ➧ known:new:

23. Speculation external worldinternal networkneuron unlearn correlations that are produced internally: BSM learn correlations that are produced externally: BSS Unlearning of higher-order correlations. Compare Crick & Mitchison 83: unlearning of any-order correlations. ➡➡

24. supported by the Swartz Foundation and the Human Frontiers Science Program