1. Cortical circuits for vision Jamie Mazer Neurobiology of Cortical Systems Lecture 7 March 12, 2012

2. Readings for Thursday

3. How much of cortex is visual? (in primates) Van Essen flat map of macaque cortex Primates are likely an extreme example or an upper bound..

4. How much of cortex is visual? Van Essen flat map of macaque cortex simplified Felleman & Van Essen hierarchy

5. Key concepts phenomenon vs. implementation vs. function centrally synthesized maps –everything we perceive must be encoded by the retina –if so, whats all that visual cortex doing? –generating explicit sensory representations –emergent properties seem to be a key feature of high-level sensory cortical function –Question: Is cortex required to generate explicit or abstract properties? –Answer: Whats emergent in the retina? What about animals with not cortex, like birds and fish? are there common motifs across sensory modalities? – computational maps in other modalites? – what about other species? are they unique to cortex?

6. Retinal bipolar cells receptive fields

7. Retinal ganglion cell RFs (only retinal output)

8. Receptive fields and center-surround opponency

9. Center-surround organization Observed phenomenon? Implementation? Function?

10. Receptive fields and center-surround opponency Center-surround organization Observed phenomenon? Characteristic RF structure Implementation? Lateral inhibition Function? Spatial derivative; contrast enhancement

11. Behavioral consequences of center surround organization herring grid mach bands

12. Behavioral consequences of center surround organization herring grid mach bands

13. Thalamus: dLGN

14. What changes between the photoreceptors and LGN? transition from receptor potentials to spiking center-surround spatial receptive fields color opponency (B-Y/R-G) instead of simple cone- based wavelength tuning segregation into parallel processing streams –sustained and transient –fast and slow –on and off channels –color and luminance

15. Which brings us to primary visual cortex (BA 17; V1) m visual association primary visual

16. Topographic organization of V1 - retinotopy - orientation columns - occular dominance columns - non-oriented blobs (L2) - orientation topography

17. Thalamocortical projections and the canonical microcircuit

18. Primary visual cortex: simple cell orientation tuning hubel & wiesel 1968 orientation tuned V1 neuron MOVIE

19. Primary visual cortex: simple cell orientation tuning hubel & wiesel 1968 orientation tuned V1 neuron hubel & wiesel model

20. Primary visual cortex: simple cell orientation tuning hubel & wiesel 1968 orientation tuned V1 neuron hubel & wiesel model Key failures for the feedforward model? - contrast invariant orientation tuning

21. Primary visual cortex: simple cell orientation tuning hubel & wiesel 1968 orientation tuned V1 neuron hubel & wiesel model Hubel & Wiesel Interpretation Observed phenomenon? preferred orientation Implementation? linear summation of LGN cells Function? feature detectors for edges

22. Primary visual cortex: simple cell orientation tuning hubel & wiesel 1968 orientation tuned V1 neuronhubel & wiesel model Spatial Vision Interpretation Observed phenomenon? preferred orientation Implementation? quasi-linear combination of LGN cells Function? spatiotemporal filtering

23. cells prefer light increments or decrements cells have orientation tuning cells have a width tuning cells have length tuning cells have speed tuning cells are feature detectors where the feature is a bar of a particular orientation, size and speed intuitively obvious, simple to understand, seems to imply obvious behavioral function cells prefer light increments or decrements cells have orientation tuning cells have spatial frequency tuning cells have temporal frequency tuning cells are half-wave rectified spatiotemporal filters (Gabors) requires some math chops to understand, but has predictive power Feature detector model spatial vision model

24. Primary visual cortex: spatial frequency tuning Robson, DeValois, Maffei etc..

25. Feature detector model spatial vision model cells prefer light increments or decrements cells have orientation tuning cells have a width tuning cells have length tuning cells have speed tuning cells are feature detectors where the feature is a bar of a particular orientation, size and speed intuitively obvious, simple to understand, seems to imply obvious behavioral function cells prefer light increments or decrements cells have orientation tuning cells have spatial frequency tuning cells have temporal frequency tuning cells are half-wave rectified spatiotemporal filters requires some math chops to understand, but has predictive power

26. Primary visual cortex: simple complex hubel & wiesel 1968 simple complex

27. Primary visual cortex: simple complex hubel & wiesel 1968 simple complex MOVIE

28. Primary visual cortex: simple complex hubel & wiesel 1968 simple complex hypercomplex +length tuning

29. Primary visual cortex: simple complex hubel & wiesel 1968 simple cells pool center-surround neurons to form orientation selectivity complex cells pool simple cells to become position or phase invariant. and turtles all the way down…

30. Complex cells and the F 1 /F 0 ratio cats monkeys Skottun et al, 1991

31. Whats the spatial vision model got to say?

32. Complex cells and the F 1 /F 0 ratio Skottun et al, 1991 cats monkeys Mechler & Ringach, 2002 is this all an artifact?

33. Reverse correlation and the spike triggered average Jones & Palmer, 1987

34. Reverse correlation and the spike triggered average Jones & Palmer, 1987

35. Reverse correlation and the spike triggered average Jones & Palmer, 1987

36. V1 neurons are Gabors and Gabors are optimal… Daugman, 1985

37. V1 neurons are Gabors and Gabors are optimal… Daugman, 1985

38. Where do Gabors come from and the efficient coding hypothesis Barlow, 1972

39. Where do Gabors come from and the efficient coding hypothesis

40. Vinje & Gallant, 2000

41. Where do Gabors come from and the efficient coding hypothesis Haider et al, 2010

42. What have we established? simple cells – simple cells are partially assembled from LGN afferents – one basic flavor: Gabor they are bar-detectors as well (glass half empty), but the Gabor-model seems like a more compact framework complex cells – complex cells are assembled from simple cells – strict dichotomy not likely, more likely is, thalamocortical direct recipient simple cells, and, cells that are a combination of simple and non-simple innputs coding in V1 – sparseness is a hallmark of an efficient code – simple cells can be learned by maximizing sparseness – sparseness in V1 is based on center-surround (intracortical) inhibitory interations – the neural representation is awful close to what the computer vision people call a wavelet or multiscale pyramid and is the basis for things like MPG and JPG compression… perhaps we need more data from more complex stimuli?

43. Reverse correlation, complex cells and natural scenes

44. Problems: 1.STA doesnt really work for natural (non-white) stimuli 2.the STA is just plain wrong for complex cells

45. Linear receptive field maps in early vision DeAngelis et al, 1995 still orientation tuned! wheres it coming from?

46. Reverse correlation, complex cells and natural scenes Problems: 1.STA doesnt really work for natural (non-white) stimuli 2.the STA is just plain wrong for complex cells

47. Reverse correlation, complex cells and natural scenes Problems: 1.STA doesnt really work for natural (non-white) stimuli 2.the STA is just plain wrong for complex cells Spike Triggered Covariance (STC)

48. What have we established? simple cells – simple cells are partially assembled from LGN afferents – one basic flavor: Gabor they are bar-detectors as well (glass half empty), but the Gabor-model seems like a more compact framework complex cells – complex cells are assembled from simple cells – strict dichotomy not likely, more likely is, thalamocortical direct recipient simple cells, and, cells that are a combination of simple and non-simple innputs coding in V1 – sparseness is a hallmark of an efficient code – simple cells can be learned by maximizing sparseness – sparseness in V1 is based on center-surround (intracortical) inhibitory interations – the neural representation is awful close to what the computer vision people call a wavelet or multiscale pyramid and is the basis for things like MPG and JPG compression… perhaps we need more data from more complex stimuli? – STRFs, STC, regression analysis, MII etc all provide new tools could complex cells and complex stimuli… what did I not talk about??

49. Direction selectivity hubel&wiesel 1968 MOVIE

50. Direction selectivity is Gabor-ish too (vs. Reichardt Detector) DeAngelis et al, 1993. 1995

51. Disparity/depth tuning focal plane near far foveae G. Poggio et al MOVIE

52. Disparity too fits in the spatial vision view… Note: Complex cells see anti-correlated bars differently than correlated, not true for perception… Ohzawa et al, 1997

53. Is spatial vision everything? the high-dimensional Gabor filter model explains a lot of the neurophysiological and psychophysical data, but.. – finding the right dimensions is non-trivial as well see next week. – even when the dimensions are likely identified, its essentially a linear or quasi-linear model and doesnt explain a range of observed phenomena, even in V1…

54. Center-surround interactions in V1 – generally NOT accounted for by the standard spatial vision model. end-stopping, length-tuning, hypercomplexity (H&W) cross-orientation inhibition (Silito et al) divisive gain control (Carandini paper!) curvature processing (Dobbins & Zucker) target pop-out (Knierim & Van Essen) attention and/or figure segmentation (Lamme et al)

55. What are the emergent properties of V1? new features extracted –orientation –binocular disparity (depth) –direction selectivity –spatial frequency –color (really transformed) new maps –orientation (columns) –ocular dominance –segregation of color info in blobs

56. What have we established? simple cells – simple cells are partially assembled from LGN afferents – one basic flavor: Gabor they are bar-detectors as well (glass half empty), but the Gabor-model seems like a more compact framework complex cells – complex cells are assembled from simple cells – strict dichotomy not likely, more likely is, thalamocortical direct recipient simple cells cells that are a combination of simple and non-simple innputs coding in V1 – sparseness is a hallmark of an efficient code – simple cells can be learned by maximizing sparseness – sparseness in V1 is based on center-surround (intracortical) inhibitory interations – the neural representation is awful close to what the computer vision people call a wavelet or multiscale pyramid and is the basis for things like MPG and JPG compression… perhaps we need more data from more complex stimuli? – STRFs, STC, regression analysis, MII etc all provide new tools could complex cells and complex stimuli… V1 exhibits multiple emergent properties What happens when you lose V1?? How much of this interpretation is primate-centric?

57. Primate-centric view? Andermann et al., 2011 Neill & Stryker, 2010 Shuler & Bear, 2006 - Rodents have striate and extrastriate analogues or homologues - Tuning is similar, but not identical - Extra-retinal effects seem more pronounced

58. Readings for Thursday