Perceptual systems: Central visual pathways Kandell et. al., Princ. of Neural Sci.

Outline Subcortical pathways (retina, LGN) Primary visual cortex (V1) Magno- & Parvo- streams Primary visual cortex (V1) Partial integration of M and P Gateway to Ventral/Dorsal Ventral pathway (V4, IT) Object processing Dorsal pathway (MT, IP) Motion processing Summary

Retinal ganglion cells (RGCs): Quick Summary Output of the retina (“optic nerve”)

Retinal ganglion cells (RGCs): Quick Summary Output of the retina (“optic nerve”) Center-surround receptive fields Kandell et. al., Princ. of Neural Sci.

RGCs: Two principal cell-types Midget RGC receptive field tiling Parasol RGCs receptive field tiling Send axons to LGN to form Parvocellular (‘P’) pathway Send axons to LGN to form Magnocellular (‘M’) pathway

RGCs: Two principal cell-types Midget RGC receptive field tiling Parasol RGCs receptive field tiling Send axons to LGN to form Parvocellular (‘P’) pathway Send axons to LGN to form Magnocellular (‘M’) pathway

RGCs: Two principal cell-types Midget RGC receptive field tiling Parasol RGCs receptive field tiling P pathway characteristics: -Smaller RFs, so better spatial resolution -Selective for color (red-green opponency) M pathway characteristics: -Better temporal resolution -More sensitive to low light levels -Monochromatic (black-white opponency)

Lateral geniculate nucleus (LGN) Thalamus is the meeting point of sensory organs LGN is the visual portion: Receives direct input from RGC cells

Lateral geniculate nucleus (LGN) LGN receives direct input from RGC cells Receptive fields have very similar RFs to their RGC inputs: Circular Center-surround Reflected in anatomy… Kandell et. al., Princ. of Neural Sci.

Optic chiasm RGCs in left hemifield send axons to right LGN, and vice-versa Kandell et. al., Princ. of Neural Sci.

LGN lamina: 3 layers for each eye -Contralateral = the eye at the opposite side of brain -Ipsilateral = the eye at the same side of the brain -Contra and ipsi inputs are segregated by LGN layers Parvocellular (P channel) Magnocellular (M channel) Kandell et. al., Princ. of Neural Sci.

LGN lamina: 4 Parvo & 2 Magno layers Parvocellular (P-channel) -Similar RFs to Midget inputs Magnocellular (M-channel) -Similar RFs to Parasol inputs Inputs from RGC Midget cells: -Smaller RFs -Selective for Color Inputs from RGC Parasol cells: -Larger RFs -Monochromatic -Respond to faster changes Kandell et. al., Princ. of Neural Sci.

LGN lamina: 4 Parvo & 2 Magno layers All combinations transmitted to cortex: M-Contra, M-Ipsi, P-Contra, P-Ipsi Kandell et. al., Princ. of Neural Sci.

Subcortical pathways summary: Unique cell-types in the retina form the first stages of 2 major parallel pathways: Magno- and Parvo- M and P pathways carry unique spatio-temporal-chromatic information LGN RFs are very similar to those in the retina (one reliable synapse from RGC) 6 layers in the LGN to maintain segregation of eye input and M/P input. Subcortical pathways are trying to take basic features about the visual scene and reliably transmit this information to cortex.

Outline Subcortical pathways (retina, LGN) Primary visual cortex (V1) Magno- & Parvo- streams Primary visual cortex (V1) Partial integration of M and P Gateway to Ventral/Dorsal Ventral pathway (V4, IT) Object processing Dorsal pathway (MT, IP) Motion processing Summary Kandell et. al., Princ. of Neural Sci.

Laminar architecture of cortex The primate cortex is ~2mm thick 6 layers within this thickness

Laminar architecture of cortex Inputs to layer IV Outputs from layers II/III The primate cortex is ~2mm thick 6 layers within this thickness Feedforward circuit: Layer IV, then layer II/III, then next cortical area

Laminar architecture of V1 Inputs to layer IV Outputs from layers II/III V1 Kandell et. al., Princ. of Neural Sci.

Laminar architecture of V1 Spatial-temporal-chromatic information of visual world is still segregated anatomically within the input layers of cortex: i.e. Parallel pathways are still segregated Kandell et. al., Princ. of Neural Sci.

Laminar architecture of V1 Finally, this information is partially combined once it hits the output layers of V1. Spatial-temporal-chromatic information of visual world is still segregated anatomically within the input layers of cortex: i.e. Parallel pathways are still segregated Now back to Kandell et. al., Princ. of Neural Sci.

V1 ocular dominance Eye inputs are also segregated at the input layer of V1 A neuron’s response preference for one eye over another is called “ocular dominance” In V1, there are alternating bands of ocular dominance across the cortical surface… “map” 1mm LGN V1 Kandell et. al., Princ. of Neural Sci.

Primary visual cortex (V1): Orientation tuning arises LGN inputs are not tuned for edges V1 neurons, one synapse from LGN, have tuning for edges of a particular orientation: “orientation tuning”

V1 orientation tuning Stimulus Response

V1 orientation tuning Classic Hubel & Weisel model: Orientation tuning arises from an alignment of LGN inputs LGN V1 Kandell et. al., Princ. of Neural Sci.

V1 orientation maps Similar tuning within column Columnar organization: similar response properties at all depths of a vertical penetration

V1 orientation maps Tuning progresses smoothly across horizontal penetration

V1 orientation maps 1mm Kandell et. al., Princ. of Neural Sci.

V1 orientation maps Continuous maps of orientation preference: “pinwheels” Consistent preference though the depth of cortex: columnar architecture Kandell et. al., Princ. of Neural Sci.

V1 retinotopic maps (V1) Each point of the visual field maps on to a local group of neurons in V1. Retinotopy = Remapping of retinal image onto cortical surface Foveal region uses more of V1 (greater magnification factor)

Classic V1 maps: summary Ocular dominance map Orientation map Retinotopic map 1mm Why have maps? How are they created? Kandell et. al., Princ. of Neural Sci.

Classic V1 maps: summary Ocular dominance map Orientation map Retinotopic map 1mm Why have maps? How are they created? Nobody knows for certain Kandell et. al., Princ. of Neural Sci.

Primary visual cortex (V1): Summary Pathways segregated in LGN are still segregated at input layers of V1 M & P separated by sublayers in IVC Eye inputs segregated into columns Tuning for edges arises at first synapse: orientation selectivity. Maps: orientation, ocular dominance, retinotopy. Preview to higher areas V1 is the “trunk” of the visual cortex hierarchy: V1 output forms dorsal and ventral streams. M-to-Dorsal (mostly) P-to-Ventral (mostly)

Outline Subcortical pathways (retina, LGN) Primary visual cortex (V1) Magno- & Parvo- streams Primary visual cortex (V1) Partial integration of M and P Gateway to Ventral/Dorsal Ventral pathway (V4, IT) Object processing Dorsal pathway (MT, IP) Motion processing Summary Kandell et. al., Princ. of Neural Sci.

Ventral pathway: Preview Two principal paths: V1-V2-V4-TEO-IT V1-V4-TEO-IT Involved in object identification. Called the “what” pathway. Late stages of inferior temporal (IT) cortex contain the celebrated “face cells” Kandell et. al., Princ. of Neural Sci.

Ventral pathway: Receptive field hierarchy Subcortical pathways compute local differences V1 identifies edges V1 projects to V2 & V4: What might V4 neurons represent? ?

Ventral pathway: Receptive field hierarchy Subcortical pathways compute local differences V1 identifies edges V1 projects to V2 & V4: What might V4 neurons represent? Curvature, contour integration, foreground/background.

IT represents complex objects, such as faces Ventral pathway IT represents complex objects, such as faces Desimone et. al. 1984

Ventral pathway: Convergence of inputs at each stage We don’t know how this selectivity is built from the circuitry, but we have very general (i.e. crude) ideas. Each region receives convergent input from the previous region (e.g. V1-to-V4), thus progressively increasing the size and complexity of receptive fields V4 V1

Ventral pathway: Convergence of inputs at each stage

Ventral pathway: Convergence of inputs at each stage What happens when we scale or rotate the image?

Ventral pathway: Convergence of inputs at each stage Simple models predict that the IT neurons will no longer respond, but instead IT neurons demonstrate scale, shift, and rotational invariance. i.e. IT responds to particular objects regardless of simple transformations.

Ventral pathway: Convergence of inputs at each stage -Lots of models attempting to explain object recognition in the ventral pathway, particular invariance (many useful engineering applications): -None of them come close to the performance of the primate visual system. 200 ms to IT spikes; Also human response time ?

Ventral Pathway: Summary Receives input from both M and P subcortical pathways, but mostly P inputs. Progressive increase in the complexity of spatial selectivity. Neurons are ultimately responsive to holistic objects in IT. IT neurons also exhibit well-known scale, shift, rotation invariance. How does this happen? Kandell et. al., Princ. of Neural Sci.

Outline Subcortical pathways (retina, LGN) Primary visual cortex (V1) Magno- & Parvo- streams Primary visual cortex (V1) Partial integration of M and P Gateway to Ventral/Dorsal Ventral pathway (V4, IT) Object processing Dorsal pathway (MT, IP) Motion processing Summary I am going to go through each step of these processing streams: from the retina to the higher visual areas, and give you an overview of the anatomy of the processing stage, along with the functional properties of the cells at that stage. Kandell et. al., Princ. of Neural Sci.

Dorsal pathway (preview) The “where” pathway Main input from M-pathway circuits within V1. V1-to-MT-to-IP MT (aka V5) neurons are highly selective to direction of motion. IP neurons tell the motor system how to respond to the environment. Kandell et. al., Princ. of Neural Sci.

Medial Temporal Area (MT): Motion Selectivity Motion selectivity seen at 2 stages of dorsal hierarchy: 1) Neurons in layer IVB of V1 2) Neurons in MT (IVB of V1 is major MT input) So what is MT doing?? IVB of V1 MT Both stages are direction selective

Medial Temporal Area (MT) computes global motion You could easily identify the motion of this object. Born & Bradley., Annu Rev Neurosci

Medial Temporal Area (MT) computes global motion Apparent motion Actual motion -Now, it would appear that movement is in the direction perpendicular to the unmasked edge. -This is what a V1 neuron “sees”. V1 neurons only detect local motion. Born & Bradley., Annu Rev Neurosci

Medial Temporal Area (MT) computes global motion Apparent motion Actual motion -MT neurons have much larger receptive fields than V1, so they can integrate multiple cues in the image. MT neurons can detect global motion of the object.

Parietal cortex: A ridiculously brief summary MT passes its output to the intraparietal cortical areas. Each IP area is responsible for a different “class” of movement. In addition to unparalleled object recognition (ventral pathway), primates have great hand-eye coordination so that we can use tools, or hit a baseball (dorsal pathway). IP uses visual input to tell the motor system how to interact with the environment.

Differences between dorsal and ventral pathway, summarized by lesion examples Patients with lesions of the parietal cortex cannot properly adjust their grip (width or angle) to grab an object. But they can describe the shape of the object in detail. Patients with lesions of the ventral path cannot describe the object but can easily adjust their grip to grab it.

Summary Kandell et. al., Princ. of Neural Sci.

Things I ignored (to name a few) Many aspects of the circuitry Local cortical circuits Intercortical feedback Contextual influences Attention

Much we still do not know We do not have a descriptive or mechanistic model that predicts response properties of downstream visual areas, or behavior. A descriptive model would vastly transform technology: the primate visual system is far superior to anything that engineers can build. A mechanistic model is the ultimate goal (a more difficult one) as this would allow us to address neurological disorders systematically. I’m not going to go into specifics, but its easy really for a naïve observer of the data on early visual cortex to say, we already know everything about the visual system. But if we did, this world we be a much different place…

Thats all!

Ventral pathway: Convergence of inputs at each stage IT