VS131 Visual Neuroscience Extra-Striate Cortex Hey stupid: remember to distribute the handouts this time!

Grossly simplified block-diagram of the cortical visual system

Another grossly simplified block- diagram of the cortical visual system

Even more grossly simplified block- diagram of the cortical visual system! You will never see the cortical visual system drawn the same way twice. You have been warned!

Nota Bene: Almost all of our current knowledge of extrastriate visual cortex comes from work in the rhesus monkey. All current textbooks refer to monkey data. Recently, work using functional Magnetic Resonance Imaging (fMRI), which unlike other research techniques can be applied in both humans and primates, has started to tie the monkey data to the human data. It appears that, indeed, much of the basic organization of the cortical visual system in monkeys is the same as that of humans. Work is ongoing and as usual there will doubtless be differences between the species, especially in ‘higher’ (farther away from V1) visual cortical areas. Often you may read of the “human homolog of V4” instead of just V4.

Rhesus monkey cortical visual areas unfolded

Human visual cortical areas have about the same layout as the rhesus monkey, but are only about 20% of cortical surface

A neurologists’ view of cerebral cortex (does not cover specific areas): Occipital: VISION Parietal: VISION + integrating sensory information different parts of the body and motor control + stuff Temporal: VISION + hearing + speech/language + memory + stuff Frontal: Motor cortex, “executive function” (planning), + stuff + vision esp. VOLUNTARY SACCADIC EYE MOVEMENTS.

Very crudely: DORSAL PATHWAY: “WHERE” stream of visual processing: magno. VENTRAL PATHWAY: “WHAT” stream of visual processing: parvo.

V2 -> Closely associated with V1 -> BOTH magno and parvo sections -> Neurons in V2 have, at first glance, roughly similar properties to those of neurons in V1. Small receptive fields, often selective for orientation of a visual stimulus.

(Macaque)

Cytochrome Oxidase stain V1 blobs -> V2 this stripes V1 ‘interblobs’ (I.e., not blobs) -> V2 inter- stripes V1 layer 4B -> V2 thick stripes

Another view – note ‘illusory’ border from offset lines in D.

V4 -> One of the larger and more important of the post-V1/V2 visual cortical areas -> More of a ‘parvo’ sort of pathway. ‘Ventral Stream’ -> More complex responsivities to shapes than just oriented lines (real or imagined). Larger RF size. -> Complex color properties (used to be thought of as the color center, now know that it does more than just color) -> Attentional effects become easier to elicit -> Lesions cause more subtle problems than just visual field defects.

V4 has larger RF’s – here we can fit two stimuli inside. This is from experimental data from a monkey recording from a single V4 neuron. If the monkey is attending to a location where an effective stimulus (I.e., one that elicits a strong response) is present, you get a strong response (left). However, if the monkey is not paying attention to that location, you get a much weaker response, even though the optical stimulus is identical (right).

As before but now the monkey is only paying or not paying attention to the entire receptive field. There are lots of different variations of this basic paradigm. Very much harder (though not impossible) to see such effects in V1/V2.

V4 neurons often have more complex shape selectivities

V4 neurons often have more complex color properties Our perception of color is NOT a simple function of the amount of red, green, and blue light. (Ever take a picture using daylight film and incandescent light?). A V4 neuron responding to a red square in the middle of a lot of different color patches (‘Mondrian’). It responds strongest when the rest of the scene is lit by white light (RGB), weakest when everything is lit by red (R) thus making color distinctions impossible.

In experimental animals lesioning V4 does not result in specific visual field defects, but in more generalized problems of shape perception. For example, an animal with a V4 lesion may be able to learn to recognize an object in one orientation, but be unable to recognize it if it is viewed at a different angle or distance. Also, lesions of V4 have been associated with problems in color vision.

IT -> Inferior temporal cortex (IT, TE, TEO, etc.) -> Last of the hierarchy of (more-or-less) purely visual areas in the ventral (‘parvo’, ‘what’) stream. -> Neurons can have very large receptive fields… -> …but at the same time, the specificity for visual stimuli can be very high -> Lesions of IT can have devastating consequences for the ability to recognize specific objects (e.g. faces) with no corresponding loss of acuity or visual field deficits. -> Large attentional effects -> Lesions of temporal cortex can cause visual field deficits by interrupting the passing fibers of Myer’s loops.

From Van Essen and Colleagues

RF sizes get larger further along in the looser hierarchy of visual areas… (And by IT they often cross the midline)

IT neurons in the same cortical column tend to have similar (not identical) properties

Remember the columns in V1? It has been speculated that there is a similar modular organization in IT, but if so, we do not know what the analogs of location in space, orientation, or ocular dominance are.

“Grandmother Cell” Hypothesis IT cells can be so specific in their response properties that it seems like a neuron might only fire when one specific object (like someone’s grandmother) comes into view. Thus, if the neuron that is ‘tuned’ to your grandmother fires, that is a signal to the rest of the brain that grandma is there. However, while IT neurons have very specific response properties, they still do respond to more than just one object. Current thinking is that the presence of a particular object (face) is coded for by a pattern of of activity of a relatively small (but still greater than one) number of IT cells. Thus, at least for now the ‘grandmother cell hypothesis’ is no longer generally accepted.

MT/V5 -> Middle Temporal cortex (MT, but only really middle temporal in owl monkeys!) related areas are Mst, and various parietal something-or-other). -> In the hierarchy of (more-or-less) purely visual areas in the dorsal (‘magno’, ‘where’) stream. -> Specialized for motion processing -> MT is not large but it appears to be important and it has been studied a lot, so it gets a lot of coverage in textbooks. -> Lots of heavily myelinated axons, fast conduction velocities (a magno trait). Stains heavily for myelin compared to other visual cortical areas. -> Cells in MT are really easy to activate with moving stimuli.

Analysis of Motion As usual, what seems easy is really incredibly hard. Figuring out what is going where in a visual scene is a very hard computational task – one that we are not even close to solving for computer vision. For starters, we sense motion when our eyes are fixed and an image moves across the retina, and equally when the eyes track a moving object and the image on the retina is fixed.

The visual system can also infer motion from isolated sightings of an object in different locations at different times: Apparent Motion.

The visual system can also solve the Aperture Problem.

Unlike neurons in V1, some (not all!) neurons in MT appear to solve the aperture problem: they respond to the integrated motion of the entire object, not just to the motion of its separate parts.

MST: Medial Superior Temporal area -> Gets most of its inputs from MT -> Neurons are tuned to flow fields.

Patterns of optic flow

The parietal lobe is in many ways the least well understood of the cortical lobes. Contains primary somatosensory cortex, and a lot of regions dealing with vision. Important for hand-eye coordination and regulating attention, possibly creating a ‘sense of space’. Lesions can cause hemi-neglect (“eyes right). Some areas found in monkey posterior parietal cortex: LIP: Lateral Intraparietal Cortex. Contains a retinotopic map of salience. Neurons respond well to recent stimuli, or to stimuli that are behaviorally relevant. Predictive remapping across saccades! VIP: Ventral Intraparietal Cortex: A multisensory map of space around the mouth and face, important for ingestive behaviors?

Other cues to relative size and motion have to be evaluated by the visual system, and integrated with all the other sources of information.

Binocular Disparity: Judging distance by triangulating between the two eyes. First neurons sensitive to binocular disparity are found in area V1. Disparity sensitive neurons are found throughout visual cortex, but are most easily and commonly found in the dorsal/parietal pathway. Near disparity = crossed Far disparity = uncrossed

Single-Image Random-Dot Stereogram! Autostereogram Random-Dot Autosterogram “Magic Eye”

So how do they work? Standard stereograms use two images, one for each eye. Various optical systems ensure that the left eye gets one image, and the right eye gets another image. For autostereograms, the bottom line is that there are two hidden pictures in a single image seen by both eyes, the two hidden images have different amounts of horizontal shift in different regions of the image that are interpreted by the brain as different depths. Using random dots hides the two images well, because as with standard random dot stereograms, you can’t see the images unless they have been binocularly fused.

Actually looking at two objects, one nearer and one farther than the fixation point. The brain interprets the shifts in the image of the objects as evidence of them being nearer or farther than fixation.

Standard stereogram, each eye gets a separate image. As before, the brain interprets the shifts in the image of the objects between the two retinas as evidence of them being nearer or farther than fixation.

Single image stereogram, two shifted copies of each object, the brain again interprets the shifts in the image of the objects between the two retinas as evidence of them being nearer or farther than fixation. BUT: How do you hide the two images?

Very crudely: use random-dot stereograms. You can’t see the objects unless they are binocularly fused, thus hiding the two pictures. At each place in the single image, you have two overlaid (transparent or alternating) random dot stereograms with different horizontal shifts. The brain uses the shift that produces the best match to estiamte distance. Why does it take time to see these images? Because in normal conditions there are false binocular matches everywhere, and the visual system ignores them unless there is confirmatory data. Initially all the little random dot stereogram patches are filtered out. Once a few manage to break through at once and give the sense of a colinera contour, this reinforces the depth sense from the random dot stereograms and the entire picture snaps into focus.

Tricks for viewing: Most autostereograms are designed for wall-eyed (diverging) viewing. Need to focus (converge) on a point behind the image plane. People with stereo vision deficits can’t fuse random-dot stereograms, including magic eye (between 1 and 5 percent of population, give or take). -> Use bright lighting: causes pupil to constrict, decreases conflicting information due to blur and depth-of-focus. -> Try to look behind the image. Look at your reflection if on glossy paper (which optically will appear twice as far away). Hold up to your nose and slowly pull away. Look at an object behind the image plane (opposite if autostereogram designed for crossed viewing).