Basic Processes in Visual Perception

What is perception good for? We often receive incomplete information through our senses. Information can be highly ambiguous Perceptual system must resolve ambiguities by drawing inferences from a large set of perceptual cues and conceptual knowledge of the world

Mapping of Visual Fields Left visual field  right visual cortex Right visual field  left visual cortex

The Retina-geniculate-striate System The parvocellular (or P) pathway Sensitive to color and to fine detail Most of its input comes from cones The magnocellular (or M) pathway Most sensitive to information about movement Most of its input comes from rods

A very simplified illustration of the pathways and brain areas involved in vision. There is much more interconnectivity within the brain (VI onwards) than is shown, and there are additional (not shown) brain areas involved in vision. Some evidence suggests that developmental dyslexia is associated with damage to the M pathway. Impaired visual motion perception correlates with reading speed and dyslexia. E.g., Demb, J.B., Boynton, G.M., Heeger, D.J. (1999)

Are there behavioral consequences for individual differences in brain anatomy? Slide from: Geoff Boyton at Salk institute Note the cortical area devoted to each area

Primary and Secondary Visual Cortex (V1 and V2) Retinotopic maps Receptive fields: On-off cells; Off-on cells Simple cells Lateral inhibition

Retinotopic maps in V1 Response in monkey primary visual cortex (V1) measured by radio-active tracers Stimulus pattern Retinotopic mapping: locations on retina are mapped to cortex in orderly fashion. Note: more of visual cortex is dedicated to foveal vision Tootell, R. B., M. S. Silverman, et al. Science (1982)

Stimulus Cortical Mapping: Left Hemisphere Cortical Mapping: Right Hemisphere

Revealing retinotopic maps with fMRI FROM GEOFF BOYNTON’S WEBSITE: “This movie illustrates the retinotopic mapping techniques used by most vision fMRI labs. The idea for the phase-encoded stimulus comes from Steve Engel (pdf), and was expanded upon by Marty Sereno and Anders Dale. I first saw a movie like this in a talk by Roger Tootell back in 1997. This is a movie of the fMRI response to a subject (Geoff Boynton) viewing a slowly expanding annulus containing a counterphase modulated (8Hz) flickering checkerboard. Each gray region represents a computationally flattened representation of the subject's left and right visual cortices, each with a radious of about 60mm. Visual areas are overlayed in color as V1 (red), v2 (green) and V3 (blue). The actual stimulus during scanning took 40 seconds to complete a full cycle, and six cycles were repeteated. The movie was made by calculating the amplitude and phase of the 6-cycle component of the Fourier transform of the temporal signal. The 'Active' region is the area of cortex that modulates at a particular temporal phase as phase is cycled through 360 degrees to make the movie. Note how the the activity spreads in a systematic manner as the stimulus moves from the fovea to the periphery.” From: Geoff Boynton, SALK institute

Revealing retinotopic maps with fMRI FROM GEOFF BOYNTON’S WEBSITE: “ This movie is the response to a wedge stimulus subtending 30 degrees of arc and containing flickering checkerboards. The wedge swept through 360 degrees every 40 seconds. Note how activity sweeps across V1 as the wedge sweeps from vertical to vertical meridian in the contralateral hemifield. Note also how V2 is divided into four regions, each representing 90 degrees of the visual field. The sweep of activity in V2 is in the opposite direction as V1. V3's organization is reversed again so that the activity sweeps in the same direction as V1.” From: Geoff Boynton, SALK institute

Measuring Neural Activity

Receptive Fields The receptive field (RF) of a neuron is the area of retina cells that trigger activity of that neuron On-off cells and off-on cells:

opposite response pattern On-off cell STIMULUS RESPONSE APPROX. FIRING RATE 4 25 5 Video LGN On cell: responses as shown on left LGN Off cell opposite response pattern

Simple Cells (bar detectors) Video:

A wiring diagram for building simple cells out of on-off cells Hierarchical organization of the brain: by aggregating responses over several on-off cells, the brain can detect more complicated features (e.g. bars and edges)

Hierarchical Organization

On-Off Cells with lateral inhibition: Lateral inhibition sets up competition between neurons so that if one neuron becomes adept at responding to a pattern, it inhibits other neurons from doing so. Light: ++ ++ - ++ ++ - ++ ++ - ++ ++ - ++ ++ - On-Off Cells with lateral inhibition: Response  Edge detection DEMO APPLETS: 1) http://serendip.brynmawr.edu/%7Ebbutoi/latinh.html 2) http://www.psychology.mcmaster.ca/4i03/demos/lateral-demo.html

Functional Specialization Theory (Zeki) Spatially different areas are functionally specialized for processing visual attributes such as shape, color, orientation, and direction of motion Examples: V1 and V2 Early stage of visual perception V3 and V3A Form, especially the shapes of objects in motion V4 Responsive to colour V5 Visual motion

Evidence for Functional Specialization Single-cell recording Patient data: Achromatopsia (damage to V4) Akinetopsia (damage to V5 or MT)

Specialization for form processing in IT (Inferotemporal-Cortex) Kobatake & Tanaka, 1994

There is some evidence for specialization to face processing Bruce, Desimone & Gross (1981)

The percentage of cells in six different visual cortical areas responding selectively to orientation, direction of motion, disparity, and colour. This figure shows specialization for some areas, especially MT (V5) but not others. For example, V1 participates in a number of different perceptual tasks.

Sensory Binding Problem If spatially different areas are functionally specialized for processing visual attributes such as shape, color, orientation, and direction of motion…. then how does the brain then “bind” together the sensory attributes of an object to construct a unified perception of the object? Binding Problem

Binding Problem

Alternative View: Hierarchical Model Lennie (1998): Visual processing is hierarchical Areas serve multiple functions (except for MT)

Hierarchical Organization

“What and Where” or “What and How” Systems Mishkin and Ungerleider (1982) Object perception (what is it?) Ventral pathway running from the primary visual area in the cortex to the inferior temporal cortex Spatial perception (where is it?) There is a dorsal pathway running from the primary visual area in the cortex to the posterior parietal cortex

Perception–Action Model Milner and Goodale (1995, 1998) Vision for perception Based on the ventral pathway Long-lasting, viewpoint-independent representations Vision for action Based on the dorsal pathway Short lasting, viewpoint-dependent representations

Evidence Double dissociation: some patients would show reasonably intact vision for perception but severely impaired vision for action, and others would show the opposite pattern Optic ataxia Visual agnosia

Differential Sensitivity to Visual Illusions Performance on a 3-D version of the Müller-Lyer illusion as a function of task (grasping vs. matching) and type of stimulus (ingoing fins vs. outgoing fins). Haart et al. (1999).

Appropriate grasping requires the retrieval of object knowledge from long-term memory Mean percentages of objects grasped appropriately in the control (grasping only), spatial imagery, and paired associate learning conditions. Creem and Proffitt (2001b).