Why isn’t vision perfect? An exercise in psychoanatomy

Why isn’t vision perfect? An exercise in psychoanatomy Tracing the flow of information through the nervous system using functional experiments

V1 LGN Parietal (action) temporal (perception) If neural representation fails at any stage, perception will fail

The most basic aspect of vision: spatial resolution Resolution has a limit: Coarse patterns are seen, fine detail is not How and where does resolution fail? We start at the beginning with the photons themselves

Campbell & Robson (1968) Spatial frequency (cycles/degree) Sensitivity Contrast-Sensitivity Function (CSF) Resolution limit: 50cpd

Factors that might limit visual resolution:  Optics of the image (including diffraction)  Sampling by the retinal mosaic  Light collection by the cone apertures  Neural convergence:  intra-retinal or retino-thalamic  thalamo-cortical  intra-cortical

Factors limiting visual resolution:  Optics of the image (including diffraction)

One test for the role of optics: Do perfect optics make vision perfect? Two possible approaches: Test resolution using interference fringe targets, bypassing the optics; or… Use Adaptive Optics to compensate for individual optical aberrations

Dave Williams

Laser interferometrybypassses optical losses: target stripes are generated directly on the retina by intereference of two uniform laser beams

Bypassing optics improves vision, but only from 45 cpd to about 60 cpd. Vision is still not perfect; neural losses are at least as important as optics.

Adaptive Optics

Adaptive Optics: supernormal, yet still imperfect vision

Factors limiting visual resolution:  Optics of the image (including diffraction)…important, but not sufficient

Two ways the retinal mosaic might limit resolution: Filtering Sampling … Effective size of cones … Spacing of cones

Sampling limits on resolution? Foveal photoreceptor mosaic frequency: 110 cpd Nyquist sampling limit for 1 row of cones: 55cpd

30 cpd

60 cpd

120 cpd

120 cpd processed by a single row of cones: aliasing

Roorda and Williams

Sampling limits on resolution? Foveal photoreceptor mosaic frequency: 110 cpd Nyquist sampling limit for 1 row of cones: 55cpd Nyquist limit for 8 rows of cones: 440 cpd ??

120 cpd processed by 8 rows of cones: no serious aliasing

Factors limiting visual resolution:  Optics of the image (including diffraction)... important, but less important than neural losses  Sampling by the photoreceptor mosaic… unimportant  Light collection by the cone apertures ?

Unresolvable high-contrast patterns appear desaturated Sherif Shady and Don MacLeod (Nature Neurosci 2002)

Factors limiting visual resolution:  Optics of the image and photoreceptor sampling…  Light collection by the cone apertures… not severely limiting (>100cpd)  Neural Losses:  intra-retinal or retino-thalamic  thalamo-cortical  intra-cortical

LGN

Peter Lennie… Matt McMahon, …Dave Williams, and Martin Lankheet

McMahon et al. J.Neurosci.1999

Factors limiting visual resolution:  Optics of the image (diffraction, etc.)  Light collection by the cone apertures not severely limiting (>100cpd)  Neural convergence:  intra-retinal or retino-thalamic still not limiting (>80cpd)  thalamo-cortical  intra-cortical ?

LGN V1

Seeing spatial pattern Physical stimuli Perceptual experience Localized neural activity correlation What does V1 see?

Physical stimuli Perceptual experience Localized neural activity V1?? Psychophysics (He and MacLeod,Nature 2001): can cause orientation-selective adaptation?

Orientation-Selective Adaptation Adapt (30 sec) Test (200 msec) Same Orientation Hard to see! Need high contrast Orthogonal Orientation Easy to see! Need low contrast

V1 is the first stage in the visual system where orientation information is extracted: orientation-selective adaptation is only possible at or after V1. If we find evidence for orientation- selective adaptation, then it implies that the orientation information hasat least reached V1. Adaptation: Psychophysicists’ microelectrode

Invisible vertical grating Adapt (30 sec) Test (200 msec) Same Orientation Need higher contrast Orthogonal Orientation Need lower contrast Result

Orientation-selective adaptation from an invisible spatial pattern at 60 to 70 cpd: invisible to us but visible at V1.

Orientation-selective adaptation from an invisible spatial pattern: invisible to us but visible at V1. There is intra-cortical loss of spatial information.

Factors that might limit visual resolution for interference targets:  Optics of the image (including diffraction): not applicable  Light collection by the cone apertures: > 100 cpd  Neural convergence:  intra-retinal or retino-thalamic: 80 cpd  thalamo-cortical…lumped with intracortical…  intra-cortical: 70 cpd  Perception: < = 60 cpd

Conclusions Neural losses slightly exceed optical losses at the limit. Sampling and light collection in the photoreceptor mosaic are not limiting. Neural losses are distributed through the system; some are intracortical, since cortex can respond to patterns too fine for conscious perception. Optional Bonus Conclusion about Consciousness Primary visual cortex is not directly represented in conscious experience (Crick and Koch).

Paper topics 1) When a computer monitor that flickers too fast for the flicker to be perceived, can the unseen flicker nevertheless activate the visual cortex of your brain? Design an experiment to investigate this. 2) Is it possible to explain He and MacLeod’s results without accepting Crick and Koch’s conclusion that primary visual cortex has no immediate representation in conscious experience? 3) How might brain imaging experiments follow up on He and MacLeod’s observations?

Temporal Resolution: When Vision is Grossly Imperfect

Shady_Fig1 120 S3 S1 S AdaptPre-AdaptTest Time

Modulation Frequency (cpd) RF Width: Results = = Threshold (log  C)

Modulation Frequency (cpd) Spatial integration is anisotropic Threshold (log  C) 8´

Does the extent of spatial integration vary with carrier contrast? 50% Contrast5% Contrast

Modulation Frequency (cpd) low contrast high contrast Results: High Contrast = = Threshold (log  C) Data for subject TLData for subject MW

High Contrast Properties Loss in sensitivity at lower modulation frequencies Spatial integration minimal both along and across contours Modulation Frequency (cpd) At high contrast: low contrast high contrast Threshold (log  C) Data for subject TL

LOW frequency:HIGH frequency: Contrast Gain Control Contrast gain locally compensates for low frequency modulation Loss in sensitivity at lower modulation frequencies

Receptive Field Shrinks LOW contrast:HIGH contrast: Receptive field shrinks at high contrast. - Supported by recent physiological evidence Leads to similar behavior for integration along and across image contours.

Contrast Constant Perceived Brightness Mean Luminance time flicker

Orientation-selective adaptation from an invisible spatial pattern: invisible to us but visible at V1. There is intra-cortical loss of spatial information. V1 does not directly support conscious vision (Crick and Koch)

RF Height: integration along bars LOW frequency: HIGH frequency: Hit Miss

Method 150ms 330ms Target Stimulus time C max - C car  C =

Modulation Frequency (cpd) Threshold (log  C) RF Height: Results =

With 8.5 cpd grating: Modulation threshold doubled at 2.5 cpd Implied RF height: 6 min, or 12 cone rows

40 cpd grating: subject HF

40 cpd grating: subject JJ

40 cpd grating: subject WW

With 8.5 cpd grating: Modulation threshold doubled at 2.5 cpd Implied RF height: 6 min, or 12 cone rows With 40 cpd grating: Modulation threshold doubled near 4 cpd Implied RF height: 4 min, or 8 cone rows Would integration over 8 rows prevent aliasing?

120 cpd

Single Line Sine Wave An 12 cycle per degree, horizontal sine wave with a height of only one pixel (0.5 minutes of arc) is accompanied by subthreshold modulation of vertically adjacent pixels at the peak and at the trough of each cycle. In the additive case, the center line is in phase with adjacent lines, and in the subtractive case, 180 degrees out of phase.

Single Line A single line Sine wave at about 12 Cycles per degree Flanking Pixels added in phase at peak and trough lower threshold Flanking pixels added 180 degrees out of phase raise threshold

JJ

Single Line Targets Photons originating from adjacent lines 0.5 minutes of arc above or below the target line were about 28% as effective as photons originating from the target line itself in supporting the perception of modulation of target line intensity. This is consistent with purely optical spread, with no neural spatial integration orthogonal to the target line, as if observers were using signals from a single row of cones in this task. But in this case too, the Nyquist “limit” remains far above the actual resolution limit, which is here only about 20 cpd.