Alan L. Yuille. UCLA. Dept. Statistics and Psychology. How well can a Blind Person see after Recovery? STATS 19 SEM Talk 5. after Recovery? STATS 19 SEM Talk 5.

Recovery from Blindness. How well do people see after long term blindness? How well do people see after long term blindness? First studies performed in 1793, but few cases reported since. First studies performed in 1793, but few cases reported since. Ione Fine & colleagues, studied patient Mike May. (Handout 5a). Ione Fine & colleagues, studied patient Mike May. (Handout 5a). These studies involve: These studies involve: (i) Psychophysics experiments. (i) Psychophysics experiments. (ii) Neuroimaging experiments. (ii) Neuroimaging experiments.

Patient: Mike May. Patient Mike May. Patient Mike May. Blinded at age 3 ½ years. Blinded at age 3 ½ years. Sight partially restored in one eye at age 43 – by corneal and limbal stem-cell transplant. Sight partially restored in one eye at age 43 – by corneal and limbal stem-cell transplant. 2 ½ years after surgery. 2 ½ years after surgery.

Psychophysics & Neuroscience. Psychophysics – show visual stimuli, ask May to perform visual tasks. Psychophysics – show visual stimuli, ask May to perform visual tasks. Neuroimaging – use Functional Magnetic Resonance Imaging (FMRI) to measure neural activity while Mike May performs visual tasks. Neuroimaging – use Functional Magnetic Resonance Imaging (FMRI) to measure neural activity while Mike May performs visual tasks. We know areas in the brain that are tuned for specific visual tasks. We know areas in the brain that are tuned for specific visual tasks. E.G. MT – Motion area, Face area, Scene area. E.G. MT – Motion area, Face area, Scene area.

Testing Visual Abilities. Resolution Tests: the ability to resolve visual patterns. Resolution Tests: the ability to resolve visual patterns. Mike May was unable to perceive high frequency patterns (rapid oscillation). Mike May was unable to perceive high frequency patterns (rapid oscillation). Mike May’s FMRI response to these patterns in visual cortical area V1 was 1/5 of normal subject. Mike May’s FMRI response to these patterns in visual cortical area V1 was 1/5 of normal subject. Problem in the cortex, not in the retina. Problem in the cortex, not in the retina.

Simple Form Tasks. Mike May was able to perform simple form tasks: Mike May was able to perform simple form tasks: 1. Detect the orientation angle of a bar. 2. Distinguish between simple shapes – triangles and circles. These tasks are performed in V1, the first stage of the visual processing. These tasks are performed in V1, the first stage of the visual processing.

More Advanced Tasks. Regular eye exams at an Oculist only test these types of abilities. Regular eye exams at an Oculist only test these types of abilities. Embodied in Retina and V1. Embodied in Retina and V1. But vision is about decoding the world – this involves estimating depth, recognizing objects, detecting motion. But vision is about decoding the world – this involves estimating depth, recognizing objects, detecting motion. How well can Mike May perform at these tasks? How well can Mike May perform at these tasks? Beyond V1. Beyond V1.

Tasks involving 3 Dimensions. Illusory Contours: Can you see the square? Occlusion Cues: Which object is in front? Mike May has difficulties with these 3-D tasks. (Guesswork).

3D Depth Cues from Single Image. Mike May performs poorly at these tasks. Mike May performs poorly at these tasks. Relative Size: Smaller cars in the image are further away. Linear Perspective: Parallel lines converge at increasing distance.

These Cues can cause illusions. But they are reliable most of the time.

Depth from Shaded Patterns. Light and Shape: Light and Shape: Concave/Convex // Light above/below. Concave/Convex // Light above/below. Beyond V1 – except for Lee’s experiment.

3D Depth and Recognition. Mike May is poor at all these 3D depth tasks – particularly those involving using shading. Mike May is poor at all these 3D depth tasks – particularly those involving using shading. Poor at recognizing 3D objects such as faces, detecting facial expressions (happy/sad), judging gender. Poor at recognizing 3D objects such as faces, detecting facial expressions (happy/sad), judging gender. Ability to perform these tasks did not get better over time – except by guesswork. Ability to perform these tasks did not get better over time – except by guesswork. “The difference between today and two years ago is that I can better guess at what I am seeing. What is the same is that I am still guessing”. “The difference between today and two years ago is that I can better guess at what I am seeing. What is the same is that I am still guessing”.

FMRI Activity for Recognition. Mike May shows very little neural activity in the “face areas” of the brain. Mike May shows very little neural activity in the “face areas” of the brain. These areas have significant activity in normal subjects. These areas have significant activity in normal subjects. It is not known where perspective cues, size cues, and shading cues are processed. It is not known where perspective cues, size cues, and shading cues are processed.

Motion Cues. Mike May performs a lot better for motion cues. Mike May performs a lot better for motion cues. These cues are processed in area MT. May shows normal activity in these areas. These cues are processed in area MT. May shows normal activity in these areas. This includes the ability to see three- dimensional shape. This includes the ability to see three- dimensional shape.

Motion Illusions. This means that Mike May see the standard motion illusions. Barber Pole. This means that Mike May see the standard motion illusions. Barber Pole.

3D Structure from Motion. Mike May can see 3D structure from motion cues. Mike May can see 3D structure from motion cues.

Perceive Biological Motion. Mike May can perceive biological motion. Mike May can perceive biological motion.

Mike May Summary. Low spatial resolution in V1. (Less than predicted by the resolution of the retina). Low spatial resolution in V1. (Less than predicted by the resolution of the retina). Poor ability to process static 3D depth cues and to recognize 3D objects such as faces. Poor ability to process static 3D depth cues and to recognize 3D objects such as faces. But good at simple shape tasks (orientation of bars). But good at simple shape tasks (orientation of bars). And good at tasks involving motion cues. MT. And good at tasks involving motion cues. MT.

Two Probable Explanations. (1). Development. Infants/Children develop different visual skills at different times. (1). Development. Infants/Children develop different visual skills at different times. They are most sensitive to motion cues – probably Mike May’s motion processing system was well developed before he lost his sight (aged 3 ½). They are most sensitive to motion cues – probably Mike May’s motion processing system was well developed before he lost his sight (aged 3 ½). (2). Plasticity. The visual cortex (& all the cortex) shows plasticity – adapting to experience. (2). Plasticity. The visual cortex (& all the cortex) shows plasticity – adapting to experience. E.g. He could have developed face recognition, but the face area would be underused, and then converted for some other use. E.g. He could have developed face recognition, but the face area would be underused, and then converted for some other use.

Development of Infant Vision. Studies in the last twenty years shows that infant’s visual systems are remarkably sophisticated. Studies in the last twenty years shows that infant’s visual systems are remarkably sophisticated. Ability to use size cues may be present within a week of birth. Ability to use size cues may be present within a week of birth. Testing: an infant will pay attention to a novel stimulus. It will get bored and look away if the stimulus is perceived to be familiar. Testing: an infant will pay attention to a novel stimulus. It will get bored and look away if the stimulus is perceived to be familiar. (Ability to count). Handout 5b. (Ability to count). Handout 5b.

Development of Infant Vision. Different visual abilities appear at different stages. Different visual abilities appear at different stages. The order and (approximate) timing of these stages is common to all infants. The order and (approximate) timing of these stages is common to all infants. E.G. certain depth cues (binocular stereo) will appear between 4 and 12 weeks. E.G. certain depth cues (binocular stereo) will appear between 4 and 12 weeks. Differences may be due to the richness of the visual environment or genetic factors. Differences may be due to the richness of the visual environment or genetic factors.

Infant Development of Vision. Example: Infants are able to form perceptual categories between ages 3 to 4 months. Example: Infants are able to form perceptual categories between ages 3 to 4 months. Distinguish between giraffes, horses, fish, etc. Distinguish between giraffes, horses, fish, etc. Environment: claim that perspective depth cues are only learnt in suitable environments. Reports that some tribes, adapted to jungles, do not develop perspective depth cues. Environment: claim that perspective depth cues are only learnt in suitable environments. Reports that some tribes, adapted to jungles, do not develop perspective depth cues.

Development of Visual Abilities Some abilities take time to develop. Some abilities take time to develop. Example, the ability to re-create spatial layouts from memory increases from age 5 to age 9. Example, the ability to re-create spatial layouts from memory increases from age 5 to age 9. Field independence – the ability to separate an image into parts (e.g. detect partially hidden objects) increases from age years. Field independence – the ability to separate an image into parts (e.g. detect partially hidden objects) increases from age years.

Motion Cues develop very early. Infants can use motion cues to detect objects, estimate 3D shape, and recognize objects. Infants can use motion cues to detect objects, estimate 3D shape, and recognize objects. They do this long before they can reliably perform these tasks from static images. They do this long before they can reliably perform these tasks from static images. Conjecture: motion cues are more reliable. Infants are genetically predisposed to use them. Conjecture: motion cues are more reliable. Infants are genetically predisposed to use them.

Motion and Mike May. Hypothesis: Hypothesis: Mike May is good at using motion cues, because this part of his visual system was well developed before his accident. And it did not decay afterwards. Mike May is good at using motion cues, because this part of his visual system was well developed before his accident. And it did not decay afterwards. But, by age 3 he should also be able to use static depth cues and recognize faces. But, by age 3 he should also be able to use static depth cues and recognize faces.

Plasticity. The cortex of humans and monkeys is plastic – experience can alter visual areas. The cortex of humans and monkeys is plastic – experience can alter visual areas. Monkey finger’s experiment. Monkey finger’s experiment. FMRI studies of blind subjects suggest that they use their higher visual areas for non visual processing (visualization). FMRI studies of blind subjects suggest that they use their higher visual areas for non visual processing (visualization). Mike May could have lost his ability to detect faces, because his brain face area was adapted for something else. (Use it or lose it). Mike May could have lost his ability to detect faces, because his brain face area was adapted for something else. (Use it or lose it).

Other Studies. Mike May is probably the best studied patient (Psychophysics and FMRI). Mike May is probably the best studied patient (Psychophysics and FMRI). But other studies give similar results – e.g. inability to perceive 3D depth from static images, poor ability at recognition, ability to see light patterns but inability to detect them. But other studies give similar results – e.g. inability to perceive 3D depth from static images, poor ability at recognition, ability to see light patterns but inability to detect them. Vision as a distraction. Tasks such as crossing the street, or skiing, can be easier when blind! Vision as a distraction. Tasks such as crossing the street, or skiing, can be easier when blind!

Studies of Other Patients. Studies of sight recovery after extended blindness show dependence on age, length of blindness, and amount of residual vision. Studies of sight recovery after extended blindness show dependence on age, length of blindness, and amount of residual vision. Mid-life visual diseases – glaucoma, macular degeneration, cataracts, etc. – are the ones that patients can best recover from. Mid-life visual diseases – glaucoma, macular degeneration, cataracts, etc. – are the ones that patients can best recover from.

Reactions to Recovered Sight. Some reactions are surprising: Some reactions are surprising: 1. “It made me kind of angry that people were walking around in this colorful world that I never had access to.” 2. “People live so much closer to each other than I had realized…I’m not used to the idea that I can tell so much about them by looking.” 3. “A lot of people have got pimples recently”.

Current Limitations of Sight Recovery. Important to understand the long-term effects of visual deprivation. Important to understand the long-term effects of visual deprivation. Currently many sight recovered patients are limited by their cortical vision system (V1 and beyond) to make use of unfamiliar visual input. Currently many sight recovered patients are limited by their cortical vision system (V1 and beyond) to make use of unfamiliar visual input. Understanding the limitations may lead to better rehabilitation – fixing the brain. Understanding the limitations may lead to better rehabilitation – fixing the brain.