Visuo-Motor Relationships: Plasticity and Development

Problem of sensory-motor coordination: How do we relate the visual and motor worlds? For reaching, a visual signal about location must be transformed into a command to the arm and hand muscles. This is not innate, but must be learnt during development.

Development of reaching Within first 2 weeks, already directing arm towards objects. Some crude control of reach direction. Improves by the 5th month; consistently touch targets. Won’t reach for targets beyond arm’s length. Catching and anticipating target motion at 6 months. Distance accuracy develops more slowly, improving by 7 months.

Increased use of visual feedback between 5 and 11 months

More evidence that visuo-motor coordination must be learnt during development. Evidence: kittens given visual experience without opportunity for movement, and motor experience without vision, don’t learn how to control their movements using vision. Correlating the two is necessary (Held & Hein study).

Held & Hein Role of Experience in Development of Visuo-motor coordination Both kittens get visual experience and motor experience 1.Visual experience correlated with motor commands/proprioceptive feedback/vision of limbs 2.Gets both, but uncorrelated. Kitten 2 -abnormal visuo- motor coordination. 1 2

Also maintain ability to adapt to new relationships.

Adaptation to different relation between visual and movement. George Stratton –Wore inverting lens for 8 days –Believed that we learn visual directions by associating visual experiences with other forms of sensory feedback (e.g. proprioceptive). –Alternatively… Adaptation results from learning correlation betweeen vision and actively generated motor commands (Held, 1965). Also - prism adaptation

Why do we need to retain plasticity for new visuo-motor relationships? 1. Need to adjust to changes in body size during development. 2. Need to adjust to damage/aging. 3. Need to adjust to environmental changes eg ice, loads etc. 4. Need to learn arbitrary mappings for tool use etc. 5. Need to acquire new motor skills. 6. Visuo-motor coordination is a computationally difficult problem for the brain. Need flexibility to correct errors.

Experimental question: 1. How much plasticity is there? Can we adapt to an arbitrary visuo-motor relationship? What are the properties of this adaptation? Eg how fast, how complete ?

Experimental question 2. Another aspect of sensori-motor adaptation/learning is the ability to learn the dynamic properties of the physical world. For example, in catching balls, we can anticipate where it is going to go. This suggests we have learnt how balls move when thrown and how they bounce. How well can we do this? How precise is our knowledge of the ball’s properties?

Experimental question 3. We also take the opportunity to compare performance in real and virtual environments.

Method Virtual environment: head mounted display with virtual balls. Head is tracked with Hi-Ball sensor and updates the display every msec. Cyberglove provides vibro-tactile stimulation when ball is caught. Ping pong paddle with Polhemus Fastrack sensor which measures position of paddle with 6 degrees of freedom. Balls are generated in the virtual environment to mimic the motion of real balls. Direction and velocity of the balls can be controlled and vary randomly around a mean value. There are two kinds of ball. Red balls bounce on average a little to the left of the blue balls. Balls are launched by the experimenter by pressing a key. The subject tries to intercept the ball with the ping-pong paddle. Eye position in the virtual environment is tracked using an ASL eye-tracker. Eye position is calibrated by asking the subject to fixate 9 points in turn on a calibration display.

Method Procedure: 1.Baseline: Subject attempts to hit 30 balls (sometimes only 20). Red and blue balls are randomly intermixed. 2.Adaptation to a translation: the paddle position in the visual scene is moved 20 cm to the right of the subject’s actual hand position. The subject attempts to hit another 30 balls. 3.Adaptation to left/right reversal of hand position: The subjects attempts to hit another 30 balls. 4.Return to baseline: 15 trials with the paddle returned to the normal position, coincident with actual hand position. Data: video record of eye movements in virtual scene. Data file with number of catches recorded..

Results 1.Adaptation: a)Find proportion of catches in first 10 trials, second 10 trials, b)and third ten trials for baseline condition. b) Find proportion of catches in first 10 trials, second 10 trials, and third ten trials for translation condition. c)Find proportion of catches in first 10 trials, second 10 trials, d)and third ten trials for left/right reversal condition. d)Find proportion of catches in first 10 trials, second 10 trials, e)and third ten trials for baseline condition. Plot these values as a function of trial block. Is baseline performance stable? Is there any impairment of performance in the translation condition? Do subjects perform equally well as the baseline after 30 trials? How good is performance in the L/R reversal condition? Does it improve over trials? Does it get as good as baseline? Is there any aftereffect of the reversal condition on the normal condition? If yes, how quickly do subjects recover?

1. Eye Movements/ Learning properties of the ball. a)What is the pattern of eye movements observed? b)How similar is it to real balls? Do Ss anticipate the bounce? Do they pursue after the bounce? Is the timing of the movements the same as real balls? c) Do Ss learn that the red and blue balls have a different distribution of bounce points? Does the S fixate on the left for red balls and on the right for blue balls?

Discussion Review findings. Evaluate extent of adaptation. Was plasticity demonstrated? Are subjects really learning a new set of relationships of just learning to ignore the visual feedback? How could we distinguish these possibilities? Is there any difference in learning a translation/ versus a left/right reversal? Why might a reversal be harder to learn?

Discussion Are the eye movements similar? Are virtual environments an adequate substitution for real environments?

Neural basis of adaptation? Possible sites: Posterior parietal cortex (AIP, MIP), supplementary motor area, pre-motor, motor cortex, cerebellum, basal ganglia …

Ability to adapt to new relationships requires cerebellum

Neural control of Reaching & Grasping

Neural control of Grasping Both vPM and AIP neurons fire for specific hand actions/objects. For example, this neuron prefers a precision grip. Precision grip Power grip

Neural control of Grasping vPM neurons fire for grasping movements made in the light and dark. In contrast, AIP neurons fire far less when moving in the dark, and in general, AIP has more visual neurons than vPM.

Neural control of Grasping

Neurons in the vPM also fires when perceiving, as well, as producing grasping movements!