Reaching and Grasping: what problems does the brain have to solve?

Autonomous control: robot is pre-programmed - no human input Problems to be solved: Flipping an Egg 1. Grasp spatula 2. Move to pan 3. Lower spatula to pan 4. Flatten 5. Locate egg locate handle (vision) some mechanism to translate location into arm movement some mechanism for controlling fingers - “passive compliance” locate pan (vision) translate location into arm movement vision or proprioception: lower until force > 0 proprioception: rotate until forces on fingers are equal vision or proprioception: move forwards until horizontal force > 0

Problems to be solved: Flipping an Egg (ctd) need to keep spatula level: vision or proprioception (keep tension constant) need to learn how much to rotate hand: rotate until forces = 0 Limitations of autonomous control: inflexible - can’t adapt to changed circumstances requires high precision 6. Lift 7. Flip

Tele-operation: movement controlled by human operator Limitations: feedback delays lead to overshoots - slow Tele-assistance: hybrid system - component of movement controlled by a single sign, or instruction. Combines flexibility of feedback control with speed of autonomous control.

Factors in control of reach: Learnt motor program for ballistic phase (feedforward) Feedback during reach from vision or proprioception. Ballistic component might rely on a simple property of muscles, that is, muscles are like springs.

Mass-Spring Model Muscles are like springs: a spring has a resting length and stiffness length and stiffness of muscles can be neurally programmed

Evidence for Mass-Spring Model Accurate (??) reaching following deflection even when vision and proprioception were absent (dorsal roots severed).

Computer Simulation of Reaching using a Mass-Spring Model Ballard and Gu, 2005

Movement Data Comparison (1) End effector trajectories human data ooo prediction Speed profile along x Speed profile along y

Movement Data Comparison (2) Joint profile comparison ShoulderElbowWrist

Walking Profile Comparison Human data from Boston dynamicsModel simulation data HipKnee Ankle

Lab 2: feedback and feedforward control of reaching Phantom Force-Feedback system

1.Latency of visuo-motor feedback loop. Measure reaction time to a change in object position during a reach. Observer makes 5-10 practice reaches from yellow disk in front of workspace to pick up cube at back of workspace. Experimenter says GO. Reach at normal speed (try to keep constant). Pick up cube, put it down, and return to starting point. After an unpredictable number of trials the experimenter will move the object to a random location during the reach. The observer must correct the reach to pick up the cube. Do 10 such trials, 5 of which have a displacement, 5 don’t. Record data for last 2 practice trials and the 10 experimental trials. Analysis: Plot XY trajectory in the plane of the table. Indicate point where displacement occurred, and point where change in trajectory is evident. The difference in these times is the visuo-motor latency. Find average and standard deviation of these values. How does this experiment relate to Experiment 1?

x x t1t1 t2t2 Visuo-motor latency = t 2 - t 1 X-Y Plot of reach trajectory

2. Catching a moving object. Measure reach trajectory to a moving object. As in the previous experiment, but object moves back and forth along the back wall. Observer must touch the front surface of the object. Do 5 practice reaches with the object stationary. Observers should reach as fast as possible in all trials. Record the last 2 of these. Do 6 more reaches with the object moving. Movement begins just before reach. Use 3 speeds - slow, medium, fast. Different speeds for different subjects Variation: fixate point on back wall while attempting to touch the object. How does performance compare to the normal condition? Analysis: Plot X position versus time for object and hand. Is there a lag? Does the lag reduce with practice? Does time to contact reduce with practice? Do observers learn to predict? Prediction indicates feedforward control. How does this experiment relate to Experiment 1?

Model Extension of Walking Bi-pedal walking: –4 Via points are defined for one cycle of walking –Every leg has 3 springs, which control both extension and flexion