Quantifying Generalization from Trial-by-Trial Behavior in Reaching Movement Dan Liu Natural Computation Group Cognitive Science Department, UCSD March, 2004

REFERENCE Shadmehr R, Mussa-Ivaldi FA (1994). Adaptive representation of dynamics during learning of a motor task, Journal of Neuroscience 14: Donchin O, Francis JT, Shadmehr R (2003). Quantifying generalization from trial-by-trial behavior of adaptive systems that learn with basis functions: theory and experiments in human motor control. Journal of Neuroscience, 23:

OUTLINE Earlier Experiments on Adaptation during Motor Learning Earlier Experiments on Adaptation during Motor Learning A Model for Adaptation during Motor Learning Evaluation of the Model Conclusion and Discussion

Earlier Experiments on Adaptation during Motor Learning Basic Idea: trial-by-trial approach How a subject adapted to the changed dynamics of a reaching task. Materials and Methods Experiment on Adaptation to the Force Field

Experimental Setup Fig1. Experimental setup. A and B, experimental setup and the coordinate system for simulations of human arm and robot arm dynamics. C and D, force field examined. C, force filed is a function of velocity of hand (invariant in end-point coordinates).

Earlier Experiments on Adaptation during Motor Learning Basic Idea Materials and Methods Experiment on Adaptation to the Force Field Experiment on Adaptation to the Force Field  Adaptation Process  Aftereffects

Adaptation Process Fig 2. Trajectory and velocity during adaptation in force field 1. Trajectory rows show the average +- SD of hand trajectories during the first to final 250 targets. Velocity rows show the tangential hand velocities. Column A is in non force field, column B is upon initial exposure to the force field, and column C is after 1000 reaching movements in the force field. Trajectory Velocity Conclusion: The recovery of the original unperturbed response constitutes a clear example of an adaptive behavior.

Aftereffects Conclusion: Subject developed an internal model during adaptation, which is a mapping from the arm’s motion into force. Fig3. Aftereffects of adaptation to the force field. A-D, averages +-SD of the hand trajectories while moving in a null field during the training period for the first to final 250 targets. F, averages +-SD of the hand trajectories upon initial exposure to the force field. Subject generalized the internal modal broadly outside the workspace explored during adaptation. V.S. force field: Set 1 Set 2 Set 3 Set 4 non force field: A B C D non force field: Set 1 Set 4 force field: F

Earlier Experiments on Adaptation during Motor Learning Conclusion:  During adaptation to a force field that significantly changes the dynamics of a reaching movement, the CNS forms an internal model of the added dynamics.  This internal model has the power to generalize well beyond the training region. Question: How to model this generalization process?

OUTLINE Earlier Experiments on Adaptation during Motor Learning A Model for Adaptation during Motor Learning A Model for Adaptation during Motor Learning Evaluation of the Model Conclusion and Discussion

A Model for Adaptation during Motor Learning Basic Idea  Internal Model: a mapping from the arm’s position and velocity space into force.  Generalization: a reflection of how this map is encoded, often globally and linearly as a function of position of the limb. Model  Internal Model  Generalization Model

Internal Model (2)  The internal model is adapted by changing the preferred force vector associated with each basis. - : Actual force acting on the hand. - : Learning rate. - : Basis element, with a receptive field covering a specific piece of the space (one of the eight directions), used to encode limb velocity. - : Preferred force vector associated with. - : Predicted force acting on the hand. (1)  Internal model uses basis elements to encode limb velocity.

Generalization Model (3) - : Generalization function, describes how errors experienced in direction k affect the internal model for any other direction l. - : Captures the linearly relationship between position and force. - : Predicted hand position during movement n. - : Error between actual position and predicted position.  Generalization model uses the previous error to update the predicted hand position, based on generalization function.

OUTLINE Earlier Experiments on Adaptation during Motor Learning A Model for Adaptation during Motor Learning Evaluation of the Model Conclusion and Discussion

Evaluation of the Adaptation Model Basic Idea: trail-by-trail approach  Test: field trials (with applied force by the robot) catch trials (without applied forces occasionally) Experimental Setup Measure of Fitness Experiment Results

Experimental Setup Fig 4. Experiment setup. A. Experimental setup and the coordinate system for simulations of human arm and robot arm dynamics. Fig. B-E. Force fields examined.

Measure of Fitness: desired output assumption

Fig 6. Averaged performance of subjects in a clockwise curl field in three successively sets. Gray is subject data, and black is model fit. Circles indicate catch trials. values for all movements within a set are 0.77, 0.80, and 0.77 respectively for sets 1-3. Conclusion: The high degree of fit shows the good performance of the model in describing the trial-by-trial adaptation. Experiment Results (1): Performance of the Generalization Model (dynamic system) Fig 5. Description of the task. Set 1 Set 2 Set 3

Experiment Results (2): Performance of the Internal Model (adaptive controller) Shape of matrix D is compatible but too stiff. Shape of generalization function is not compatible with bimodal shape. Fig 7. Parameters of the fit using Equation 3. A and B are on data from subjects. C and D are on data from simulations with different. A and C represent matrix D. B and D represent generalization function. (3)

Conclusion: T he bases of the internal model have a bimodal shape. Fig 8. Generalization function. A is the shape of a primitive (0.21,0.21) implied by the generalization functions found in the subjects data. B is a model-free assessment of the generalization function in a clockwise curl field as explained in C. Experiment Results (3): Bimodal of the generalization function

Fig 9. Errors and the fit of the model to a set of 192 movements in variations of the standard task (each row). In A, the out-and-back structure is removed. In B-D, the force field is changed. Column (1) represents the average error across all directions. Column (2) represents the error when the movement is 180 o. Column (3) shows the shape of matrix D. Column (4) shows the shape of generalization function. Conclusion: The model works well in different force fields and target sets. Experiment Results (4): Robustness of the Model

Basic Idea Experimental Setup Measure of Fitness Experiment Results  Internal Model approximates force as a function of limb states via a fixed set of basis elements.  Basis elements broadly encodes limb position, but have a bimodal representation of limb velocity. Evaluation of the Adaptation Model

OUTLINE Earlier Experiments on Adaptation during Motor Learning A Model for Adaptation during Motor Learning Evaluation of the Model Conclusion and Discussion

Conclusion The theory relates encoding to generalization by using trial-by-trial changes during adaptation. Internal model approximates force as a function of limb states via a fixed set of basis elements. Basis elements broadly encodes limb position, but have a bimodal representation of limb velocity.

Discussion The internal model is evaluated in a feed forward manner and depends on a desired trajectory. Learning rate, stiffness and viscosity of the arm are all fixed. Basis elements are fixed, while only the force vectors associated with each basis are adapted. Reasonable but not optimal: introduction of nonlinearity could be a key factor in further development.

Thank You !