1. Dec, 2003 Amir Karniel Jerusalem in Motion Feedback, Adaptation, Learning or Evolution: How Does the Brain Coordinate and Time Movements? Amir Karniel Department of Biomedical Engineering Ben Gurion University of the Negev The studies presented were done in collaboration with: Gideon Inbar, Ronny Meir, and Eldad Klaiman - Technion Sandro Mussa-Ivaldi - Northwestern University The first workshop of THE CENTER FOR MOTOR RESEARCH December 18-21, 2003

2. Dec, 2003 Amir Karniel Jerusalem in Motion The Hierarchy of Wide Sense Adaptation Equilibrium trajectories and internal models Reaching movements muscle models and adaptation Adaptation to Force Perturbations Time representation Sequence learning and switching Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models Summary and Future Research Outline

3. Dec, 2003 Amir Karniel Jerusalem in Motion Two Important Concepts in the Theory of Motor Control Equilibrium Inverse Model xy ydyd F -1 (y d ) F(x) Feldman Bizzi et al. +Minimum Jerk, Flash and Hogan +Force fields, primitives, Mussa-Ivaldi Albus (cerebellum) Inbar and Yafe (signal adaptation) +feedback error, Kawato +distal teacher, Jordan Adaptation Change of Impedance Change of the inverse

4. Dec, 2003 Amir Karniel Jerusalem in Motion Reaching movements Feed-Forward Control Invariant Features: Roughly straight line, bell shaped speed profile (Flash & Hogan 1985) Key Questions What is the origin of the invariance ? How do we handle external perturbations ? MJT

5. Dec, 2003 Amir Karniel Jerusalem in Motion A Hill-type mechanical muscle model The viscose element B is not a constant !

6. Dec, 2003 Amir Karniel Jerusalem in Motion Linear Vs. Nonlinear Muscle Model Linear modelThe nonlinear Hill-type model The physiologically plausible nonlinear model can produce the typical speed profile with a simple control signals Karniel and Inbar (1997) Biol. Cybern. 77:173-183

7. Dec, 2003 Amir Karniel Jerusalem in Motion Other typical features of rapid movements are also facilitated by the nonlinear muscle properties In this set of simulations the one-fifth power law model was used. Karniel and Inbar (1999) J. Motor Behav. 31:203-206

8. Dec, 2003 Amir Karniel Jerusalem in Motion Adaptation to force perturbations Modified with permission from Patton and Mussa-Ivaldi No Force After-Effects Force Field After Learning Force Field Initial Exposure Force field exposure  recovery of unperturbed pattern Removal of field  “after-effects” (Shadmehr & Mussa-Ivaldi 1994)

9. Dec, 2003 Amir Karniel Jerusalem in Motion Hierarchical system with feedback adaptation and learning Musculoskeletal system Dynamics determine the control signal (e.g., EPH, CPG, …) Internal models for control Desired Target Actual Performance Feedback Learning Adaptation

10. Dec, 2003 Amir Karniel Jerusalem in Motion Time Scale Change Scale No Change Feedback Parameters Change Structural Change Functional Change Adaptation Learning Evolution mSecMinutesYears Myears The hierarchy of wide sense adaptation Karniel and Inbar (2001), Karniel (In preparation)

11. Dec, 2003 Amir Karniel Jerusalem in Motion The Hierarchy of Wide Sense Adaptation Equilibrium trajectories and internal models Reaching movements muscle models and adaptation Adaptation to Force Perturbations Time representation Sequence learning and switching Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models Summary and Future Research Outline

12. Dec, 2003 Amir Karniel Jerusalem in Motion What are the limitations of adaptation? Key Questions: Plant & Environment Controller Example: Force Field Internal Representation of the field What is the structure of the modifier ? Could it be a function of position, velocity, time, … ?

13. Dec, 2003 Amir Karniel Jerusalem in Motion Time Representation These systems are indistinguishable therefore 1.The existence of time variable isn’t sufficient to define time representation. 2.It is sufficient to consider the following form: 

14. Dec, 2003 Amir Karniel Jerusalem in Motion Time Representation - Definition The system is said to be capable of time representation if there exists a deterministic function h(x) such that for any u(t). The system is said to be capable of time representation of up to T seconds with ε accuracy if there exists a deterministic function h(x) such that for t<T and for any u(t).

15. Dec, 2003 Amir Karniel Jerusalem in Motion The experiment Null Learning Generalization No external field External Force field time/state/sequence dependent Number of movements ~100 ~500 ~100

16. Dec, 2003 Amir Karniel Jerusalem in Motion Time Varying Force Field The force field is not correlated with the movement initiation, therefore there is no way to use state information. Only time representation would allow adaptation and after-effects for this field.

17. Dec, 2003 Amir Karniel Jerusalem in Motion Result: No adaptation to this TV force field A control experiment with the viscous curl field The maximum distance from a straight line during “learning” Karniel and Mussa-Ivaldi (2003) Biol. Cybern.

18. Dec, 2003 Amir Karniel Jerusalem in Motion Viscous Curl Force Field

19. Dec, 2003 Amir Karniel Jerusalem in Motion Result: There is Significant Adaptation with This Sequence of Force Fields The maximum distance from a straight line during “learning” A control experiment with the viscous curl field

20. Dec, 2003 Amir Karniel Jerusalem in Motion Direction Error Calculation “B + ” DE is Positive Therefore: Positive DE: Yielding to the field Negative DE: Over resisting the field 2.If the deviation is to the right multiply by –1 1.Find the Euclidean distance from a straight line at the point of maximum velocity (The feed-forward part of the movement) 3.If the curl field in the sequence is B - multiply by –1

21. Dec, 2003 Amir Karniel Jerusalem in Motion Catch trials – After Effects A few trials without force field were introduced unexpectedly. The left bar is the mean of the error (DE) during these trials in the first part of the learning. The right bar is in the last part. Significant expectation to the correct field after learning i.e., learning of an internal model of the force field

22. Dec, 2003 Amir Karniel Jerusalem in Motion Mid – Summary No adaptation in the case of the time dependent force field Adaptation in the case of the simplest sequence of curl viscous fields with four targets. What is learned in the second case?

23. Dec, 2003 Amir Karniel Jerusalem in Motion Odd and Even Movement During the learning it is possible to assign a unique force field to each movement instead of learning the sequence of force fields. The generalization phase would violate this representation. Force Field: B + B -

24. Dec, 2003 Amir Karniel Jerusalem in Motion Refuting the Sequence Learning Assumption 1. Analysis of errors in the last part where diagonal movements are introduced Force Field: B + B - The same sequence is applied in this part; sequence learning predicts similar errors

25. Dec, 2003 Amir Karniel Jerusalem in Motion Distance Error Analysis of movements in part 1 and part 5 The sequence learning assumption predicts similar errors in the right two bars that is smaller than the first, left bar Left bar: Catch trials in part 1. Middle bar: Movements in part 5 that are inconsistent with the learning phase. Right bar: Movements in part 5 that are consistent with the learning phase. All movements are consistent with the sequence of force field. However, ANOVA of the data shows similar error in the first two bars and significantly smaller error in the right bar!

26. Dec, 2003 Amir Karniel Jerusalem in Motion Refuting the Sequence Learning Assumption We found that when the perturbation can be modeled both as a function of sequence and as a function of the state, the brain generates a state dependent model. We tried to train subject with the same sequence but with three targets. In this case one needs to follow the temporal sequence in order to adapt Can we design an experiment where only sequence representation would allow adaptation? Would the brain adapt to this perturbation?

27. Dec, 2003 Amir Karniel Jerusalem in Motion Result: No Adaptation to the Sequence of Force Fields! A control experiment with the viscous curl field The maximum distance from a straight line during “learning” Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21

28. Dec, 2003 Amir Karniel Jerusalem in Motion Catch trials – No After Effects A few trials without force field were introduced unexpectedly. The left bar is the mean of the error (DE) during these trials in the first part of the learning. The right bar is in the last part. No significant expectation to the correct field after learning i.e., no learning of an internal model to the sequence!

29. Dec, 2003 Amir Karniel Jerusalem in Motion Mid – Summary (2) No adaptation in the case of time dependent force field Adaptation when the temporal sequence coincide with single state mapping No adaptation in the case of sequence of force fields Maybe it is too difficult to construct two internal models simultaneously Multiple Models Conjecture (“soft” version): If each force field is experienced separately and enough time is given for consolidation of each model, then the multiple model would be constructed Karniel and Mussa-Ivaldi (2003) Biol. Cybern. 89:10-21

30. Dec, 2003 Amir Karniel Jerusalem in Motion Day 1 Day 2 Day 3 Day 4 Early Training Late Training Catch-Trials Karniel and Mussa-Ivaldi EBR 2002

31. Dec, 2003 Amir Karniel Jerusalem in Motion Result: Clear learning of each perturbation, but No evidence for ability to utilize multiple models and context switching (Subject E) Error [DE, mm] during early and late training Error [DE, mm] during catch trials Day 1 Day 2 Day 3 Day 4

32. Dec, 2003 Amir Karniel Jerusalem in Motion Does the brain employs clocks counters or switches ? In contrast to artificial devices that are based on clock counters and switches the brain seems to prefer state dependent maps

33. Dec, 2003 Amir Karniel Jerusalem in Motion The Hierarchy of Wide Sense Adaptation Equilibrium trajectories and internal models Reaching movements muscle models and adaptation Adaptation to Force Perturbations Time representation Sequence learning and switching Bimanual Coordination Symmetry at the perceptual level as an invariant feature Tapping experiments and first indications for internal models Summary and Future Research Outline

34. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Coordination (1) Preference for in-phase symmetry Stable vs. Unstable Homologous muscles Figure from Kelso and Schöner (1988)

35. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Coordination (2) It was recently shown that the preference for symmetry in bimanual coordination is perceptual Figure from Mechsner et al. (2001)

36. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Coordination (3) Untrained individuals are unable to produce non-harmonic polyrhythms However, with altered feedback (gear) they are able to generate symmetrical movement of the flags and non-symmetrical movements of the hands. Again: The preference for symmetry is perceptual Figure from Mechsner et al. (2001)

37. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Coordination (4) The preference for symmetry was explained in terms of stable solution of dynamic system without employing internal models. Following the vast literature about reaching movements we propose an alternative Hypothesis: The brain contains internal representation of the transformation between the perceptual level and the execution level in order to maintain the symmetry invariance in face of altered feedback or other external perturbations. Predictions: 1. Learning curves, 2. After effects

38. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Index Tapping Experiment The right hand received slower feedback such that when the display shows rotation at equal speeds the subject eventually produces a non-harmonic polyrhythm, with a left/right tapping frequency ratio of 2/3

39. Dec, 2003 Amir Karniel Jerusalem in Motion Learning Curve Regression (Standardized Data) From: Karniel A, Klaiman E, and Yosef V, Society for Neuroscience 2003

40. Dec, 2003 Amir Karniel Jerusalem in Motion After-Effect Indications After-Effect Indications The last 60 seconds of each half in the experiment

41. Dec, 2003 Amir Karniel Jerusalem in Motion Bimanual Adaptation Hypothesis Symmetry Invariance Adaptable transformation from the perception level to the execution level After effects The structure, learning rates and generalization capabilities are subjects for future research

42. Dec, 2003 Amir Karniel Jerusalem in Motion Future Research Relative role of each level, muscles, spinal cord, central nervous system The structure of internal models (learning capabilities and generalization capabilities) Virtual Haptic Reality The Robo-Sapiens age Mathematical Analysis, Simulation, Experiments

43. Dec, 2003 Amir Karniel Jerusalem in Motion Turing-like test for motor intelligence: The Robo-Sapiens age Building a robot that would be indistinguishable from human being