1 Evaluation and Modeling of Learning Effects on Control of Skilled Movements through Impedance Regulation and Model Predictive Control By: Mohammad Darainy Supervisor: Dr. Towhidkhah Advisors: Dr. Rostami, Dr. Ostry

2 Contents Introduction Control of Human Arm Impedance in Statics Transfer and Durability of Acquired Patterns of Human Arm Stiffness The Role of Phasic Muscle Activity on Hand Stiffness during Reaching Movements Modeling of Human Motor Control System through Impedance control and Model Predictive Control

3 What is The Joint Impedance?  The Automatic Capability of the System to Resist an Applied Load before Voluntary Intervention Takes Place (Winters et al. 1988)  Stiffness is defined as the ratio of change in force or torque to change in length or angle  Viscosity is defined as the ratio of change in force or torque to change in velocity or angular velocity

4 Which Parameters Contribute to the Joint Impedance?  Musculoskeletal Geometry has a Great Impact on Joint Impedance

5 Mussa-Ivaldi et al Roughly Speaking, The Axis of Maximum stiffness is a Line that Connect The Hand to a Point between Shoulder and Half Way Through the Elbow

6 Which Parameters Contribute to the Joint Impedance?  Musculoskeletal Geometry has a Great Impact on Joint Impedance  Level of Muscle Co-Contraction

7 Adapted From Gomi & Osu 1998

8 Which Parameters Contribute to the Joint Impedance?  Musculoskeletal Geometry has a Great Impact on Joint Impedance  Level of Muscle Co-Contraction  Reciprocal Activation of Agonist – Antagonist Muscles (Hunter & Kearney et al. 1982)  Stretch Reflex Gain (Bennett 1994)

9 What is Impedance Control?  The Idea that the Nervous System Can Modify the Mechanical Behavior of the Neuromuscular Periphery, Known as Impedance Control Theory (Hogan 1985)  Empirical Evidence: It was Shown That Subject Increase the Level of Co-Contraction in Proportion to The Level of Destabilizing Load (Milner & Cloutier 1998) During Two-Joints Movement, It was Shown That The Subject Learn To Stabilize Unstable Dynamics Using Selective Control of Impedance Geometry. (Burdet et al. 2001)

10 Burdet et al Co-Contraction Increase The Mechanical Stability of The Joint But at Greater Metabolic Cost

11 The Focus of This Thesis  The Effect of Learning on Joint Stiffness.  The Coordinate System under which Impedance Control will be Learned and Generalized.  The Impedance Control Limitation, Both in Statics and Dynamics.  To Develop a Model of Human Motor System Based on the Empirical Studies on Joint Impedance.

12 1: Learning to Control Arm Stiffness under Static Conditions  Aim To Examine The Idea That Impedance Control Involves a Process of Learning That is Acquired Over Time and Permits the Voluntary Control of The Pattern of Stiffness at The Hand.  Prediction We Predicted That The Subjects Can Voluntarily Control Their Hand Impedance Geometry and With Practice They Can Improve Their Ability of Impedance Control

13 Experimental Setup Interactive Motion Technologies, Cambridge MA Arm Configuration During The Experiment

14 Experimental Study  Subjects trained for three successive days to resist one of three loads single axis loads in lateral direction. single axis loads in forward/backward direction. isotropic loads in eight directions about a circle.   Unpredictable servo-position controlled perturbations used to assess stiffness.

15 Position Servo Controls Were Used to Estimate The Stiffness

16 Results

17 Results

18 Modeling Results

19 Conclusion 1  Stiffness Ellipse Modified With Learning in an Adaptive Fashion, Change in Orientation Matches The Direction of Load.  Modeling Studies Suggest Separate Commands for Shoulder and Elbow Co-Contraction.  Even with Practice, Impedance Change in Statics is Relatively Small in Magnitude. Simulations Suggest This is at a Limit Regardless of Level of Co-Contraction.  Clockwise Rotation of Stiffness Ellipse Greater Than Anti- Clockwise Rotation. Same Asymmetry in Simulations. Asymmetry Due to Geometry Since Simulated Commands Equate for Impedance of Shoulder and Elbow Muscles.

20 2: Transfer and Durability of Acquired Patterns of Human Arm Stiffness  Aims To Explore Whether Subjects Could Transfer Stiffness Learning to Other Parts of The Workspace And Also The Specific Pattern of Generalization as a Means to Identify The Coordinate System in Which The Learning Occurs. To Evaluated The Durability of Stiffness Training. Is The Effect of Learning Persist Well beyond The Time of Initial Learning?

21 Experimental Setup Arm Configuration During The Experiment 1 and 2

22 Experimental Study  The Tests Conducted With The Arm Stationary.   In Experiment 1 Subjects Were Trained at The Left And Transfer Test Was Conducted at The Right. The Experiment Take Place in One Day.  In Experiment 2 Subjects Were Tested for Three Consecutive Days at The Left. The Direction of The Force Application at The Third Day Was Perpendicular to The Last Two days. After The Training Phase Subjects Were Randomly Assigned to One of Two Groups. One Group of Subjects Experienced Similar Torque at The Left And Right (Group J) While for The Other Group The Experienced Force at The Hand Was Similar (Group H).

23 Results Experiment 1

24 Results Experiment 1

25 Result Experiment 2

26 Conclusion 2  Experiment 1: Both Groups of Subjects in The Training Phase of The Experiment Learned Successfully to Modify The Orientation of The stiffness Ellipse.. Transfer of Stiffness Control Was Observed only When The Direction of Load Resulted in Torques That Were Similar to Those Experienced During Training.  Experiment 2: Subjects in The First Two Days of Experiment Learned to Modify The Orientation of The stiffness Ellipse. The Change in Stiffness Ellipse Orientation Matched The Direction of Force Application Higher Stiffness Ellipse Size, Longer Time Outside The Target as Well as The Orientation of The Stiffness Ellipse on The Third Days of Experiment Shows That The Stiffness Learning Result in Durable Changes in Neural Signals Underpinning Co-Contraction of Muscles.

27 3: The Role of Phasic Muscle Activity on Hand Stiffness during Reaching Movements  Aims To Develop a New Method for Stiffness Estimation During Reaching Movements. To Explore The Effect of Phasic Muscle Activity On Hand Stiffness During Reaching Movements By Comparing The Static Stiffness with The Stiffness During Movements.

28 Experimental Setup Arm Configuration During The Experiment

29 Experimental Study  The Tests Were Conducted in Three Consecutive days  The Stiffness Ellipses Were Estimated in Three Different Conditions: Static, Down to Up and Left to Right Reaching Movements.  The Average of Last Eighty Trials During Reaching Movements and AR Model of Trial By Trial Variability Were Used as a Method for Prediction of Ongoing Movements.

30 Material and Method The Result of Position Servo Control in Both X and Y Direction For Reaching Movement in Y Direction.

31 Results

32 Results

33 Conclusion 3  The Result Shows Considerable Counter Clockwise Rotation of Stiffness Ellipse in Compare to The Static Stiffness Ellipse During Down to Up Reaching Movements.  The Rotation of Stiffness Ellipse During Lateral Reaching Movement Wasn’t Statistically Reliable.  Impedance Modification During Reaching Movements in Unstable Environments May Actually Have Limitations That Are Comparable to Those Observed Under Static Conditions. The Large Empirical Difference May be Due Simply to Phasic Muscle Activity During Movement.

34 4: A Model of Motor Control System Based on Impedance and Model Predictive Control  Aims To Develop a New Model for Motor Control System Based on The Result of Empirical Impedance Measurements During Acquisition of a New Skill. Try to Model The Effect of Learning on The Joints Impedance.

35 Model Structure A Combination of Model Based Controller (MPC) and Impedance Controller as a Model of Motor Control System

36 Simulations Conditions Reaching Movements of 15cm Toward Each of Eight Equidistance Targets Were Simulated

37 Robot Model The Model Behavior Evaluated in Curl Field The Model of Robot Defined as: In First Step 0 Assigned to K and b=[0 8; -8 0]N/m. This is The Model of a Curl Field.

38 Simulation Results To Control our Arm Perfectly We Not Only Need The Model of Environmental Dynamics But Also We Need to Know The Optimal Impedance of The Joints in Each Situation.

39 Simulation Results

40 Simulation Results

41 Simulation Results

42 Simulation Results

43 Simulation Results The Model of Robot Defined as: In This Case 0 Assigned to b and k=[150 0; 0 0]N/m. This is The Model of a Divergent Force Field.

44 Simulation Results The Controller Doesn ’ t Benefit too much from Having The Model of Disturbance.

45 Conclusion 4  To do a Skilled Movement, We Need a Precise Model of Environmental Dynamics Also The Optimal Impedance of Participating Joints are Necessary as well.  The Empirical Evidence Shows that The Subject’s Performance Decay With The Distance from The Location Where The Adaptation Take place. Our Simulation Argue That The Unavailability of Optimal Impedance in Neighboring Location is The Reason for The Above Phenomena.  Adaptation to The Stable And Unstable Dynamics Have Some Fundamental Deference. While The Adaptation to Stable Dynamics Seems to Happen Mostly in Model Based Controller, Adaptation to The Unstable Dynamics Achieved By Impedance Control.

46 General Conclusions  Joint Impedance Can be Modified With Learning in an Adaptive Fashion. Geometrical Changes in Impedance Matches The Direction of Load.  Impedance Control Learned and Generalized in Intrinsic Coordinate System.  A Combination of Model Based Controller With Impedance Controller Can Mimic The Behavior of Human Subjects in Adaptation to Stable and Unstable Dynamics.  Learning Impedance Results in Durable Changes to The Neural Signals That Underlie Impedance Control.  Impedance Control in Dynamics Have The same Limitation That Were Seen in Statics.

47 Contribution to Knowledge  For The First Time, The Effect of Learning on Geometrical Pattern of Hand Stiffness Was evaluated in This Thesis.  The Coordinate System Under Which The Impedance Control Learned and Generalized, Were Explored. These Finding Help us to Understand How Subjects Percept The Environmental Dynamics.  The Empirical Studies Showed That Although The Stiffness Control Under Static Conditions is Limited But Stiffness Learning is Durable and The Effect of Learning Can be Seen Well Beyond The Time of Initial Learning.

48 Contribution to Knowledge  A New Method of Stiffness Estimation During Reaching Movements Developed and Applied.  For The First Time, Both Static and Dynamic Stiffness at The Same Musculoskeletal Geometry were Estimate. The Results Argue That Stiffness Control During Movements May Have The Same Limitation as it Was Shown Under Static Conditions.  A New Model for Motor Control System Were Presented Based on Combination of a Model Based Controller and Impedance Controller.