1. Using Virtual Intelligent Environments to Understand and Enhance Human Motor Learning Yoky Matsuoka Division of Engineering and Applied Sciences Harvard University

2. 2 What is “Virtual Intelligent Environment”? Virtual “Visual” Environment Virtual “Haptic” Environment Virtual “Adaptive” Environment YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

3. 3 Virtual Intelligent Environments Visual environment computer generated objects computer generated body Haptic environment (sensory input) positional sensors force sensors Haptic environment (motor output) actuators joints Computation control adaptation InterfaceInterface YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

4. 4 Ultimate Goal Use virtual intelligent environments to enhance human motor learning ability –Beyond natural capability –Faster YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

5. 5 Robotic rehabilitation after injuries (stroke) Movement enhancement (Parkinson’s, CP, etc.) Electrical stimulation during athletic training Training surgeons Virtual training to prevent injuries Examples of Learning Enhancement Courtesy of Krebs and Hogan, MIT Newman Laboratory for Biomechanics and Human Rehabilitation YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

6. 6 Advantages of VIE Integration in Learning Enhancement Recording Ability –exact execution recorded to be analyzed No Biological Limitation –no neural delays Portability –remote training. Use it to Understand Biological Systems YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

7. 7 Issues with Virtual Intelligent Environments Limitations in the haptic devices Not adaptable to the changes in interacting neuromuscular systems YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

8. 8 Haptic Environment Limitations Currently available haptic device –Workspace is too small for whole-body movements active and not safe –Difficult to apply appropriate forces during task- specific movements machine constraints not matching human constraints YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

9. 9 A New Haptic Device 6 DOF (3 actuated) –yaw, pitch, linear Large workspace –1.1m radius half sphere Completely passive Actuated by magnetic particle brakes Cable driven = no backlash YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

10. 10 A New Haptic Device Designed specifically to allow task-specific whole body movement Software controlled (adaptable to individual’s need) Designed to be completely scalable. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

11. 11 New Haptic Device Controller Mechanical Device Brakes EncodersComputer DSP Motion Controller Current Amplifier Monitor Positional information Visual information Desired force/trajectory CurrentMotor command YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

12. 12 New Haptic Device Real-Time Calculations All interface is in Cartesian coordinates –Jacobians –Cartesian coordinate variables, a, v, x. – Force applied YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

13. 13 Incorporate the Biological Parameters in Virtual Intelligent Environments Making virtual intelligent environment adaptable to the neuromuscular changes How does haptic input (force) affect adaptation –for muscles –for central nervous system YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

14. 14 Adaptation to External Force: CNS YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Representation of “learned information” –How is it represented? –How does it change over time? –What are the characteristics?

15. 15 Baseline conditions of arm control Plant Internal model of force field + F ext YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Internal Model of External Force Field

16. 16 Hypotheses Modularity in generalization –the learned information transfer to other movements (Shadmehr and Mussa-Ivaldi, 1994; Wolpert et al., 1995). –the generalization is partitioned in the workspace (Gandolfo, et al. 1996). What domains do generalization occur? What are the limitations? YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

17. 17 Experimental Environment: Test with Simple Movements Haptic environment –2 link planar robot arm that applied force perturbation (F = Bv). Visual environment –hand cursor and targets displayed on the computer screen. Processing –recorded the hand location at 100 Hz. Computer Haptic Device Human YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

18. 18 Training Conditions Subjects were asked to make one continuous movement from one target to another. Force perturbation is applied perpendicular to the movement. Distort normal movement with force perturbation = motivates the CNS to produce counterbalancing forces 200 movements are executed under force perturbation. Forces are removed to observe the effect learned (aftereffect). YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

19. 19 Early TrialsLater Trials Aftereffects Spatial Generalization Test

20. 20 Directional Generalization Test YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

21. 21 Motor Primitive Position Velocity YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

22. 22 Position&velocity Learning Capacity and Size of Primitives Small primitives can represent high frequency fields Large primitives can only represent low frequency fields

23. 23

24. 24 Actual aftereffects Small PrimitivesLarge Primitives Model aftereffects YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

25. 25 Large Primitives Overlap Interfering Trajectory Training Location Middle Trajectory Training Location Interfering Trajectory Training Location Middle Trajectory Training Location 45 degrees 7.5 degrees

26. 26 Increase in Frequency of Force Field Function Human Band-limited frequency (1/cm) 285 120 67 YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

27. 27 Low Frequency Field Learning High Frequency Field Learning Learning Curve for Various Fields Initial velocity profiles Aftereffects profiles # of peaks in LFF: 2 # of peaks in HFF: 7 # of peaks in HFF: 1 YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

28. 28 Sequential Study: Difference in Aftereffects for High Frequency Field YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES HFFHFFHFF LFFMFFHFF LFFMFFnull LFFMFFUHFF

29. 29 Summary of Results Understanding how the CNS learns to interact with a virtual environment –modular representations in space and direction of the movement –effects sum and negative interference occurs when motor primitives overlap –primitives are large –high frequency components are not learnable but can help retain lower frequency components YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

30. 30 Adaptation to External Force: Muscles Can we identify the muscle impedance while interacting with a haptic device? Can we capture the change during learning? Neural Inputs BKp Ks YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

31. 31 Identify Impedance Learning Strategy in Human What is the initial strategy used to cope with unknown/unstructured environments? How does impedance change over time? After learning, what does the biology pick as the good solution for impedance for a given environment? YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

32. 32 Comparison Between Analytical and Biological Solutions We can mathematically derive optimal impedance for a linear world. –Biological system converges to the analytical solution. --- great! –Biological system converges to a different solution. --- what and why: put the biological solution back in the equations and reverse engineer. What about a nonlinear varying world where it is difficult to derive the optimal impedance? –What does the biological system do? Can it be modeled? YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

33. 33 Goal: Find the “best” impedance. –For this case, find best K hand. Uncertainty in the world –m ball, k ball, ball (0), and k hand m hand k hand m ball k ball x ball x hand Example: Linear World --- Catching a Ball ball hand m ball m hand k ball k hand YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

34. 34 Cases: 1. Hand stiffnes (k hand) is too high hand < 0 bounces up 2. Hand stiffness (k hand) is too low x hand > Thresholdbottoms out 3. Hand stiffness (k hand) is just right x ball x hand until switch is pressed k hand 0 infinite 2 3 1 Example: Linear World --- Catching a Ball m hand k hand m ball k ball x ball x hand YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

35. 35 Solve for x hand (t) and x ball (t) –initial condition ball (0) > 0 x ball (0) = 0 hand (0)= 0 x hand (0) = 0 x hand m hand k hand m ball k ball x ball YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Example: Linear World --- Catching a Ball

36. 36 Analytical Linear World to Biological Motor Control The example relates task performance to limb impedance and optimal solution. Now measure human strategy…. –“System identification” –Need a new technique YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

37. 37 Existing System Identification Techniques Time invariant systems --- easy –assume constant m, b, and k over time. –apply external impulse perturbation force. –repeat the same condition and average. YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

38. 38 Existing System Identification Techniques Time varying systems Cannot apply impulses close to each other. Need multiple impulses to solve for multiple unknowns. –PRBS (Lacquaniti, et al. 1993)

39. 39 Setup Handle Accelerometer Data Acquisition System Processor Human Subject New System Identification Technique to Observe Learning Robot Force Sensor Monitor YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

40. 40 New System Identification Technique to Observe Learning Use short duration before reflexes Clean data from force/acceleration sensors Least square fit or window analysis F m*a b*v k*x m=F/ab= (F-ma)/vk= (F-ma-bv)/x

41. 41 Phantom robot is used as the perturbation/measurement tool. Task: balance the moving ball on paddle. –ball moves at constant speed –dies when the ball falls off the paddle –perturbation applied every second Testing the New Technique

42. 42 Impedance Change with Learning k change over time b change over time m change over time

43. 43 Observe the impedance change within one catch Observe the impedance change between catches ** under development --- pilot studies underway Contact Interaction Task kbkb

44. 44 Virtual Intelligent Environments Visual environment computer generated objects computer generated body Haptic environment (sensory input) positional sensors force sensors Haptic environment (motor output) actuators joints Computation control neural networks (adaptation) YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES Brain Biological Parameters Internal Model Impedance etc.

45. 45 Example of Learning Enhancement: Application Injury prevention for manual material handling workers (recently initiated) Typical material handling work: –pick up, carry, place –pick up, toss YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

46. 46 Application: Injury Prevention Almost 100% of workers complain about some lower back pain after 5 years. Severe injury occurs and chronic pain starts when a sudden load change occurs. Nothing Some verbal instructions on the right posture Current Solutions YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

47. 47 Application: Questions What is the correct posture that reduces the chance of injuries Is there a movement execution strategy that is robust under unexpected perturbation? Can a computer teach these robust movements? YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

48. 48 Application: Training Robust Performance Training with unexpected perturbations. –pick up and toss the device handle while it applies weight and perturbation. –perturbation: under development Preventing injuries during training. –let the haptic device act as an assistive device –gravity compensated –if velocity is too high, F= - k(x-xo) YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

49. 49 Application: Measurement of the Outcome Pickup and throw a sack to a force plate under unexpected perturbation –Performance accuracy (landing location) –Performance strength (landing force) –Lower back load (5-sagittal plane model) –Movement consistency (model) YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

50. 50 Summary Goal: understand and enhance learning with virtual intelligent environments Built a large passive haptic device Investigated the change in the neuromuscular system –internal model –impedance Applications: injury prevention training YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

51. 51 Future Work Expand the neuromuscular parameter analysis to 3D movements Add active components for perception Incorporate more biological components (stress, etc.) to the virtual intelligent environments YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

52. 52 Future Work Wearable devices Implantable devices Brain image input Functional muscle stimulation output YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES

53. 53 Future Work Develop intelligent systems to enhance human learning while making fundamental discoveries –for elderly people to keep learning more –for disabled children to learn more –for injured people to recover better and faster –for normal people to learn more than what is possible without intelligent systems YOKY MATSUOKA HARVARD UNIVERSITY DIVISION OF ENGINEERING AND APPLIED SCIENCES