1. Movement Imitation: Linking Perception and Action Advanced Topics in Computer Vision, 2004 Lior Noy Department of Computer Science and Applied Mathematics Weizmann Institute of Science

2. Movement Imitation - Example

3. semantic world (objects, actions) realm of raw-data (pixels, muscles activation) Action Perception Imitation: Linking Perception and Action Imitation

4. Outline 2. Programming By Demonstration 1. Movement Imitation 3. Robotic Movement Imitation Primitives Based Approach (Mataric’) Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

5. A Variety of Probes into Imitation Imitation Ethology Cognitive psychology Developmental psychology Neurophysiology Human Brain Imaging Robotics

6. Possible Questions In Imitation Research 1. What is the content of imitation? 2. How perceptions are transformed to actions? 3. What are the processes of learning by imitation? 4. How to evaluate imitation? What is a “ good ” imitation? 5. How does the ability to imitate develop?

7. Evaluating Imitation Robot Following in a Hilly Environment

8. Evaluating Imitation

10. Programming By Demonstration (PbD) Methods to program a robot 1. Human Programming 2. Reinforcement Learning 3. Programming by Demonstration

11. Programming By Demonstration (PbD) Applications  Navigation  Locomotion  Playing air-hockey  Manipulating blocks  Balancing a pole  Hitting a tennis-serve  Grasping unfamiliar objects  Imitating dancing movement

12. PbD – Application Example The “Golden Maze”

13. PbD – Application Example Playing Air-Hockey

14. PbD – Application Example Box Manipulations

15. Three Approaches for PbD 1.Symbolic 1.Symbolic 2.Control-Based 3.Statistical

16. Symbolic Approach for PbD o Analyze observed actions in terms of sub-goals o Match actions needed to fulfill these sub-goals o Create a symbolic description of the environment ( ”object A is above object B” ) o Learn a series of symbolic if-then rules ( ”if object A is above object B then grasp-object[ object B ]” )

17. Example: Symbolic Approach for PbD (Kunyushi et al., 1994) … but how do you symbolically describes “hitting a tennis serve”?

18. Control-Based Approach for PbD  No symbolic parsing of perceived actions  Assume a pre-defined control policy  Acquire needed parameters from observation

19. Control-Based Approach Inverse Models Sometime assume known inverse models (converting desired effect to needed commands) motor commands joint angles end-effector position Forward Models Inverse Models

20. Example: Control-Based Approach for PbD (Schaal, 2003) Tennis movie

21. Statistical Approach for PbD  No prior assumption on used control policy  Statistically match perception and action  Can this be done? More on this later…

22. Example: Statistical Approach for PbD PCA (Asada, 1995)

23. Example: Statistical Approach for PbD Learning: Perform random action A(i) Record resulted optical flow f(i) Compute principal-component p1(i), p2(i) Learn the connection A(i) – {p1(i), p2(i)}

24. Outline 2. Programming By Demonstration 1. Movement Imitation 3. Robotic Movement Imitation Primitives Based Approach (Mataric’) Real Time Tracking (“mirror-game”) (Ude et al.) 4. Direct Perception and Imitation

25. PbD for Movement Imitation Pre-Cursor 1: Cartoons Retargeting (Bregler et al., 2002)

26. Cartoons Retargeting Two Types of Deformations 1. Affine deformation 1. Key shape deformation y x

27. Cartoons Retargeting Affine Deformations  Affine parameters y x

28. Cartoons Retargeting Affine Deformations

29. Cartoons Retargeting Key Shape Deformations  S k are the key shapes

30. Cartoons Retargeting Key Shape Deformations

31. Cartoons Retargeting - Results More on: http://www.cs.weizmann.ac.il/~hassner/cv03/ “Animating human motion”, Speakers : Simon Adar, Yoram Atir

32. PbD for Movement Imitation Pre-Cursor 2: Guided Movement Synthesis (Zelnik-Manor, Hassner & Irani, 2004)

33. Event-Based Analysis of Video (Zelnik-Manor & Irani, 2001)

34. Guided Movement Synthesis (a.k.a. “Movement Imitation”?)

35. PbD for Movement Imitation Pre-Cursor 2: Movement Synthesis

36. PbD for Movement Imitation Case Study: Primitive-Based Approach The Problem: How to convert visual input to motor output? A Possible Solution: Use a common, sparse representation: sensory-motor primitives. X1X2...XnX1X2...Xn J1J2...JmJ1J2...Jm == … but what primitives to use? movement primitive1 movement primitive2... movement primitive K

37. Movement Imitation Using Sensory-Motor Primitives Motor primitives: Sequences of action that accomplish a complete goal-directed behavior. Examples: 1. Move hand in “straight line”, “parabola” (Felix…). 2. Perform “grasping”, “a tennis serve”.

38. Imitation Learning Using Sensory-Motor Primitives (Schaal, Ijspeert & Billard, 2003)

39. Inspiration for Using Sensory-Motor Primitives (Rizzolatti et al., 2002; Gallese et al. 1996) Evidence for: 1. C oding of goal-directed actions. 2. Shared representations of perception and action. Example – Mirror Neurons.

40. Movement Imitation Using Sensory-Motor Primitives (Mataric’,1998) General Principles: 1.Selective attention focusing on end-points movements. 2.Sensory-motor primitives as integrative representation. 3.Learning new skills as compositions of primitives. 4.Experimental test-beds.

41. What Sensory-Motor Primitives to Use? Primitives Innate Pre-defined control policies (e.g., central pattern generators) Learned Un-supervised clustering (using PCA, Isomap ) Joints Space (“motor space”) End-Points Space (“visual space”)

42. “Experiment in Imitation Using Perceptuo-Motor Primitives”, (Weber, Jenkins & Mataric’,2001) 1.Extract hand (end-point) movements. 2.Perform Vector-Quantization to get invariant representation.

43. 3.Classify movement to primitives (line, arc, circle). 4.Group adjacent similar primitives. “Experiment in Imitation Using Perceptuo- Motor Primitives”

44. 5.Determine primitives parameters. 6.Project to ego-centric space. “Experiment in Imitation Using Perceptuo- Motor Primitives”

45. (Weber, Jenkins & Mataric’,2001) “Experiment in Imitation Using Perceptuo- Motor Primitives”

46. PbD for Movement Imitation Case Study: Real-Time Tracker (Ude et al.,2001) o The Goal: Mimic movements in real-time o The Problem: Large amount of data to process (6 MB/Sec) Need “continuous success” o The Solution: Probabilistic approach to prevent excessive data interactions

47. “Real-Time Visual System for Interaction with Humanoid Robot” (Ude, Shibata & Atkeson, 2001) Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

48. Real-Time Tracker Tracking “Blobs” In a Bayesian Setting probability for the pixel at location u to have Intensity I u Given the process k a-priori probability for process k

49. Real-Time Tracking Minimize Log-Likelihood overall probability to observe image I Goal: determine the parameters that are most likely to produce this image – Maximal Likelihood Problem. computationally easier to minimize the negative log likelihood

50. Find minimum (using Lagrange Multipliers) and get: probability that pixel u stems from process l Real-Time Tracking Minimize Log-Likelihood

51. Real-Time Tracking Find Probabilities Parameters The above equations are solved iteratively by the Expectation-Minimization (EM) algorithm Expectation stage: compute P u,l using the current estimate for Ө and ω. Minimization stage: compute new Ө and ω assuming P u,l are constant. from probabilities of pixels to belong to a certain process (e.g. – the human hand) …

52. “Real-Time Visual System for Interaction with Humanoid Robot” … to object locations

53. Real-Time Tracking General Stages Estimate positions of tracked “blobs” in the image Compute 3D coordinates of tracked objects using stereo Transform into via-points for robot hand trajectory Compute motor commands from desired trajectory

54. Real-Time Tracking Estimate Trajectories with B-splines

55. Real-Time Tracking - Results Robot Compliance Movie

56. References “Vision-Based Robot Learning for Behavior Acquisition” M. Asada, T. Nakamura, and K. Hosoda. Proc. of IEEE International Conference on Intelligent Robots And Systems 1995 (IROS '95) Workshop on Vision for Robots, pp.110-115, 1995. “Turning to the masters: Motion capturing cartoons” Bregler C, Loeb L, Chuang E, Deshpande H ACM TRANSACTIONS ON GRAPHICS 21 (3): 399-407 JUL 2002 “Movement, activity and action: The role of knowledge in the perception of motion” Bobick AF PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY OF LONDON SERIES B- BIOLOGICAL SCIENCES 352 (1358): 1257-1265 AUG 29 1997 “Challenges in Building Robots That Imitate People”, Breazeal C. and Scassellati B, in "Imitation in Animals and Artifacts", Kerstin Dautenhahn and Chrystopher Nehaniv, eds. The MIT Press, 2002. “Action recognition in the premotor cortex” Gallese V, Fadiga L, Fogassi L, Rizzolatti G BRAIN, 119: 593-609 Part 2 APR 1996 “Learning by watching - extracting reusable task knowledge from visual observation of human-performance” Kuniyoshi Y, Inaba M, Inoue H IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, 10 (6): 799-822 DEC 1994 “Sensory-Motor Primitives as a Basis for Learning by Imitation: Linking Perception to Action and Biology to Robotics.” Maja J Mataric, in "Imitation in Animals and Artifacts", Kerstin Dautenhahn and Chrystopher Nehaniv, eds., MIT Press, 2002, 392-422

57. References “From mirror neurons to imitation: facts and speculations”, Rizzolatti G, Fadiga L, Fogassi L and Gallese V, in: Meltzoff AN and Prinz W (Eds.) "The imitative mind: development, evolution, and brain bases", New York: Cambridge University Press, 2002 “Computational approaches to motor learning by imitation” Schaal S, Ijspeert A, Billard A PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY OF LONDON SERIES B- BIOLOGICAL SCIENCES, 358 (1431): 537-547 MAR 29 2003 “Movement planning and imitation by shaping nonlinear attractors” Schaal S, PROCEEDINGS OF THE 12TH YALE WORKSHOP ON ADAPTIVE AND LEARNING SYSTEMS 2003 “Robots that imitate humans” Scassellati B. Breazeal C. Trends in Cognitive Science, 6(11):481 487, November 2002. “Real-time visual system for interaction with a humanoid robot”, Ude A., Shibata T. and Atkeson C. G., Robotics and Autonomous Systems, 37:115 125, 2001. Stefan Weber, Odest C. Jenkins, and Maja J. Mataric´. "Imitation Using Perceptual and Motor Primitives". In International Conference on Autonomous Agents, pages 136-137, Barcelona, Spain, Jun 2000 "Event-Based Analysis of Video “, Zelnik-Manor L. and Irani M., IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION, December 2001 (CVPR'01).

58. “The great end of life is not knowledge but action.” (Thomas H. Huxley) Perception? Action?