1. National University of Singapore
IEEE International Conference on 
Robotics and Automation (ICRA 2017)
Singapore, May 29 - June 3, 2017
Learning and Control of Robots in Interacting with Unknown Environments
Yanan Li PhD BSc, Imperial College London
Shuzhi Sam Ge FIEEE, FIFAC, FIET, FSAE, PhD, DIC, BSc
National University of Singapore

2. Vision of Social Robots
Social robots: Robots that are able to interact and communicate among themselves, with humans, and with the environment, within the social and cultural structure attached to their roles.
-International Journal of Social Robotics
More robots out of the laboratory into schools, homes and clinic places…
Interacting with people in their own space
Changing everyday lifestyle

3. International Journal of Social Robotics
Editor in Chief:
Shuzhi Sam Ge, NUS
Co-Editor in Chief:
Oussama Khatib, Stanford University, USA
Aims and scope:
Provide a platform for presenting findings and latest developments in social robotics, covering relevant advances in engineering, computing, arts and social sciences.
Provide an overview of the current state of the social robotics scene.
1 Volume(-s) with 5 issue(-s) per annual subscription
IF: 1.407

4. Outline Introduction Impedance Learning Intention Estimation
Zero Force Regulation
Conclusion and Future Work

5. Control: Industry vs Social
Accuracy
Fast response
Fixed environment
Interaction
Safety
Compliant behavior
Unknown environment

6. Intelligent Control
Human adjusts their limb impedance and trajectory when catching a ball.
It is possible to apply impedance and trajectory learning skills to robot control.
Human Limb
Robot Arm

7. Intelligent Control
Robot will act as a load to the human partner if it plays a follower role.
Robot’s trajectory is adapted according to human’s intention.
Human-Robot Collaboration
Mass-Damping-Stiffness System
Central Nervous System: Motion Intention

8. Intelligent Control
Hybrid Position/Force Control: controlling force and position in a non-conflicting way
Impedance Control [N. Hogan, 1985]: developing a relationship between force and position
Passive
Robot
Environment
Stable [J. E. Colgate, 1988]
Passive systems include arbitrary combinations of masses, springs and dampers, linear or nonlinear.
require the decomposition of two directions

9. Problems and Solutions
How to impose a target impedance model on a robot arm, in the presence of uncertainties and unknown robot dynamics?
Impedance control design (available in the literature)
How to find a target impedance model?
How to determine the impedance parameters, subject to unknown environment dynamics?
Impedance learning
Impedance adaptation
How to determine the rest position/desired trajectory in the impedance model?
Intention Estimation
Zero Force Regulation

10. Outline Introduction Impedance Learning Intention Estimation
Zero Force Regulation
Conclusion and Future Work

11. Related Works Damping control Known environment dynamics
Passivity assumption
Modest performance [S. P. Buerger]
Known environment dynamics
Stability
Optimal performance [R. Johansson; M. Matinfar]
Unknown environment dynamics
Environment identification and modeling
Impedance learning and adaptation
Earlier works: simple tasks
State-of-the-art: reinforcement learning – complex computation [J. Buchli]
Still an open problem

12. Human Learning Skill A new door/ball is an unknown environment.
Human beings learn to adjust their limb impedance iteratively when opening a door/catching a ball.
It is possible to apply impedance learning skill to robot control.
Source:

13. Problem Formulation
The dynamics of a robot arm follow an impedance model:
Consider that the environment is unknown and dynamically changing, and it is described by
Problem: How to determine so that a desired interaction performance is guaranteed?
Define a cost function to measure the interaction performance:

14. Impedance Learning 1. According to gradient-following scheme:
2. The target impedance model:
3. How to obtain ?

15. Impedance Learning Consider the environment dynamics: Choose states
The environment is described as
Define
The environment dynamics become
which is a time-varying system.

16. Betterment Scheme Theorem [S. Arimoto, 1984]:
Consider the linear time-varying system described by
The control input is iteratively updated in the following manner
The betterment process is convergent in the sense that as
Convergence guarantee:

17. Impedance Learning According to the betterment scheme, we have
By employing gradient-following and betterment schemes, impedance parameters are updated as:

18. Experiment Study ATI 6 axis force/torque sensor
Scenario: In each iteration with a period of 18s, the interaction
starts at t = 5s and ends at t = 16s.
Unknown environment: human hand
Initial impedance parameters:
Learning rate:

19. The First Case
Cost function:
Cost Function
Stiffness

20. The First Case
Cost function:
Tracking Error
Interaction Force

21. The Second Case
Cost function:
Cost Function
Stiffness

22. The Second Case
Cost function:
Tracking Error
Interaction Force

23. Discussion
By choosing different cost functions, it is determined that the control objective can be trajectory tracking, integral force tracking or the combination/compromise of these two.
The proposed impedance learning guarantees that the defined cost function becomes smaller iteratively, subject to unknown dynamic environments, and the expected interaction performance has been achieved.

24. Discussion
The advantage of the proposed impedance learning over impedance control with fixed impedance parameters lies in: a modest performance can be obtained if a good set of fixed impedance parameters is predefined (when k = 0), and a better performance can be obtained only with variant
impedance parameters because the environments are dynamically changing.
While impedance parameters have been obtained through iterative learning, how to determine the rest position/desired trajectory in a target impedance model?

25. Outline Introduction Impedance Learning Intention Estimation
Zero Force Regulation
Conclusion and Future Work

26. Scenario: Human-Robot Collaboration
Most tasks that are either too complex to automate or too heavy to manipulate manually are impractical and even impossible to be solely taken by either fully automated robots or human beings.
Robots and human beings may share the same workspace and they
have complementary advantages.

27. Scenario: Human-Robot Collaboration
The robot will act as a load to the human partner if x0 is far away from x.
x0 is supposed to be adapted according to human partner’s motion intention.
Trajectory adaptation is another human beings’ learning skill which can be applied to robot control.

28. Related Works Intention estimation Force regulation
Motion characteristics of the human limb: minimum jerk model [B. Corteville, 2007]
Motion intention state: hidden Markov model [Z. Wang, 2009]
Intentional walking direction: Kalman filter [K. Wakita, 2011]
Robot and human partner: separately analyzed
Force regulation
Force regulation under impedance control [K. Lee, 2008]
Existing works: constant rest position and only stiffness in the environment (human limb)

29. Problem Formulation Human limb model [M. M. Rahman 2002]:
motion intention planned in the central nervous system
Assumption: In a typical collaborative task, the motion intention of the human partner (in each direction) is determined by the interaction force, position and velocity at the interaction point (in the corresponding direction) of the human limb and robot arm.
intention estimation

30. Intention Estimation
NN is employed to estimate human partner’s motion intention, as below:
Define a cost function:
Updating law:

31. Intention Estimation

32. Experiment Study
Scenario: The left wrist of Nancy is moved by human being’s hand.
Note: The actual trajectory of the robot arm cannot be compared with the motion intention directly.
Impedance parameters:
NN setting:

33. Point-to-Point Movement
Joint angle

34. Point-to-Point Movement
External torque

35. Time-Varying Trajectory
Joint angle

36. Time-Varying Trajectory
External torque

37. Discussion
The motion intention of the human partner has been observed by employing the human limb model.
With the proposed trajectory adaptation, the robot arm is able to “actively” follow its human partner.
Human partner and robot are considered to be two separated subsystems, and the performance of the whole coupled interaction system is yet to be rigorously analyzed.

38. Outline Introduction Impedance Learning Intention Estimation
Zero Force Regulation
Conclusion and Future Work

39. Problem Formulation Human limb model [M. M. Rahman 2002]:
Control objective: design x0 in
so that
Three cases:
Point-to-point movement
Periodic trajectory
Arbitrary continuous trajectory

40. Zero Force Regulation Trajectory adaptation: Point-to-point movement:
Periodic trajectory:
Arbitrary continuous trajectory:
Similar to the intention estimation method!

41. Main Result
Theorem:
With the above trajectory adaptations, the following performance of the closed-loop interaction system can be guaranteed:
The robot is able to “actively” follow its human partner.
Lemma:
The above result will not be affected by the transient performance of the inner position control loop, if it is asymptotically stable.

42. Experiment Study
Scenario: The left wrist of Nancy is moved by human being’s hand.
Note: The actual trajectory of the robot arm cannot be compared with the motion intention directly.
Impedance parameters:

43. Point-to-Point Movement
Joint angle

44. Point-to-Point Movement
External torque

45. Periodic Trajectory
Joint angle

46. Periodic Trajectory
External torque

47. Arbitrary Continuous Trajectory
Joint angle

48. Arbitrary Continuous Trajectory
External torque

49. Discussion
Zero force regulation has been investigated for human-robot collaboration, so that the robot is able to “actively” follow its human partner.
Adaptive control has been proposed to deal with the
point-to-point movement, and learning control and NN control have been developed to generate periodic and non-periodic trajectories, respectively.

50. Outline Introduction Impedance Learning Intention Estimation
Zero Force Regulation
Conclusion and Future Work

51. Conclusion
Impedance learning is developed to obtain desired impedance parameters when robots interact with unknown and dynamically changing environments.
Trajectory adaptation is investigated for human-robot collaboration, so that robot is able to “actively” follow its human partner.

52. Future Work
Simultaneous learning/adaptation of impedance and trajectory will be studied.
Interaction control with other sensory information (e.g., image) will be investigated.
The applications of the proposed methods in specific areas (e.g., rehabilitation, tele-operation, and human-robot collaboration) will be considered.
Critical issues in practical implementations, such as time-delay and human factor, will be investigated.

53. References
C. Wang, Y. Li, S. S. Ge and T. H. Lee, “Reference Adaptation for Robots in Physical Interactions with Unknown Environments,” IEEE Transactions on Cybernetics, 2016
Y. Li and S. S. Ge, “Human-Robot Collaboration Based on Motion Intention　Estimation,” IEEE Transactions on Mechatronics, 2014
Y. Li and S. S. Ge, “Impedance Learning for Robot Interacting with Unknown Environments,” IEEE Transactions on Control Systems Technology,　2013
S. S. Ge and Y. Li, “Force Tracking Control for Motion Synchronization in Human-Robot Collaboration,” Robotica, 2013
S. S. Ge, Y. Li and C. Wang, “Impedance Adaptation for Optimal Robot-　Environment Interaction,” International Journal of Control, 2013

54. Published Books
Published Books
Intelligent Control:

55. Published Books
Switched Dynamical
Systems:
Robotic Systems:

56. International Journal of Social Robotics
Editor in Chief:
Shuzhi Sam Ge, NUS
Co-Editor in Chief:
Oussama Khatib, Stanford University, USA
Aims and scope:
Provide a platform for presenting findings and latest developments in social robotics, covering relevant advances in engineering, computing, arts and social sciences.
Provide an overview of the current state of the social robotics scene.
1 Volume(-s) with 5 issue(-s) per annual subscription
IF: 1.407

57. Welcome
2017 International Conference on Social Robotics (ICSR 2017) will be held in Tsukuba City, November 22-24, 2017—first time in Japan

58. Acknowledgement Machine Technology Intelligent Control Robot Sensing
Machine Learning
Cooperation Mechanisms
Affective Sciences
Behavioral Studies
Psychological Impact
Ethical Implications
Interface Design