1. Bilateral Teleoperation of Multiple Cooperative Robots over
Delayed Communication Network: Theory
Dongjun Lee
Mark W. Spong
Research partially supported by the Office of Naval Research (N and N ), the National Science Foundation (IIS and CCR ), and the College of Engineering at the University of Illinois.

2. Outline 1. Motivations 2. Problem Formulation
3. Passive Decomposition of Slave Robots
4. Control Design
5. Conclusions
Part II: Simulation and Semi-Experiment

3. Bilateral Teleoperation
Motivations
Applications:
1. Space Structure Construction/Maintenance
- Hubble telescopes, International Space Station,…
2. Remote Construction/Maintenance of Civil Structures
- Bridge, Highway, Tall buildings,…
3. Operations in Hazardous Environments
- Nuclear plants, Deep water, …
Bilateral Teleoperation
- Human’s intelligent intervention
in uncertain environments
Bilateral Teleoperation
of Multiple
Cooperative Robots
Multi-Robot Cooperation
- Mechanical strength and dexterity
- Robustness and safety

4. Challenges and Requirements
1. Abstraction
- human is able to operate only small DOF simultaneously
2. Secure grasping
- no dropping of the grasped object
3. Haptic feedback
- crucial for manipulation tasks
4. Interaction safety and stability
- stably coupled with humans, objects, and environments

5. Outline 1. Motivations 2. Problem Formulation
3. Passive Decomposition of Slave Robots
4. Control Design
5. Conclusions

6. (n=n1+n2+…+nN-dimensional)
Dynamics of Master and Multiple Slave Robots
Dynamics of
a single master
(m-DOF)
inertia
human
force
Coriolis
velocity
control
Dynamics of multiple
slave robots
(n1+n2+…+nN-DOF)
Stack-up
n-DOF product system
(n=n1+n2+…+nN-dimensional)

7. Grasping Shape Function: Holonomic Constraints
master’s DOF
- m-dim. holonomic constraints on the config. space of slave robots (m < n)
- assumed to address the internal formation shape for cooperative grasping
- smooth and full-rank Jacobian (i.e. smooth submersion)
- overall group motion evolving on m-dim. level sets (submanifold) q1 q2 q3
m-dim.
level sets
Grasping shape control objective
desired (constant)
grasping shape

8. Communication and Control (C&C) Structure
- C&C delay between the master and the slaves
- Centralized C&C module for multiple slaves
- negligible delays among the slaves
- workspaces of slaves are close to each other (e.g. cooperative grasping)

9. Semi-Autonomous Teleoperation Architecture
Observation:
- secure grasping is of foremost importance for safety
- the system cannot be completely free from time-delay,
i.e. system performance would be compromised in some aspects
Semi-autonomous teleoperation:
1. local grasping control
- secure grasping immune to communication-delay
- autonomous control would be enough due to simplicity of
cooperative grasping control objective
2. delayed bilateral teleoperation
- communication-delay effect confined in bilateral teleoperation
- sluggish response could be taken care of by intelligent humans
- delayed teleoperation is relatively well-studied areas

10. Energetic Passivity for Safe/Stable Interaction
total slave-ports
mechanical power
master-port
- passive with total master/slave mechanical power as supply rate
- stable interaction with any E-passive humans[Hogan]/objects/environments

11. Outline 1. Motivations 2. Problem Formulation
3. Passive Decomposition of Slave Robots
4. Control Design
5. Conclusions

12. Passive Decomposition of Multiple Slaves Robots
behavior of overall group
(and grasped object) Locked System
Coupling:
dropping object!!!
internal group coordination
(cooperative grasping)
Shape System
The Passive Decomposition [Lee&Li, CDC03] decouples the locked and
shape systems from each other while enforcing passivity
- Can achieve tight/secure grasping regardless of overall group behavior
- Ensure secure grasping and interaction stability simultaneously

13. basis of orthogonal space
Orthogonal Decomposition w.r.t. Inertia Metric
Grasping shape function
Locked system velocity vL :
parallel w.r.t. the level sets of qE:
(behavior of grasped object and total group)
Shape system velocity vE :
orthogonal complement w.r.t. inertia matrix
(cooperative grasping)
locked system
velocity vL
shape system
velocity vE
Tangent space
decomposition
basis of kernel of qE
basis of orthogonal space

14. Passive Decomposition of Slave Group Dynamics
Original Slave
Dynamics
Passive
Decomposition
Decomposed
Dynamics
- Shape system ((n-m)-DOF) explicitly represents cooperative grasping shape qE(q)
- Locked (m-DOF) system describes overall group behavior
- Locked and shape dynamics are similar to usual mechanical systems:
- ML(q), ME(q) : symmetric and positive-definite
- ML(q)-2CL(q,q), ME(q)-2CE(q,q) : skew-symmetric
- Coupling is energetically conservative: Passive Decoupling
- CLE(q,q) =-CELT(q,q) -> vLTCLE(q,q)qE + qETCELT(q,q)vL=0
- Power and kinetic energy are also decomposed

15. Energetic Structure of Decomposed Dynamics
Original System
Decomposed System
passive
decoupling
- We can decouple the shape system (cooperative grasping) and the locked system (overall group) from each other while enforcing passivity
- Desired cooperative grasping and overall group behavior can be achieved
simultaneously while enforcing interaction stability

16. Outline 1. Motivations 2. Problem Formulation
3. Passive Decomposition of Slave Robots
4. Control Design
5. Conclusions

17. Semi-Autonomous Control
Decomposed
Dynamics
Scattering-based teleoperation
control for decoupled
locked system
Local grasping control
shape system
Passive decoupling
Total Slave Control
- Adjusting qEd, and PD-gains, fixtureless grasping can be achieved for flexible object
- Although dynamics is decoupled, other effects (e.g. inertia of object) can still perturb
the shape system via the internal force FE: feedforward cancellation is necessary
Grasping Dynamics
(Decoupled Shape System)
internal force
PD/FF-based Control
estimate of
desired grasping shape
Local Grasping Control

18. Master Robot and Slave Locked System
Scattering-Based Teleoperation of Locked System
control
human/combined
external forces
Dynamics of
Master Robot and Slave Locked System
(both are m-DOF)
Shape system
(locally controlled)
Locked
System
(decoupled)
By operating the master robot of manageably small DOF, human operators
can tele-control the behavior of the grasped object over the delayed
master-slave communication channel while perceiving combined external
forces acting on the grasped object and slaves

19. (Power Decomposition)
Scattering-Based Symmetric Teleoperation
line impedance
(user-specific)
Scattering Variables
(Power Decomposition)
reflected (from comm.)
incident (to comm.)
Impedance Control
(PI-Control)
Symmetric Scattering-Based Teleoperation:
- scattering communication (to passify comm. delays) and impedance (PI) controls
- asymptotic position-error convergence proof with Z=Kv
(i.e. matching condition [Stramigioli et al, TRA03])
: so far, only boundedness of position-error has been established.
- force reflection in static manipulation (negligible acceleration/velocity)

20. Conclusions
We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel:
- passive decomposition: the decoupled shape (cooperative grasping)
and locked (behavior of the grasped object) systems
- local grasping control for the shape system: high precision
cooperative grasping regardless of human command/comm. delays
- scattering-based bilateral teleoperation of the locked system:
human can tele-control behavior of the cooperatively grasped
object by operating a small-DOF of the master robot, while
perceiving combined force on the slaves and the grasped object
over the delayed communication channel
- enforce energetic passivity: interaction safety and stability are enhanced
Part II will present simulation and semi-experiment results.