Bilateral Teleoperation of Multiple Cooperative Robots over Delayed Communication Network: Theory Dongjun Lee Mark W. Spong d-lee@control.csl.uiuc.edu, mspong@uiuc.edu Research partially supported by the Office of Naval Research (N00014-02-1-0011 and N00014-05-1-0186), the National Science Foundation (IIS 02-33314 and CCR 02-09202), and the College of Engineering at the University of Illinois.

Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions Part II: Simulation and Semi-Experiment

Bilateral Teleoperation Motivations Applications: 1. Space Structure Construction/Maintenance - Hubble telescopes, International Space Station,… 2. Remote Construction/Maintenance of Civil Structures - Bridge, Highway, Tall buildings,… 3. Operations in Hazardous Environments - Nuclear plants, Deep water, … Bilateral Teleoperation - Human’s intelligent intervention in uncertain environments Bilateral Teleoperation of Multiple Cooperative Robots Multi-Robot Cooperation - Mechanical strength and dexterity - Robustness and safety

Challenges and Requirements 1. Abstraction - human is able to operate only small DOF simultaneously 2. Secure grasping - no dropping of the grasped object 3. Haptic feedback - crucial for manipulation tasks 4. Interaction safety and stability - stably coupled with humans, objects, and environments

Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

(n=n1+n2+…+nN-dimensional) Dynamics of Master and Multiple Slave Robots Dynamics of a single master (m-DOF) inertia human force Coriolis velocity control Dynamics of multiple slave robots (n1+n2+…+nN-DOF) Stack-up n-DOF product system (n=n1+n2+…+nN-dimensional)

Grasping Shape Function: Holonomic Constraints master’s DOF - m-dim. holonomic constraints on the config. space of slave robots (m < n) - assumed to address the internal formation shape for cooperative grasping - smooth and full-rank Jacobian (i.e. smooth submersion) - overall group motion evolving on m-dim. level sets (submanifold) q1 q2 q3 m-dim. level sets Grasping shape control objective desired (constant) grasping shape

Communication and Control (C&C) Structure - C&C delay between the master and the slaves - Centralized C&C module for multiple slaves - negligible delays among the slaves - workspaces of slaves are close to each other (e.g. cooperative grasping)

Semi-Autonomous Teleoperation Architecture Observation: - secure grasping is of foremost importance for safety - the system cannot be completely free from time-delay, i.e. system performance would be compromised in some aspects Semi-autonomous teleoperation: 1. local grasping control - secure grasping immune to communication-delay - autonomous control would be enough due to simplicity of cooperative grasping control objective 2. delayed bilateral teleoperation - communication-delay effect confined in bilateral teleoperation - sluggish response could be taken care of by intelligent humans - delayed teleoperation is relatively well-studied areas

Energetic Passivity for Safe/Stable Interaction total slave-ports mechanical power master-port - passive with total master/slave mechanical power as supply rate - stable interaction with any E-passive humans[Hogan]/objects/environments

Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

Passive Decomposition of Multiple Slaves Robots behavior of overall group (and grasped object) Locked System Coupling: dropping object!!! internal group coordination (cooperative grasping) Shape System The Passive Decomposition [Lee&Li, CDC03] decouples the locked and shape systems from each other while enforcing passivity - Can achieve tight/secure grasping regardless of overall group behavior - Ensure secure grasping and interaction stability simultaneously

basis of orthogonal space Orthogonal Decomposition w.r.t. Inertia Metric Grasping shape function Locked system velocity vL : parallel w.r.t. the level sets of qE: (behavior of grasped object and total group) Shape system velocity vE : orthogonal complement w.r.t. inertia matrix (cooperative grasping) locked system velocity vL shape system velocity vE Tangent space decomposition basis of kernel of qE basis of orthogonal space

Passive Decomposition of Slave Group Dynamics Original Slave Dynamics Passive Decomposition Decomposed Dynamics - Shape system ((n-m)-DOF) explicitly represents cooperative grasping shape qE(q) - Locked (m-DOF) system describes overall group behavior - Locked and shape dynamics are similar to usual mechanical systems: - ML(q), ME(q) : symmetric and positive-definite - ML(q)-2CL(q,q), ME(q)-2CE(q,q) : skew-symmetric - Coupling is energetically conservative: Passive Decoupling - CLE(q,q) =-CELT(q,q) -> vLTCLE(q,q)qE + qETCELT(q,q)vL=0 - Power and kinetic energy are also decomposed

Energetic Structure of Decomposed Dynamics Original System Decomposed System passive decoupling - We can decouple the shape system (cooperative grasping) and the locked system (overall group) from each other while enforcing passivity - Desired cooperative grasping and overall group behavior can be achieved simultaneously while enforcing interaction stability

Outline 1. Motivations 2. Problem Formulation 3. Passive Decomposition of Slave Robots 4. Control Design 5. Conclusions

Semi-Autonomous Control Decomposed Dynamics Scattering-based teleoperation control for decoupled locked system Local grasping control shape system Passive decoupling Total Slave Control - Adjusting qEd, and PD-gains, fixtureless grasping can be achieved for flexible object - Although dynamics is decoupled, other effects (e.g. inertia of object) can still perturb the shape system via the internal force FE: feedforward cancellation is necessary Grasping Dynamics (Decoupled Shape System) internal force PD/FF-based Control estimate of desired grasping shape Local Grasping Control

Master Robot and Slave Locked System Scattering-Based Teleoperation of Locked System control human/combined external forces Dynamics of Master Robot and Slave Locked System (both are m-DOF) Shape system (locally controlled) Locked System (decoupled) By operating the master robot of manageably small DOF, human operators can tele-control the behavior of the grasped object over the delayed master-slave communication channel while perceiving combined external forces acting on the grasped object and slaves

(Power Decomposition) Scattering-Based Symmetric Teleoperation line impedance (user-specific) Scattering Variables (Power Decomposition) reflected (from comm.) incident (to comm.) Impedance Control (PI-Control) Symmetric Scattering-Based Teleoperation: - scattering communication (to passify comm. delays) and impedance (PI) controls - asymptotic position-error convergence proof with Z=Kv (i.e. matching condition [Stramigioli et al, TRA03]) : so far, only boundedness of position-error has been established. - force reflection in static manipulation (negligible acceleration/velocity)

Conclusions We propose a control framework for bilateral teleoperation of multiple cooperative robots over delayed master-slave comm. channel: - passive decomposition: the decoupled shape (cooperative grasping) and locked (behavior of the grasped object) systems - local grasping control for the shape system: high precision cooperative grasping regardless of human command/comm. delays - scattering-based bilateral teleoperation of the locked system: human can tele-control behavior of the cooperatively grasped object by operating a small-DOF of the master robot, while perceiving combined force on the slaves and the grasped object over the delayed communication channel - enforce energetic passivity: interaction safety and stability are enhanced Part II will present simulation and semi-experiment results.