1. Formation Control of Nonholonomic Mobile Robots with Omnidirectional Visual Servoing and Motion Segmentation
René Vidal Omid Shakernia Shankar Sastry
Department of EECS, UC Berkeley

2. Motivation Formation control is ubiquitous Safety in numbers
Decreased aerodynamic drag
Higher traffic throughput
Applications in defense, space exploration, etc
Formation control is a problem that has received a lot of attention in the control/robotics community over the past few years. It is a problem that is ubiquitous both in nature and in man made environments.
On the top left, we have a duck being followed by baby ducks in a straight line: “String formation”
On the bottom left we have a school of fish swimming together to reduce the probability of being
eaten by a bigger fish (safety in numbers)
On the middle left, we have geese flying in a V-Formation. This formation is actually quite interesting:
A geese in the formation can fly 71% further than a geese outside the formation, due to reductions
In the aerodynamic drag.
On the right, we have man made applications in
Defense: the commander can fly in the middle and be protected by the rest of the team
Automatic highways: a platoon of cars can increase traffic throughput, while maintaining safety
Space exploration: a cluster of satellites with telescopes can better explore the space by moving
In a formation
Top left duck duckling, line formation
School of fish
Goose, Geese flying in a V-formation:
reduces the drag can fly 71% further than a geese outside the formation
They fly on top of the vortex : when the guy in front moves the flags,
It forms a vortex of air raising up that cushiones
safety in numbers means for example, if a fish is swimming in a large school of other fish he is less likely to get eaten than if he is swimming alone
Commander protection in the middle.
Explore space better by having a cluster with many satelites with telescopes are better for space exploration if they move in a formation

3. Previous Work
Formation control has a very rich literature (short list):
String stability for line formations [Swaroop et.al. TAC96]
Formation can become unstable due to error propagation
Mesh stability of UAVs [Pant et.al. ACC01]
Generalization of string stability to a planar mesh
Input-to-state stability [Tanner et.al ICRA02]
Structure of interconnections and amount of information communicated affects ISS
Feasible formations [Tabuada et.al. ACC01]
Differential geometric conditions on feasibility of formations under kinematic constraints of mobile robots
Vision-based formation control [Das et.al. TAC02]
Leader position estimated by vision; Formation control in task space

4. Our Approach Distributed formation control (no explicit communication)
Formation specified in image plane of each follower
Multi-body motion segmentation to estimate leader position
Followers employ tracking controller in the image plane
Naturally incorporate collision avoidance by exploiting geometry of omni-directional images

5. Outline Omnidirectional vision Distributed formation control
Central panoramic cameras, back-projection ray
Central panoramic optical flow equations
Multi-body motion segmentation
Distributed formation control
Leader-follower dynamics in image plane
Feedback linearization control design
Collision avoidance using navigation functions
Experimental results
Motion segmentation of robots in real sequence
Vision-based formation control simulations

6. Central Panoramic Camera
Catadioptric camera is lens-mirror combination
Central panoramic: single effective focal point
Parabolic mirror, orthographic lens
Hyperbolic camera, perspective lens
Efficiently compute back-projection ray associated with each pixel in image

7. Central Panoramic Optical Flow
Optical flow induced by a planar camera motion with velocities and
Central Panoramic Projection Model
Central Panoramic Optical Flow

8. Central Panoramic Motion Segmentation
Optical flows of pixels in frames
live in a 5 dimensional subspace.
Optical flows can be factorized into structure and motion
Given independent motions
Number of independent
motions is obtained as:

9. Central Panoramic Motion Segmentation
Independent motions lie in orthogonal subspaces
Block diagonal structure of W implies leading singular vector will have the same value for pixels corresponding to same rigid motion
Eigenvector has different value for pixels corresponding to different motions
The largest group of pixels (static background) is flow induced by camera motion
Roots of a polynomial

10. Image Leader-Follower Dynamics
Kinematic model
Inputs
Leader position in follower’s camera
Write as drift-free control system:
Recover leader velocity using optical flow of background
Central panoramic leader-follower dynamics

11. Omnidirectional Visual Servoing
Cartesian coordinates
Polar coordinates
Controlling in Cartesian coordinates,
leader trajectory intersects circle
Controlling in polar coordinates,
follower mostly rotates
Trajectory passing through
inner circle is a collision

12. Omnidirectional Visual Servoing
Leader position , desired leader position
Degenerate configurations
due to nonholonomy of mobile robot
Robot can not move sideways instantaneously
due to geometry of central panoramic cameras
Corresponds to horizon points at infinity
Feedback Linearization Control Law in Polar Coordinates

13. Omnidirectional Visual Servoing
Can avoid degenerate configurations with pseudo-feedback linearizing control law
However, the formation is only Input-to-State Stable (ISS)
Can easily modify control law to achieve collision avoidance by using a Navigation Function

14. Experimental Results
Multi-body motion segmentation

15. Wedge Formation
Green follows red
Blue follows red

16. String Formation
Green follows red
Blue follows green

17. Conclusions
A framework for distributed formation control in the omni-directional image plane
An algorithm for multi-body motion segmentation in omni-directional images
Future work
Generalize formation
control to UAV dynamics
Hybrid theoretic formation
switching control
Implement on BEAR fleet
of UGVs and UAVs