1. Omnidirectional Vision-Based Formation Control
René Vidal Omid Shakernia Shankar Sastry
Department of EECS, UC Berkeley

2. Motivation Formation control is ubiquitous Safety in numbers
Decreased aerodynamic drag
Higher traffic throughput
Applications in defense, space exploration, etc
Formation control is a problem that has received a lot of attention in the control/robotics community over the past few years. It is a problem that is ubiquitous both in nature and in man made environments.
On the top left, we have a duck being followed by baby ducks in a straight line: “String formation”
On the bottom left we have a school of fish swimming together to reduce the probability of being
eaten by a bigger fish (safety in numbers)
On the middle left, we have geese flying in a V-Formation. This formation is actually quite interesting:
A geese in the formation can fly 71% further than a geese outside the formation, due to reductions
In the aerodynamic drag.
On the right, we have man made applications in
Defense: the commander can fly in the middle and be protected by the rest of the team
Automatic highways: a platoon of cars can increase traffic throughput, while maintaining safety
Space exploration: a cluster of satellites with telescopes can better explore the space by moving
In a formation
Top left duck duckling, line formation
School of fish
Goose, Geese flying in a V-formation:
reduces the drag can fly 71% further than a geese outside the formation
They fly on top of the vortex : when the guy in front moves the flags,
It forms a vortex of air raising up that cushiones
safety in numbers means for example, if a fish is swimming in a large school of other fish he is less likely to get eaten than if he is swimming alone
Commander protection in the middle.
Explore space better by having a cluster with many satelites with telescopes are better for space exploration if they move in a formation

3. Previous Work
Formation control has a very rich literature (short list):
String stability for line formations [Swaroop et.al. TAC96]
Formation can become unstable due to error propagation
Mesh stability of UAVs [Pant et.al. ACC01]
Generalization of string stability to a planar mesh
Input-to-state stability [Tanner et.al ICRA02]
Structure of interconnections and amount of information communicated affects ISS
Feasible formations [Tabuada et.al. ACC01]
Differential geometric conditions on feasibility of formations under kinematic constraints of mobile robots
Vision-based formation control [Das et.al. TAC02]
Leader position estimated by vision; Formation control in task space

4. Our Approach Distributed formation control (no explicit communication)
Formation specified in image plane of each follower
Multi-body motion segmentation to estimate leader position
Followers employ tracking controller in the image plane
Naturally incorporate collision avoidance by exploiting geometry of omni-directional images

5. Outline Omnidirectional vision Distributed formation control
Catadioptric cameras, back-projection ray
Paracatadioptric optical flow equations
Multi-body motion segmentation
Distributed formation control
Leader-follower dynamics in image plane
Feedback linearization control design
Collision avoidance using navigation functions
Experimental results
Motion segmentation of robots in real sequence
Vision-based formation control simulations

6. Catadioptric Camera Catadioptric camera is lens-mirror combination
Single effective focal point
Parabolic mirror, orthographic lens
Hyperbolic camera, perspective lens
Efficiently compute back-projection ray associated with each pixel in image

7. Paracatadioptric Projection Model
Paraboloidal mirror surface:
Projection model:
Back-projection ray for pixel
Paracatadioptric Projection
Perspective Projection

8. Paracatadioptric Optical Flow
Optical flow induced by camera motion of angular velocity and linear velocity
Camera motion restricted to X-Y plane
Paracatadioptric Optical Flow
Perspective Optical Flow

9. Multi-body Motion Segmentation
Optical flows of pixels in frames
live in a 5 dimensional subspace.
Optical flows can be factorized into structure and motion
Given independent motions
Number of independent
motions is obtained as:

10. Motion Segmentation Independent motions lie in orthogonal subspaces
Block diagonal structure of W implies leading singular vector will have the same value for pixels corresponding to same rigid motion
Eigenvector has different value for pixels corresponding to different motions
The largest group of pixels (static background) is flow induced by camera motion
Roots of a polynomial

11. Leader-Follower Dynamics
Kinematic model
Inputs
Leader position in follower’s camera
Write as drift-free control system:
Recover leader velocity using optical flow of background
Paracatadioptric leader-follower dynamics

12. Cartesian coordinates
Control in Image Plane
Cartesian coordinates
Polar coordinates
Controlling in Cartesian coordinates,
leader trajectory intersects circle
Controlling in polar coordinates,
follower mostly rotates
Trajectory passing through
inner circle is a collision

13. Feedback Linearization Control Law in Polar Coordinates
Leader position , desired leader position
Degenerate configurations
due to nonholonomy of mobile robot
Robot can not move sideways instantaneously
due to geometry of catadioptric projection
Corresponds to horizon points at infinity
Feedback Linearization Control Law in Polar Coordinates

14. Distributed Formation Controller
Enhanced collision avoidance using a Navigation Function
Control law incorporating collision avoidance

15. Experimental Results
Multi-body motion segmentation

16. Wedge Formation
Green follows red
Blue follows red

17. String Formation
Green follows red
Blue follows green

18. Conclusions
A framework for distributed formation control in the omni-directional image plane
An algorithm for multi-body motion segmentation in omni-directional images
Future work
Generalize formation
control to UAV dynamics
Hybrid theoretic formation
switching control
Implement on BEAR fleet
of UGVs and UAVs