Pursuit Evasion Games and Multiple View Geometry René Vidal UC Berkeley Part I: Motivation, 10 minutes, 9 slides -> 8 Part II: Previous work, 35 minutes, 21 slides -> 20 Part II: Future work, 15 minutes, 8 slides -> 10

Current Research Robotics and Control Computer Vision Pursuit-Evasion Games (ICRA’01-CDC’01-TRA’02) Vision-Based Landing (ICRA’02) Decidable Classes of DTHS (HSCC’00-CDC’01) Computer Vision Multi-body Structure from Motion (IFAC’02) Multiple View Geometry (sub. IJCV’01) Planar motions with Small Baselines (ECCV’02) Optimal Motion Estimation (ICCV’01) Camera Self-Calibration (ECCV’00)

Pursuit-Evasion Games Jin Kim, David Shim, Cory Sharp, Omid Shakernia Shankar Sastry

Pursuit-Evasion Game Scenario Evade!

PEG: Map Building (Hespanha CDC’99) Given measurements build a probabilistic map Measurement step: model for sensor detection Prediction step: model for evader motion

PEG: Pursuit Policies (Hespanha CDC’99) Greedy Policy Pursuer moves to the reachable cell with the highest probability of having an evader The probability of the capture time being finite is equal to one (Hespanha CDC ’99) The expected value of the capture time is finite (Hespanha CDC ’99) Global-Max Policy Pursuer moves to the reachable cell which is closest to the cell in the whole map with the highest probability of having an evader May not take advantage of multiple pursuers (all the pursuers could move to the same place)

PEG: Control Architecture (ICRA’01) position of evader(s) position of obstacles strategy planner position of pursuers map builder communications network evaders detected obstacles pursuers positions Desired pursuers positions tactical planner trajectory regulation tactical planner & regulation actuator positions [4n] lin. accel. & ang. vel. [6n] inertial [3n] height over terrain [n] obstacles detected evaders detected vehicle-level sensor fusion state of helicopter & height over terrain obstacles detected control signals [4n] agent dynamics actuator encoders INS GPS ultrasonic altimeter vision Exogenous disturbances terrain evader

PEG: Vision System

PEG: Experimental Results (Summer’00)

PEG: Architecture Implementation (CDC’01) Navigation Computer Serial Vision Computer Strategic Planner Helicopter Control GPS: Position INS: Orientation Camera Control Color Tracking UGV Position Estimation Communication UAV Pursuer Map Building Pursuit Policies Communication Runs in Simulink Same for Simulation and Experiments TCP/IP Serial Robot Micro Controller Robot Computer Robot Control DeadReck: Position Compass: Heading Camera Control Color Tracking GPS: Position Communication UGV Pursuer UGV Evader

PEG: Experimental Results (Spring’01)

PEG: Experimental Results (Spring’01) Add simulation video before

Pursuit Policy: Sensing, Intelligence, Speed Greedy Global-max Visibility Region Forward View Omni-directional View Evasion Policy Random Global-min Evader speed Evaluated policies against different vision capabilities Trapezoidal (narrow FOV) vs. OMNI-directional (wide FOV) Both vision systems covered same number of cells Narrow field of view Can see farther into distance Can sweep a larger area simply by rotation Omni-directional (wide angle) Can see in all angles Rotation does not help see more

PEG: Pursuit-Policy v.s. Vision System

PEG: Evader Speed v.s. Intelligence

Vision Based Landing

Landing: (Sharp & Shakernia ICRA’01)

PEG: Summary Proposed a probabilistic framework and a hierarchical control architecture for pursuit-evasion games Implemented architecture with UGVs and UAVs in real-time Sensor fusion, helicopter control, vision-based detection and vision-based landing Evaluated strategies v.s. speed, sensing and intelligence Global-max outperforms greedy in a real scenario Forward view outperforms Omni-view Vision It also works for intelligent evaders

Multiple View Geometry The Multiple View Matrix Yi Ma (UIUC), Jana Kosecka (GMU), Shankar Sastry (UCB)

Multiple View Geometry (MVG)

MVG: Anatomy of cases (state of the art) surface curve line point theory algorithm practice Euclidean affine projective 2 views 3 views 4 views m views algebra geometry optimization

MVG: A need for unification Euclidean surface curve line point 2 views 3 views 4 views m views theory algorithm practice affine projective algebra geometry optimization rank deficiency of Multiple View Matrix

MVG: Formulation Homogeneous coordinates of a 3-D point Homogeneous coordinates of its 2-D image Projection of a 3-D point to an image plane Either Euclidean or Projective

MVG: Classical Approach Given corresponding images of points recover motion, structure and calibration from This set of equations is equivalent to

MVG: The Multiple View Matrix WLOG choose frame 1 as reference Theorem: [Rank deficiency of Multiple View Matrix] (generic) (degenerate)

MVG: Bilinear and Trilinear Constraints Multiple View Matrix implies bilinear constraints Multiple View Matrix implies trilinear constraints Constraints among more than three views are algebraically dependent (quadrilinear in particular)

MGV: Line Features Point Features Line Features

MGV: Planar Features Point Features Line Features Besides multilinear constraints, it simultaneously gives homography:

Multiple View Landing (ICRA’02)

MGV: Multiple View Planar Algorithm Results

Multiple View Matrix: Summary Points M implies bilinear and trilinear constraints Quadrilinear constraints do not exist Lines All indep. constraints are among 3 views Some are trilinear, some are nonlinear Mixed points and lines There are multilinear constraints among 2-3 views There are nonlinear constraints among 3-4 views

Current Work: Multi-Body SFM Geometry of multiple moving objects Given a set of corresponding image points, obtain: Number of evaders Evader to which each point belongs to (segmentation) Depth of each point Motion of each pursuer and evader Orthographic case (Costeira-Kanade’95) Multiple moving points (Shashua-Levin’01)

Current Work: Multi-Body SFM [IFAC’02] Costeira-Kanade in perspective Form optical flow matrices Obtain number of objects Segment the points

Multi-body SFM: Results

Conclusions Computer Vision for Real-Time Control Pursuit Evasion Games Vision Based Landing A new approach to Multiple View Geometry Simple: just linear algebra Unifying: Euclidean, projective, 2 views, 3 views, multiple views, points, lines, curves, surface? New Constraints: There are nonlinear constraints