Pursuit-Evasion Games with UGVs and UAVs René Vidal R. Shahid, C. Sharp, O. Shakernia, J. Kim, S. Sastry University of California at Berkeley 05/25/01

Introduction: BErkeley Aerial Robot (BEAR)

Introduction: BEAR Research Challenges Probabilistic map building Coordinated multi-agent operation Networking and intelligent data sharing Path planning Identification of vehicle dynamics and control Sensor integration Vision system Helicopter Control Visual Based Landing of a Helicopter Pursuit Evasion Games 3. Pursuit Evasion Games

Outline Probabilistic Pursuit Evasion Games Hierarchical Control Architecture Implementation Experimental Results Conclusions and Current Research

Pursuit Evasion Games: Scenario Evade!

Pursuit Evasion Games: Scenario Rules of the Game Pursuers can only move to adjacent empty cells Evader moves randomly to adjacent cells Pursuers have perfect knowledge of current location Pursuers can recognize each evader individually Sensor model: false positives (p) and negatives (q) for evader and obstacle detection Probabilistic Approach Conventional approaches: build a map first then play the game Here map building and pursuit evasion games are combined into a single probabilistic framework (Hespanha et.al CDC’99) Objective: capture all the evaders in minimum time

Pursuit Evasion Games: Map Building At each t, + y(t) ={v(t),e(t),o(t)} model for sensor 1. Measurement step 2. Prediction step model for evader’s motion

Pursuit Evasion Games: Pursuit Policies Greedy Policy Pursuer moves to the adjacent cell with the highest probability of having an evader over all maps The probability of the capture time being finite is equal to one (Hespanha CDC ’99) The expected value of the capture time is finite (Hespanha CDC ’99) Global-Max Policy Pursuer moves towards the place with the highest probability of having an evader in the map May not take advantage of multiple pursuers (may move to the same place)

Hierarchical System Architecture position of evader(s) position of obstacles strategy planner position of pursuers map builder communications network evaders detected obstacles pursuers positions Desired pursuers positions tactical planner trajectory regulation tactical planner & regulation actuator positions [4n] lin. accel. & ang. vel. [6n] inertial [3n] height over terrain [n] obstacles detected evaders detected vehicle-level sensor fusion state of helicopter & height over terrain obstacles detected control signals [4n] agent dynamics actuator encoders INS GPS ultrasonic altimeter vision Exogenous disturbances terrain evader

Agent Architecture Segments the control of each agent into different layers of abstraction The same high-level control strategies can be applied to all agents Strategic Planner Mission planning, high level control, communication Tactical Planner Trajectory planning, Obstacle Avoidance Regulation Low level control and sensing

Implementation: Unmanned Ground Vehicles Pioneer 2AT Ground Robots Micro-controller Running P2OS Regulation and control Sensing: dead-reckoning, compass and sonar Tactical Planner: obstacle avoidance Vision Computer: Pentium 266 running Linux OS Visual estimation of obstacles and evaders, camera control Sensor Fusion: GPS Communication

Implementation: Unmanned Aerial Vehicles Yamaha R-50 helicopter Navigation Computer Pentium 233, running QNX OS Regulation and Low Level Control: attitude, hover, etc. Sensor Fusion: GPS,INS, ultrasonic sensors, inertial sensors, compass Vision Computer Pentium 233 running Linux OS Tactical Planner: Vehicle Control Language (VCL) Visual estimation of obstacles and evaders, camera control Serial communication to receive state of the helicopter

Implementation: Vision System Motion Model Image Model Camera position and orientation Helicopter orientation relative to ground Camera orientation relative to helicopter Camera calibration Width, height, zoom Robot position estimate

Implementation: Vision System Hardware Onboard Computer: Linux Sony pan/tilt/zoom camera PXC200 frame grabber Camera Control Software in Linux Send PTZ Commands Receive Camera State ACTS System Captures and processes video 32 color channels 10 blobs per channel Extract color information and sends it to a TCP socket Number of blobs, Size and position of each blob

Implementation: Architecture Strategic Planner Navigation Computer Serial Vision Computer Helicopter Control GPS: Position INS: Orientation Camera Control Color Tracking UGV Position Estimation Communication Map Building Pursuit Policies Communication Runs in Simulink Same for Simulation and Experiments UAV Pursuer TCP/IP Serial Robot Micro Controller Robot Computer Robot Control DeadReck: Position Compass: Heading Camera Control Color Tracking GPS: Position Communication UGV Pursuer UGV Evader

Experimental Results: Pursuit Evasion Games with 1UAV and 2 UGVs (Summer’ 00)

Experimental Results: Pursuit Evasion Games with 4UGVs and 1 UAV (Spring’ 01)

Conclusions and Current Research Combined map building and pursuit evasion games in a single probabilistic framework that guarantees finite capture time Proposed an architecture for real time control of multiple agents The proposed approach has been successfully applied to the control of multiple agents for the pursuit evasion scenario Experimental results confirm theoretical results Current Research Strategy Planner: Montecarlo based learning of pursuit policies Tactical Planner: Collision avoidance and UAV path planning Sensing: Multiview motion estimation of multiple evaders Communication:

robotics.eecs.berkeley.edu/bear robotics.eecs.berkeley.edu/~rvidal The BEAR Project robotics.eecs.berkeley.edu/bear robotics.eecs.berkeley.edu/~rvidal

Experimental Results: Evaluation of Policies

Experimental Results: Evaluation of Policies