1. Pursuit-Evasion Games with UGVs and UAVs
René Vidal
R. Shahid, C. Sharp, O. Shakernia, J. Kim, S. Sastry
University of California at Berkeley
05/25/01

2. Introduction: BErkeley Aerial Robot (BEAR)

3. Introduction: BEAR Research Challenges
Probabilistic map building
Coordinated multi-agent operation
Networking and intelligent data sharing
Path planning
Identification of vehicle dynamics and control
Sensor integration
Vision system
Helicopter Control
Visual Based Landing of a Helicopter
Pursuit Evasion Games
3. Pursuit Evasion Games

4. Outline Probabilistic Pursuit Evasion Games
Hierarchical Control Architecture
Implementation
Experimental Results
Conclusions and Current Research

5. Pursuit Evasion Games: Scenario
Evade!

6. Pursuit Evasion Games: Scenario
Rules of the Game
Pursuers can only move to adjacent empty cells
Evader moves randomly to adjacent cells
Pursuers have perfect knowledge of current location
Pursuers can recognize each evader individually
Sensor model: false positives (p) and negatives (q) for evader and obstacle detection
Probabilistic Approach
Conventional approaches: build a map first then play the game
Here map building and pursuit evasion games are combined into a single probabilistic framework (Hespanha et.al CDC’99)
Objective: capture all the evaders in minimum time

7. Pursuit Evasion Games: Map Building
At each t,
+ y(t) ={v(t),e(t),o(t)}
model for sensor
1. Measurement step
2. Prediction step
model for evader’s motion

8. Pursuit Evasion Games: Pursuit Policies
Greedy Policy
Pursuer moves to the adjacent cell with the highest probability of having an evader over all maps
The probability of the capture time being finite is equal to one (Hespanha CDC ’99)
The expected value of the capture time is finite (Hespanha CDC ’99)
Global-Max Policy
Pursuer moves towards the place with the highest probability of having an evader in the map
May not take advantage of multiple pursuers (may move to the same place)

9. Hierarchical System Architecture
position of evader(s)
position of obstacles
strategy planner
position of pursuers
map builder
communications network
evaders
detected
obstacles
pursuers
positions
Desired pursuers
positions
tactical
planner
trajectory
regulation
tactical planner & regulation
actuator
positions
[4n]
lin. accel.
& ang. vel.
[6n]
inertial
[3n]
height over terrain [n]
obstacles detected
evaders detected
vehicle-level sensor fusion
state of helicopter & height over terrain
obstacles detected
control signals
[4n]
agent dynamics
actuator
encoders
INS
GPS
ultrasonic altimeter
vision
Exogenous
disturbances
terrain
evader

10. Agent Architecture
Segments the control of each agent into different layers of abstraction
The same high-level control strategies can be applied to all agents
Strategic Planner
Mission planning, high level control, communication
Tactical Planner
Trajectory planning, Obstacle Avoidance
Regulation
Low level control and sensing

11. Implementation: Unmanned Ground Vehicles
Pioneer 2AT Ground Robots
Micro-controller
Running P2OS
Regulation and control
Sensing: dead-reckoning, compass and sonar
Tactical Planner: obstacle avoidance
Vision Computer:
Pentium 266 running Linux OS
Visual estimation of obstacles and evaders, camera control
Sensor Fusion: GPS
Communication

12. Implementation: Unmanned Aerial Vehicles
Yamaha R-50 helicopter
Navigation Computer
Pentium 233, running QNX OS
Regulation and Low Level Control: attitude, hover, etc.
Sensor Fusion: GPS,INS, ultrasonic sensors, inertial sensors, compass
Vision Computer
Pentium 233 running Linux OS
Tactical Planner: Vehicle Control Language (VCL)
Visual estimation of obstacles and evaders, camera control
Serial communication to receive state of the helicopter

13. Implementation: Vision System
Motion Model
Image Model
Camera position and orientation
Helicopter orientation relative to ground
Camera orientation relative to helicopter
Camera calibration
Width, height, zoom
Robot position estimate

14. Implementation: Vision System
Hardware
Onboard Computer: Linux
Sony pan/tilt/zoom camera
PXC200 frame grabber
Camera Control
Software in Linux
Send PTZ Commands
Receive Camera State
ACTS System
Captures and processes video
32 color channels
10 blobs per channel
Extract color information and sends it to a TCP socket
Number of blobs,
Size and position of each blob

15. Implementation: Architecture
Strategic Planner
Navigation Computer
Serial
Vision Computer
Helicopter Control
GPS: Position
INS: Orientation
Camera Control
Color Tracking
UGV Position Estimation
Communication
Map Building
Pursuit Policies
Communication
Runs in Simulink
Same for Simulation and Experiments
UAV Pursuer
TCP/IP
Serial
Robot Micro Controller
Robot Computer
Robot Control
DeadReck: Position
Compass: Heading
Camera Control
Color Tracking
GPS: Position
Communication
UGV Pursuer
UGV Evader

16. Experimental Results: Pursuit Evasion Games with 1UAV and 2 UGVs (Summer’ 00)

17. Experimental Results: Pursuit Evasion Games with 4UGVs and 1 UAV (Spring’ 01)

18. Conclusions and Current Research
Combined map building and pursuit evasion games in a single probabilistic framework that guarantees finite capture time
Proposed an architecture for real time control of multiple agents
The proposed approach has been successfully applied to the control of multiple agents for the pursuit evasion scenario
Experimental results confirm theoretical results
Current Research
Strategy Planner: Montecarlo based learning of pursuit policies
Tactical Planner: Collision avoidance and UAV path planning
Sensing: Multiview motion estimation of multiple evaders
Communication:

19. robotics.eecs.berkeley.edu/bear robotics.eecs.berkeley.edu/~rvidal
The BEAR Project
robotics.eecs.berkeley.edu/bear
robotics.eecs.berkeley.edu/~rvidal

20. Experimental Results: Evaluation of Policies

21. Experimental Results: Evaluation of Policies