K. KRISHNA, D. CASBEER, P. CHANDLER & M. PACHTER AFRL CONTROL SCIENCE CENTER OF EXCELLENCE MACCCS REVIEW APRIL 19, 2012 UAV Search & Capture of a Moving Ground Target Approved for public release; distribution unlimited; case number: 88ABW

UGS Sensor Range UGS Communication Range Valid Intruder Path Scenario UAV Communication Range BASE

UAV/UGS Framework UAV engaged in search and capture of intruder on a road network Intersections in road instrumented with Unattended Ground Sensors (UGSs) Intruder is a goal oriented random walker UAV has a speed advantage over the intruder Passing intruder triggers UGS and the event is time- stamped and stored in the UGS UAV has no sensing capability Capture occurs when UAV and intruder are at an UGS location at the same time

Manhattan Grid All edges of the grid are of same length Intruder starts at node S and proceeds towards goal nodes marked G UAV also starts at node S and observes red UGS with delay d Intruder dynamics - randomly move North, East or South but cannot retrace path UAV actions - move North, East or South or Loiter at current locationGG G S

System Dynamics

Solution Method Pose the problem as a POMDP  unconventional POMDP since observations give delayed intruder location information with random time delays!  Use observations to compute the current intruder position probabilities Dual control problem  choose UAV action that minimizes the uncertainty associated with intruder location (localization) or choose action that gets the UAV closest to the most probable intruder location (“end game”/ capture)

A few notes Assumptions:  The intruder finishes in finite time and  can not retrace his steps (no cycles). Guaranteed intruder capture is hard because  Incomplete, out-of-sequence, and delayed information (Unconventional POMDP formulation)

Worst Case Guaranteed Capture (Perfect Information) Related to control with uncertainty modeled as bounded sets [Witsenhausen 68, Bertsekas and Rhodes 73] Guaranteed capture is related to reachable sets [Bertsekas and Rhodes 71]

Full Information Scenario From end of finite horizon  Find set of UAV “safe’’ locations that guarantee capture  Proceed backwards in time looking for “safe locations” Under full information, we have a necessary and sufficient condition for guaranteed capture [Bertsekas and Rhodes 71]

Full Information strategy Case 1: Intruder is at a goal  The UAV must be at the same goal location Case 2: Intruder is not at the goal  Safe locations: 1. Capture now OR 2. Ensure that UAV is in a position to capture the intruder in the future regardless of intruder’s moves.

Proceed in a fashion similar to the full information case except using (estimate) set of intruder locations instead of the true (unknown) intruder location. Partial and Delayed observation

Out of sequence Estimation Process

Set of safe UAV locations

Backward (DP) recursion Either the UAV is at the current intruder location leading to immediate capture or the UAV can take action such that it goes to a favorable location in the next time step; leading to eventual capture.

Sufficient condition The condition is however not necessary for guaranteed capture due to the dual control aspect of the problem.

2x2 Grid Example Problem UAV starting location 1 satisfies sufficient condition (see entry 0 in table)

b1b1b1b1 b2b2b2b2 Min-Max Optimal Paths For bottom row delays 1 and 2. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter).

c1c1c1c1 b1b1b1b1 Min-Max Optimal (End Game) For middle row delay 1. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter).

c1c1c1c1 b1b1b1b1 b3b3b3b3 Min-Max Optimal Paths For bottom row delay 3. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter).

c1c1c1c1 b1b1b1b1 c2c2c2c2 For middle row delay 2. Intruder path is orange and corresponding optimal UAV path is in green (dotted circle represents loiter). Min-Max Optimal Paths

c1c1c1c1 b1b1b1b1 cDcDcDcD c D-1 (D-2) steps For middle row delay D>0 - Induction argument (to prove optimality) leading to the “End Game” Min-Max Optimal Paths

c1c1c1c1 b1b1b1b1 bDbDbDbD c D-2 (D-3) steps For bottom row delay D>2 - Induction argument (to prove optimality) leading to the “End Game” Min-Max Optimal Paths

Min-Max optimal Steps to capture CaseNumber stepsNumber columns needed bottom row - delay 1 (b 1 ) 51 middle row - delay 1 (c 1 ) 113 bottom row - delay 2 (b 2 ) 62 middle row - delay 2 (c 2 ) 124 bottom row - delay 3 (b 3 ) 134 middle row - delay 3 (c 3 ) 135

Conclusions Sufficient condition for guaranteed capture in partial information case Currently trying to tighten the gap between necessity and sufficiency  Include possible future observations to reduce the predicted future uncertainty set An induction argument for guaranteed capture on longer horizons (exploit structure in the graph)

Search & Capture Video