Instantaneous Geo-location of Multiple Targets from Monocular Airborne Video

Tracking algorithm overview The OF module is responsible for calculating a differential optical flow, which eliminates the camera motion (airborne imagery) from the motion of the objects as observed by this camera. The second phase is formed by a module that performs the actual tracking of the objects segmented from phase 1 Phase 1 No Phase 2 Read images Yes Confidence ? LowHigh Geo location Star t Found ROI? Fig 3: Algorithm flow chart

Optical flow based target segmentation 1. Extract OF on feature points2. Analyze histogram of OF2. Identify BG flow 3. Subtract BG flow from the entire OF 4. Segment out the dominating target

Tracking module and thread expansion Thread 1Thread 2 The algorithm them processes the region of interest and detects as many features as possible, according to KLT Feature Tracker. If the confidence index drops below a predetermined level, the Phase 2 of the algorithm aborts and Phase 1 is invoked again and the process starts from the beginning. the tracking module is capable of doing multi-tracking task by thread and semaphore coding scheme. Fig 4: Phase2 expanded to two different targets

Tracking demo on ground vehicles

Image plane x Z Y Inertial frame Pt Pn Camera focal Point Target Normal point h Po One shot target localization

x Z Y Inertial frame dt C  t C t I dt I  h s P UAV altitude estimation using a ground feature point Figure 6: Feature points available on the ground Figure 7: Altitude estimation using stereo camera positions.

Particle filtering The performance of the target localization algorithm needs to be refined since it only depends on noisy sensor perception (GPS/IMU). It is a nonlinear Bayesian tracking method which recursively estimates the current state based on the previous states. Given a particle, the main structure of the algorithm is repeating prediction steps and measurement steps based on the current observation

Particle filtering Update step: The measurement is done by comparing the current observation vector with the projected observation vector. The observation vector is defined as, where u and v are the target position in the image coordinate frame, and h is the altitude of the camera. Prediction step: Our target is assumed to obey the following dynamic model.