1. Multi-UAV to UAV Tracking
Brandon Silva, Waqas Sultani, Dr. Mubarak Shah,
University of Central Florida
Introduction
Experiment
Results continued
UAVs (Unmanned Aerial Vehicles) are gaining in popularity for personal, commercial, and government use. Whether it be drone racing, autonomous delivery drones, or security drones; a fast and accurate collision avoidance system is needed as the airways become more cluttered. This project aims to create an end to end network for detection and tracking of multiple UAVs in videos.
The best results obtained used these parameters:
Input resolution of 640x640
Learning rate of with the Adam optimizer
MSE loss, with a constant k=3 multiplied to entire loss function to weight detections in ground truth higher than pixel values of zero
20% test split: 40 videos for training and 10 for testing
Ground truth is red, and predictions are green.
Results
Dataset
The resulting chart shows testing F1-Scores as the model trains over 8 epochs. Different thresholds are used to determine how large of an output value should be to count as a detection, ranging from 0.2 to 0.8.
Detections are counted as correct if their centers are within 5 pixels of the ground truth.
This chart shows the number of false positives detected per epoch on the testing data.
The best F1-Score was 78.46% with a false alarm rate of so far, and the model is improving with more fine tuning. This was calculated with a threshold of 0.2.
Multi-Target Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (Purdue):
50 Videos, 30 FPS, frames
Bounding box annotations around the UAVs
Up to 8 UAVs in a single frame
1920x1080 and 1280x960 resolution
Resized to 640x640
Method
The UAV videos had extensive camera shake due to the nature of recording from a moving platform, so the input video is stabilized, using a Euclidean transformation and the Lucas-Kanade method for obtaining and matching key points between frames. 
2. These stabilized frames are converted to grayscale and fed into a full 2D convolutional neural network that outputs a pixel level segmentation, where values close to 1 are considered UAV detections. This model is based on FoveaNet1.
A middle frame is randomly selected from a video, then 2 consecutive frames before and after are stacked together and passed into the network.
The network is trained on segmentations generated from the bounding boxes by calculating the center coordinate and filling in a circle around the point.
These training labels can either be binary (1’s are detections) or gaussian (where a gaussian filter is applied about the center point). Empirically, the network performs similarly when trained on either label.
2D Conv
Filters: 16
Kernel:
9 x 9
Max Pool
Stride: 2
Kernel:
2 x 2
2D Conv
Filters: 32
Kernel:
7 x 7
2D Conv
Filters: 64
Kernel:
7 x 7
2D Conv
Filters: 256
Kernel:
5 x 5
2D Conv
Filters: 512
Kernel:
5 x 5
2D Conv
Filters: 256
Dropout:
0.5
Kernel:
3 x 3
2D Conv
Filters: 128
Dropout:0.5
Kernel:
3 x 3
2D Conv
Filters: 1
Kernel:
1 x 1
Conclusion
1x320x320
Output
Our method of solving the issue yieled a very reasonable F1-Score. The ridgeline and hills in the videos posed quite an issue with the network as it caused false detections in some of the videos, since all videos were recorded in the same area. Overall though, this end to end lightweight network for detecting these UAVs is shown to be a valid solution to this problem, without relying on optical flow or extensive preprocessing. This will further be enhanced with a novel tracking method to help reduce false alarms and increase the F1-Score.
5x640x640
Frames
References
1. Lalonde, Rodney, et al. “ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, doi: /cvpr