Multi-UAV to UAV Tracking Brandon Silva, Waqas Sultani, Dr. Mubarak Shah, University of Central Florida Introduction Experiment Results continued UAVs (Unmanned Aerial Vehicles) are gaining in popularity for personal, commercial, and government use. Whether it be drone racing, autonomous delivery drones, or security drones; a fast and accurate collision avoidance system is needed as the airways become more cluttered. This project aims to create an end to end network for detection and tracking of multiple UAVs in videos. The best results obtained used these parameters: Input resolution of 640x640 Learning rate of 0.00006 with the Adam optimizer MSE loss, with a constant k=3 multiplied to entire loss function to weight detections in ground truth higher than pixel values of zero 20% test split: 40 videos for training and 10 for testing Ground truth is red, and predictions are green. Results Dataset The resulting chart shows testing F1-Scores as the model trains over 8 epochs. Different thresholds are used to determine how large of an output value should be to count as a detection, ranging from 0.2 to 0.8. Detections are counted as correct if their centers are within 5 pixels of the ground truth. This chart shows the number of false positives detected per epoch on the testing data. The best F1-Score was 78.46% with a false alarm rate of 0.2323. so far, and the model is improving with more fine tuning. This was calculated with a threshold of 0.2. Multi-Target Detection and Tracking from a Single Camera in Unmanned Aerial Vehicles (Purdue): 50 Videos, 30 FPS, 70250 frames Bounding box annotations around the UAVs Up to 8 UAVs in a single frame 1920x1080 and 1280x960 resolution Resized to 640x640 Method The UAV videos had extensive camera shake due to the nature of recording from a moving platform, so the input video is stabilized, using a Euclidean transformation and the Lucas-Kanade method for obtaining and matching key points between frames.  2. These stabilized frames are converted to grayscale and fed into a full 2D convolutional neural network that outputs a pixel level segmentation, where values close to 1 are considered UAV detections. This model is based on FoveaNet1. A middle frame is randomly selected from a video, then 2 consecutive frames before and after are stacked together and passed into the network. The network is trained on segmentations generated from the bounding boxes by calculating the center coordinate and filling in a circle around the point. These training labels can either be binary (1’s are detections) or gaussian (where a gaussian filter is applied about the center point). Empirically, the network performs similarly when trained on either label. 2D Conv Filters: 16 Kernel: 9 x 9 Max Pool Stride: 2 Kernel: 2 x 2 2D Conv Filters: 32 Kernel: 7 x 7 2D Conv Filters: 64 Kernel: 7 x 7 2D Conv Filters: 256 Kernel: 5 x 5 2D Conv Filters: 512 Kernel: 5 x 5 2D Conv Filters: 256 Dropout: 0.5 Kernel: 3 x 3 2D Conv Filters: 128 Dropout:0.5 Kernel: 3 x 3 2D Conv Filters: 1 Kernel: 1 x 1 Conclusion 1x320x320 Output Our method of solving the issue yieled a very reasonable F1-Score. The ridgeline and hills in the videos posed quite an issue with the network as it caused false detections in some of the videos, since all videos were recorded in the same area. Overall though, this end to end lightweight network for detecting these UAVs is shown to be a valid solution to this problem, without relying on optical flow or extensive preprocessing. This will further be enhanced with a novel tracking method to help reduce false alarms and increase the F1-Score. 5x640x640 Frames References 1. Lalonde, Rodney, et al. “ClusterNet: Detecting Small Objects in Large Scenes by Exploiting Spatio-Temporal Information.” 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018, doi:10.1109/cvpr.2018.00421.