REU Program 2019 Week 6 Alex Ruiz Jyoti Kini

Outline Weak-supervision based Multi-Object Tracking Research Papers: Neighbourhood Consensus Networks MOT16: A Benchmark for Multi-Object Tracking GMMCP MOT17 Dataset Pre-processing Module Fine-tuning Model Complete Tracking Trajectory of an Object Outline

Weak-supervision based Multi-Object Tracking Design a MOT system with weakly supervised learning algorithm Pixel-level object tracking in a sequence of frames Intend to find reliable dense correspondences between a pair of images Enhance the key point matches algorithm to get better tracklet association

Neighbourhood Consensus Networks

MOT16: A Benchmark for Multi-Object Tracking MOT16 Benchmark with a fair evaluation metric 14 videos with different viewpoints, camera motions and weather conditions Most of them are filmed in high resolution Pedestrians, vehicles, and objects are annotated

Evaluation Measures Name Description True Positive TP The actual annotated target (Inside Distance) False Positive/False Alarm FP The number of false positive (Outside Distance) False Negative FN Number of missed targets False Alarms per Frame FAF The Number of False Alarms per Frame Identity Switches ID Sw Number of identity switches Mostly Tracked MT Ratio of GT tracked for at least 80% of its life span Mostly Lost ML Ratio of GT tracked less than 20% of its life span Multiple Object Tracking Accuracy MOTA Combines three error sources: FP, FN and ID Sw Multiple Object Tracking Precision MOTP The misalignment between the annotated and the predicted bounding boxes

GMMCP Tracker: Globally Optimal Generalized Maximum Multi Clique Problem for Multiple Object Tracking Enhanced data association where the input is a complete k-partite graph Find K cliques (tracks) by selecting K nodes (tracklets) from every clusters Uses Aggregated Dummy Nodes for handling occlusion Low-level tracklets -> Mid-level tracklets -> Full Trajectories

MOT17 Dataset Pre-processing Module

Fine-tuning Model Using the Pf-Pascal and MOT17 Datasets: 20 Epochs/400 Image Size/16 Batch Size Only Using MOT17 Dataset (Upscaled Version Model): 5 Epochs/640 Image Size/4 Batch Size

Fine-tuning Model Using the Pf-Pascal and MOT17 Datasets: 20 Epochs/400 Image Size/16 Batch Size

Pf-Pascal (20 Epochs/400 Image Size/16 Batch Size)

Mot17 (20 Epochs/400 Image Size/16 Batch Size)

Fine-tuning Model Only Using MOT17 Dataset (Upscaled Version Model): 5 Epochs/640 Image Size/4 Batch Size

5 Epochs/640 Image Size/4 Batch Size

Complete Tracking Trajectory of an Object

Thank You!

Rocco I, Cimpoi M, Arandjelović R, Torii A, Pajdla T, Sivic J Rocco I, Cimpoi M, Arandjelović R, Torii A, Pajdla T, Sivic J. Neighbourhood Consensus Networks. InAdvances in Neural Information Processing Systems 2018 (pp. 1651-1662). Milan, A., Leal-Taixé, L., Reid, I., Roth, S. & Schindler, K. MOT16: A Benchmark for Multi-Object Tracking. arXiv:1603.00831 [cs], 2016., (arXiv: 1603.00831). Dehghan A, Modiri Assari S, Shah M. Gmmcp tracker: Globally optimal generalized maximum multi clique problem for multiple object tracking. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition 2015 (pp. 4091-4099). References