Tracking with CACTuS on Jetson Running a Bayesian multi object tracker on a low power, embedded system School of Information Technology & Mathematical Sciences September 2015 David Webb Supervisor: David Kearney

Object Tracking Overview What is object tracking? Object tracking, also referred to as video tracking, is a sub domain of computer vision, concerned with detecting, correlating and tracing the path of an object(or objects) as it travels, relative to the background, over multiple frames of video (or a sequence of images).

Object Tracking Overview How is object tracking used? Real world applications of object tracking include Security and Surveillance  Loitering  Traffic  Suspicious packages  Theft Business intelligence  People counting

Object Tracking Overview How is object tracking used? Military  Situational awareness Robotics  Object manipulation  Environment awareness Human Computer Interaction  Gesture Recognition  Gaze Tracking

Object Tracking Overview What is involved in object tracking? Detection  Determining the existence of an object at a given location (or pixel) in an image Tracking  Associating the location of an object over multiple images Recognition  Classification of the nature of the object being tracked

Object Tracking Overview Classic detection tracking recognition Classic algorithms suffer from ambiguity in measurements due to noise In the presence of noise, classic trackers make hard and fast assumptions about the nature of the target object This can lead to poor robustness, where the tracker is unable to correctly maintain the track of an object

Object Tracking Overview Competitive Attentional Correlation Tracker using Shape (CACTuS) CACTuS is designed to avoid problems associated with measurements taken in the presence of noise by using a Bayesian approach to propagate uncertainty through the tracking chain.

CACTuS Tracker CACTuS Multiple Object Tracker Shape Estimating Filter (SEF)  Each SEF is capable of tracking and describing one object  Describes an object’s location, velocity and shape  Uses a Bayesian approach to propagate uncertainty by representing each state as a 2D array of probabilities The following operations are performed on each SEF  Prediction  Observation  Update

CACTuS Tracker CACTuS Multiple Object Tracker CACTuS Combines Multiple SEFs  SEFs are compelled by a competitive mechanism to focus on different objects Multiple SEFs in Parallel  The computation of each SEF is independent of the other SEFs, allowing for all SEF’s to be computed in parallel, excluding the competition phase

CACTuS Tracker CACTuS Algorithm Computational Complexity  SEFs perform the prediction phase using 2D convolution operations on several 2D matrices  The complexity of the convolution of a 2D matrix A with 2D matrix B is approximately O(MNmn), where A is M x N and B is m x n

CACTuS Tracker Demonstration

CACTuS Tracker CACTuS Algorithm Current Implementations and Performance  In the testing of the current CACTuS implementation, the following performance figures were observed on our test PC Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 16MATLAB C++ CPU C++ OpenMP C++ OpenMP + SSE PC Specs:Intel Core i7 4790K 4.0GHz 16GB DDR RAM NVIDIA GeForce GTX960 2GB DDR5

NVIDIA Jetson Platform NVIDIA Jetson TK1 DevKit Tegra K1 System on Chip (SoC)  4 Plus 1 Quad Core ARM Cortex A15 2.3GHz CPU  Kepler GPU with 192 CUDA Cores 2 GB memory 16GB eMMc storage Ubuntu 14.04

NVIDIA Jetson Platform Why the Jetson? As the majority of computational time is taken by the convolution operations, and the convolution operation is easily parallelised, it is expected that the GPU of the Jetson platform will provide increased performance due to it’s highly parallel nature The Jetson is also a convenient form factor and conforms to a low power budget of only 10W

Tracking on Jetson with CACTuS Project Goals Create an implementation of CACTuS that compiles and runs on the Jetson platform Accelerate the implementation of CACTuS using CUDA Achieve a minimum of 10 frames per second of processing with 2 or more SEFs on the Jetson platform

Performance Enhancement Using CUDA to increase the performance of CACTuS Performance profiling was used to focus efforts. Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 16MATLAB C++ CPU C++ OpenMP + SSE PC Specs:Intel Core i7 4790K 4.0GHz 16GB DDR RAM NVIDIA GeForce GTX960 2GB DDR5 16C++ CUDA Conv C++ CUDA + OMP C++ CUDA Predict2713

Performance on Jetson Benchmarking the performance of CACTuS on Jetson Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 16MATLAB Desktop C++ CPU Desktop C++ CPU Jetson C++ OMP Jetson C++ CUDA Conv C++ CUDA Predict C++ Hybrid19379 A hybrid CPU and GPU approach performed best

Performance on Jetson By reducing the number of SEFs, performance can be gained, this is a trade off that reduces the number of targets that can be tracked and the overall accuracy and track latency of the tracker Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 4MATLAB Desktop4018 4C++ OMP Desktop3533 4C++ OMP Jetson7745 4C++ CUDA Predict8941 4C++ Hybrid6033

Conclusion The performance of the platform is acceptable for marginal usage as a tracking platform. Achieving 16 frames per second with 4 SEFs, the 10 fps goal has been exceeded. As a side note, running CACTuS with CUDA acceleration on the desktop in under 30ms per frame for 16 SEFs, demonstrates Real Time processing of the synthetic video, which has not been previously achieved with this version of the CACTuS algorithm.