1. Tracking with CACTuS on Jetson Running a Bayesian multi object tracker on an embedded system School of Information Technology & Mathematical Sciences September 2015 David Webb Supervisor: David Kearney

2. Object Tracking Overview What is object tracking? Object tracking, also referred to as video tracking, is a sub domain of computer vision, concerned with detecting, correlating and tracing the path of an object(or objects) as it travels, relative to the background, over multiple frames of video (or a sequence of images).

3. Object Tracking Overview How is object tracking used? Real world applications of object tracking include Security and Surveillance  Loitering  Traffic  Suspicious packages  Theft Business intelligence  People counting

4. Object Tracking Overview How is object tracking used? Military  Situational awareness Robotics  Object manipulation  Environment awareness Human Computer Interaction  Gesture Recognition  Gaze Tracking

5. Object Tracking Overview What is involved in object tracking? Detection  Determining the existence of an object at a given location (or pixel) in an image Tracking  Associating the location of an object over multiple images Recognition  Classification of the nature of the object being tracked

6. Object Tracking Overview Classic detection tracking recognition Classic algorithms suffer from ambiguity in measurements due to noise In the presence of noise, classic trackers make hard and fast assumptions about the nature of the target object This can lead to poor robustness, where the tracker is unable to correctly maintain the track of an object

7. Object Tracking Overview Competitive Attentional Correlation Tracker using Shape (CACTuS) CACTuS is designed to avoid problems associated with measurements taken in the presence of noise by using a Bayesian approach to propagate uncertainty through the tracking chain.

8. CACTuS Tracker CACTuS Multiple Object Tracker Shape Estimating Filter (SEF)  Each SEF is capable of tracking and describing one object  Describes an object’s location, velocity and shape  Uses a Bayesian approach to propagate uncertainty by representing each state as a 2D array of probabilities The following operations are performed on each SEF  Prediction  Observation  Update

9. CACTuS Tracker CACTuS Multiple Object Tracker CACTuS Combines Multiple SEFs  SEFs are compelled by a competitive mechanism to focus on different objects Multiple SEFs in Parallel  The each phase of each SEF is independent of each other SEF, allowing for SEF’s to be computed in parallel, excluding the competition phase

10. CACTuS Tracker CACTuS Algorithm Computational Complexity  SEFs perform the prediction phase using 2D convolution operations on several 2D matrices  The complexity of the convolution of a 2D matrix A with 2D matrix B is approximately O(MNmn), where A is M x N and B is m x n

11. CACTuS Tracker Demonstration

12. CACTuS Tracker CACTuS Algorithm Current Implementations and Performance  In the testing of the current CACTuS implementation, the following performance figures were observed on our test PC Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 16MATLAB15069 16C++ CPU707572 16C++ OpenMP139107 16C++ OpenMP + SSE129107 PC Specs:Intel Core i7 4790K 4.0GHz 16GB DDR3-1600 RAM NVIDIA GeForce GTX960 2GB DDR5

13. NVIDIA Jetson Platform NVIDIA Jetson TK1 DevKit Tegra K1 System on Chip (SoC)  4 Plus 1 Quad Core ARM Cortex A15 CPU  Kepler GPU with 192 CUDA Cores 2 GB memory 16GB eMMc storage Runs custom Ubuntu 14.04

14. NVIDIA Jetson Platform Why the Jetson? As the majority of computational time is taken by the convolution operations, and the convolution operation is easily parallelised, it is expected that the GPU of the Jetson platform will provide increased performance due to it’s highly parallel nature The Jetson is also a convenient form factor and conforms to a low power budget of only 10W

15. Tracking on Jetson with CACTuS Project Goals Create an implementation of CACTuS that compiles and runs on the Jetson platform Accelerate the implementation of CACTuS using CUDA Achieve a minimum of 10 frames per second of processing with 2 or more SEFs on the Jetson platform

16. Performance Enhancement Using CUDA to increase the performance of CACTuS Profiling was used along side existing timing code to gauge the impact of specific functions. Number of SEFs ImplementationTime per frame (ms) (average) Prediction phase time per frame (ms) 16MATLAB15069 16C++ 16C++ OpenMP + SSE129107 16C++ CUDA Conv2915 16C++ CUDA Predict2713 PC Specs:Intel Core i7 4790K 4.0GHz 16GB DDR3-1600 RAM NVIDIA GeForce GTX960 2GB DDR5

17. Jet