Loris Bazzani*, Marco Cristani*†, Vittorio Murino*† Speaker: Diego Tosato* *Computer Science Department, University of Verona, Italy †Istituto Italiano di Tecnologia (IIT), Genova, Italy Collaborative Particle Filters for Group Tracking This research is founded by the EU-Project FP7 SAMURAI,grant FP7-SEC No

Analysis of the problem (1) 2 Multi-Target Tracking: Estimate the trajectories of objects of interest, keeping their identification over the time Well-investigated problem State-of-the-art methods are very effective and efficient Multi-Group Tracking: Estimate the trajectories of the groups of objects, keeping their identification over the time Not Well-investigated problem Few methods in the State of the art

Analysis of the problem (2) Why it is a hard task Methods for multi-target tracking fails Groups are highly structured entity Hard to model the complex dynamics Strong appearance variations over the time Intra- and inter-group occlusions phenomena What is a group? Motivation: Highlighting social behaviors among individuals 3

Outline 4 Overview of the proposed method Particle Filtering Multi-Object Tracking (MOT) Multi-Group Tracking (MGT) Collaborative Particle Filters (Co-PF) Results Conclusions

Overview of the proposed method 5 Two separate particle filters Multi-object tracker (MOT) models each individual separately Multi-group tracker (MGT) focuses on groups as atomic entities Coupling of the two processes in a formal probabilistic framework Co-PF Model

Particle Filtering for Target Tracking Recursively calculating the posterior distribution is defined by The dynamical model The observation model The first frame distribution Monte Carlo approximation by a set of weighted particles 6

Multi-Object Tracking Extension to Multi-target Hybrid Joint-Separable (HJS) Filter [Lanz 2006] Approximation to decompose the joint state space in single state spaces HJS is efficient and models the interactions among targets We “just” need to define Single-object dynamical and the single-object observation models 7 [Lanz 2006] O. Lanz, “Approximate bayesian multibody tracking,” IEEETPAMI, 28(9):1436–1449, 2006.

Multi-Group Tracking Use HJS filter State of the group: Gaussian model Observation model Projection of the cylinder into the image Histogram-based feature as descriptor Dynamical model : linear motion, perturbed by Gaussian noise : Gaussian perturbation of its principal axes, i.e., by varying its eigenvalues and eigenvectors 8

Collaborative Particle Filters Inject the information collected by the MOT into the MGT Marginalization over the MOT state space After some approximations, we end up with It is a combination of MOT and MGT posteriors at time (t-1) 9 MOT posterior at time (t-1) MGT posterior at time (t-1) Linking probability

Collaborative Particle Filters The linking probability connect the MGT state space to the MOT state space Approximation through the Mixed-memory Markov Process Linking likelihood is decomposed in three components Appearance similarity: distance between color histograms Dynamics consistency: same direction between group and person Group membership: spatial proximity between person and group 10 Linking likelihood

Results 11 Compare Co-PF against MGT (without collaboration) An annotated dataset for group tracking does not exist Quantitative evaluation on a synthetic dataset emulating real scenarios ATA = Average Tracking Accuracy MOTA = Multiple Object Tracking Accuracy MOTP = Multiple Object Tracking Precision FP = False Positive MO = Multiple Objects FN = False Negative TSR = Tracking Success Rate [Kasturi et al 2009] [Kasturi et al 2009] R Kasturi, D Goldgof, P Soundararajan, V Manohar, J Garofolo,R Bowers, M Boonstra, V Korzhova, and J Zhang,“Framework for performance evaluation of face, text, and vehicledetection and tracking in video: Data, metrics, and protocol,”IEEE TPAMI, 31(2):319–336, 2009.

Results 12 Qualitative evaluation on publicly available dataset PETS 2009 dataset MGT

Results 13 Qualitative evaluation on publicly available dataset PETS 2009 dataset Co-PF

Results 14 Qualitative evaluation on publicly available dataset PETS 2009 dataset MGT

Results 15 Qualitative evaluation on publicly available dataset PETS 2009 dataset Co-PF

Conclusions 16 A probabilistic, collaborative framework for multi-group tracking have been proposed Additional evidence on the individuals helps the group tracking in an effective way The results prove that the collaboration between trackers improve the performances Future directions: Collaboration on the other direction (MGT MOT) Detection, split, and merge of the groups