1. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 1 Integration of shape constraints in data association filters Integration of shape constraints in data association filters Giambattista Gennari, Alessandro Chiuso, Fabio Cuzzolin, Ruggero Frezza University of Padova chiuso@dei.unipd.it www.dei.unipd.it/~chiuso chiuso@dei.unipd.it www.dei.unipd.it/~chiuso

2. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 2 Outline of the Talk Tracking and Data Association Classical solution: independent dynamics Our approach : integration of shape Occlusions Experiments

3. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 3 Tracking and Data Association PROBLEM: PROBLEM: Set of targets generating UNLABELLED measurements Associate and Track Occlusions Clutter

4. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 4 SHAPE AND COORDINATION Motion invariant properties of targets: Rigid or Articulated bodies Formations of vehicles (Flock of birds) Deformable objects Distances and/or angles Connectivity – distances Relative velocity Group of admissible deformations (probabilistic or deterministic)

5. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 5 Probabilistic Tracking and Data Association CLASSICALLY: JPDAF – MHT JPDAF – MHT + Dynamical Models Dynamical Models Full (joint) model -not flexible -computationally expensive Model targets Independently -flexible and easy -not robust occlusions exchange tracks OUR APPROACH: JPDAF- (MHT) + Independent Dynamical Models + Shape Information + Flexible + Robust to occlusions and track proximity - Computation (Monte Carlo)

6. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 6 Independent Motion Targets are described by independent dynamics Flexible and easy Lack of robustness in presence of occlusions, false detections and closely spaced targets Index of Target

7. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 7 Associations An association is a map matching unlabelled measurements to targets Employ the overall model to compute the probability of each association Association MeasurementsMeasurements matched to clutter Measurements matched to targets

8. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 8 Description of Shape Probabilistic Model Example: pairwise distances of non perfectly rigid bodies Motion Invariant Prior Knowledge Learn from Data Targets positions

9. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 9 Shape Integration We assume the overall model can be factored into two terms describing the mutual configuration and single target dynamics Kalman filters and independent dynamical models Shape constraints

10. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 10 Occlusions To compute marginalize over the occluded : Detected pointsMissing points (occlusions) Compute the integral through Monte Carlo techniques

11. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 11 Monte Carlo Integration Sample: Weight: Integrate: Fair sample from the posterior

12. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 12 Summary Conditional state estimates SHAPE INDEPENDENT KALMAN FILTERS T1TNT2 …. Monte Carlo fair samples for occluded points state estimation OVERALL MODEL Association probabilities Measurements

13. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 13 State Estimation An overall state estimate can be obtained summing the conditional state estimates weighted by the corresponding association probabilities Alternatively, several state estimates can be propagated over time (multi hypothesis tracker ) Necessary in the learning phase !

14. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 14 Results Real data from a motion capture system Rapid motion High numbers of false detections Occlusions lasting several frames

15. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 15 Results Commercial system: looses and confuses tracks With shape knowledge learned from data

16. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 16 Conclusions Algorithm for integrating shape knowledge into data association filter Robust in presence of occlusions and clutter Provide a framework for learning shape models (this requires use of multiple hypothesis kind of algorithms) (In the example shape was learned from data)

17. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 17

18. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 18 Shape Constraints In many cases, coordinated points exhibit properties which are invariant with respect to their motion, they satisfy some sort of shape constraints: – pairwise distances of rigidly linked points are constant – the position and velocity of a point moving in group are similar to those of its neighbors

19. IEEE CDC 2004 - Nassau, Bahamas, December 14-17 19 Coordinated Motion Rigid motion Articulated bodies, Groups of people moving together, Formations Taking into account coordination improves tracking robustness We describe shape and motion separately and combine them together ( more flexible than joint models )