1. Robust Tracking by Ilya Levner ilya@cs.ualberta.ca

2. Outline Introduction Survey of existing techniques –Anti-failure –Post-failure IFA –Pros & Cons Research Direction

3. Introduction x*(t) – the true state of target object X in (t) – the input state set X out (t) – the output state set Accuracy – x*(t) є X out (t) Precision - |X out (t)| -1 Robustness - The ability to accurately and precisely track objects under less than ideal conditions.

4. RobustnessCategories Robustness Categories Anti-failure – prevention of errors (I.e. precision loss) and failure. Post-failure – recovery from failure

5. Anti-failure Seeded by attempts in 1953 to create robust statistical estimators. Has received the most attention from the vision community w.r.t post-failure.

6. Anti-failure Approaches Window Processing and Foveation –Ignore/blur the image around the target to avoid/remove background distractions. Robust Matching Techniques – Handle occlusion by detection of outliers Color cue concentration – Enables a tracker to handle changes in lighting. CONDENSATION –Generalized Kalman filters –Handle agile motion

7. Post-failure Robustness “The synthesis of reliable organisms from unreliable parts” von Neumann, 1952 Implies non-catastrophic error/ failure recovery

8. Post-failure (cont) Most active research in field of planning –Replanning paths Behaviour-Based Robotics –“emergent” intelligence known to display post- failure robustness. (subsumption architecture) Limited work in the visual tracking domain

9. Incremental Focus of Attention System (IFA) Biologically inspired methodology –Pre-attentive mechanisms select a target subregion –Post-attentive examine the subregion for relevance IFA uses a hierarchy of: – selectors that search for a subregion containing the target – trackers which keep the focus on the target.

10. color thresholding blob tracking template-based tracking target state full configuration space algorithmic layers feature-based tracking IFA (Face Tracking)

11. Technique Comparison Robustness to Occlusion Anti-failure Method –robust statistics to filter out the non-signal data. SSD tracker with oulier detection Post-failure Method –IFA Same SSD tracker without outlier mechanism at the top layer.

12. Results Ideal Conditions (no occlusion) –Equal precision –AF is 15-20% slower due to overhead processing Small Occlusions –AF tracks at full precision –PF drops to color blob tracking, resulting in a significant loss of precision. (Recovery within 100msec) Large-Full Occlusions (>40% of target) –AF looses target region and never recovers –PF takes between 150ms to several seconds for recovery to full precision.

13. Anti-failurePros Handles specific perturbations well. Can avert catastrophic failureCons Different modes of failure require individual contingencies In ideal conditions slows down the system Difficult if not impossible to achieve in real-world systems (too many things can go wrong)Post-failurePros A single procedure usually enough to recover from any failure type. Dormant in ideal conditionsCons Meaningless in a catastrophic failure case(s). Fixed (hand-coded) hierarchy Slow recovery at times

14. Extensions Add Learning Module(s) to IFA Motion prediction (AF / PF – recovery optimization) Dynamic Tracker Selection ?

15. Motion Prediction Past attempts used Kalman Filter Why not try HMM/CHMM to predict motion. –Let δμ* be the true change in motion parameters (state) –~δμ computed motion parameters (observations)

16. Plan of Action 1)Construct a rudimentary IFA system, 3-4 layers consisting of: x,y translation trackers at 1,1/2,1/4 resolution Color blob tracker 2) Construct an HMM for each level within the hierarchy 3) Couple the HMM’s together (CHMM)

17. Hidden Markov Model(s)

18. Coupled HMM, A matrix.

19. Issues Relearning –The A matrix will become uniform, i.e. all state transitions have equal probabilities. –How can recent states transitions have more weight in the Baum-Welch training phase ?? Search depth

20. References