1. Boris 2 Boris Babenko 1 Ming-Hsuan Yang 2 Serge Belongie 1 (University of California, Merced, USA) 2 (University of California, San Diego, USA) Visual Tracking with Online Multiple Instance Learning

2. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusions 2

3. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusions 3

4. First frame is labeled

5. Classifier Online classifier (i.e. Online AdaBoost)

6. Grab one positive patch, and some negative patch, and train/update the model. negative positive Classifier

7. Get next frame negative positive Classifier

8. Evaluate classifier in some search window negative positive Classifier

9. Evaluate classifier in some search window negative positive old location X Classifier

10. Find max response negative positive old location new location X X Classifier

11. Repeat… negative positive negative positive Classifier

12. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusion 12

13. What if classifier is a bit off? Tracker starts to drift How to choose training examples?

14. Classifier MIL Classifier

15. Ambiguity in training data Instead of instance/label pairs, get bag of instances/label pairs Bag is positive if one or more of it’s members is positive

16. Problem: Labeling with rectangles is inherently ambiguous Labeling is sloppy

17. Solution: Take all of these patches, put into positive bag At least one patch in bag is “correct”

18. Classifier MIL Classifier

19. MIL Classifier

20. Supervised Learning Training Input MIL Training Input

21. Positive bag contains at least one positive instance Goal: learning instance classifier Classifier is same format as standard learning

22. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusion 22

23. Need an online MIL algorithm Combine ideas from MILBoost and Online Boosting

24. Train classifier of the form: where is a weak classifier Can make binary predictions using

25. Objective to maximize: Log likelihood of bags: where (Noisy-OR)

26. Objective to maximize: Log likelihood of bags: where (Noisy-OR) (as in LogitBoost)

27. Train weak classifier in a greedy fashion For batch MILBoost can optimize using functional gradient descent. We need an online version…

28. At all times, keep a pool of weak classifier candidates

29. At time t get more training data Update all candidate classifiers Pick best K in a greedy fashion

30. Frame tFrame t+1 Get data (bags) Update all classifiers in pool Greedily add best K to strong classifier

32. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusion 32

33. MILTrack = Online MILBoost + Stumps for weak classifiers + Randomized Haar features + greedy local search

34. 34

35. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusions 35

36. Compare MILTrack to: OAB1 = Online AdaBoost w/ 1 pos. per frame OAB5 = Online AdaBoost w/ 45 pos. per frame SemiBoost = Online Semi-supervised Boosting FragTrack = Static appearance model

37. 37

38. 38

39. Best Second Best

40. Introduction Multiple Instance Learning Online Multiple Instance Boosting Tracking with Online MIL Experiments Conclusions 40

41. Proposed Online MILBoost algorithm Using MIL to train an appearance model results in more robust tracking