2. Stanford CS223B Computer Vision, Winter 2006 Lecture 12 Filters / Motion Tracking 2 Professor Sebastian Thrun CAs: Dan Maynes-Aminzade, Mitul Saha, Greg Corrado

3. Sebastian Thrun Stanford University CS223B Computer Vision Team Names 1.noname 2.Skynet 3.AED 4.WeeDrivers 5.Fear Factor 6.Dangerous Driver 7.First Star on the Left 8.Team Lee 9.The Five-Seconds Rule 10.Colorado 11.Pien 12.Triangulated 13.Making Drivers Obsolete 14.LyricalAssasins 15.Gal 16.Brad & Evan 17.Three Blind Mice 18.Optical Optima 19.The Visionaries 20.typedef destruct 21.IBM 22.Subway 23.Cyclops 24.Team Sad 25.monkeysee

4. Sebastian Thrun Stanford University CS223B Computer Vision n Online data cs223b.stanford.edu/competition

5. Sebastian Thrun Stanford University CS223B Computer Vision USB Frash Drive n team.txt -replace with you team name, number n /data /data/clip1 /data/clip2 /data/clip3 n /results –Will not be available n /submit clip1.png clip2.png clip3.png

6. Sebastian Thrun Stanford University CS223B Computer Vision Competition n 5 seconds of driving (150 frames) n 3 seconds of blackout

7. Sebastian Thrun Stanford University CS223B Computer Vision Important Information n Memorize your team number n Pick up flash drive today (SCPD: will post on Web) –Add files in submit directory –Change team.txt n Return on Monday

8. Sebastian Thrun Stanford University CS223B Computer Vision Moving Objects

9. Sebastian Thrun Stanford University CS223B Computer Vision Kalman Filter Tracking

10. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filter Tracking

11. Sebastian Thrun Stanford University CS223B Computer Vision Mixture of KF / PF (Unscented PF)

12. Sebastian Thrun Stanford University CS223B Computer Vision Review: Kalman Filters prior Measurement evidence posterior

13. Sebastian Thrun Stanford University CS223B Computer Vision Summary Kalman Filter n Estimates state of a system –Position –Velocity –Many other continuous state variables possible n KF maintains –Mean vector for the state –Covariance matrix of state uncertainty n Implements –Time update = prediction –Measurement update n Standard Kalman filter is linear-Gaussian –Linear system dynamics, linear sensor model –Additive Gaussian noise (independent) –Nonlinear extensions: extended KF, unscented KF: linearize n More info: –CS226 –Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press)

14. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filters n An alternative technique for tracking n Easier to implement n Nonlinear n Better for data association n In CV, known as “Condensation Algorithm”

15. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filter

16. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filters: Basic Idea (equality for ) set of n particles X t See e.g., [Doucet 98, deFreitas 98]

17. Sebastian Thrun Stanford University CS223B Computer Vision Basic Particle Filter Algorithm Initialization: X 0  n particles x 0 [i] ~ p(x 0 ) particleFilters( X t  1 ){ for i=1 to n x t [i] ~ p(x t | x t  1 [i] ) (prediction) w t [i] = p(z t | x t [i] ) (importance weights) endfor for i=1 to n include x t [i] in X t with probability  w t [i] (resampling) }

18. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filters: Illustration With: Wolfram Burgard, Dieter Fox, Frank Dellaert

19. Sebastian Thrun Stanford University CS223B Computer Vision Particle Filter Explained

20. Sebastian Thrun Stanford University CS223B Computer Vision Monte Carlo Localization (1)

21. Sebastian Thrun Stanford University CS223B Computer Vision Monte Carlo Localization (2)

22. Sebastian Thrun Stanford University CS223B Computer Vision Particles = Robustness

23. Sebastian Thrun Stanford University CS223B Computer Vision Tracking People from Moving Platform  robot location (particles)  people location (particles)  laser measurements (wall) With Michael Montemerlo

24. Sebastian Thrun Stanford University CS223B Computer Vision Tracking People from Moving Platform With Michael Montemerlo  robot location (particles)  people location (particles)  laser measurements (wall)

25. Sebastian Thrun Stanford University CS223B Computer Vision Examples Siu Chi Chan McGill University

26. Sebastian Thrun Stanford University CS223B Computer Vision Examples Particle filterOptical flow D. Stavens. D. Lieb. A. Lookingbill

27. Sebastian Thrun Stanford University CS223B Computer Vision Another Example Mike Isard and Andrew Blake

28. Sebastian Thrun Stanford University CS223B Computer Vision Tracking Fast moving Objects K. Toyama, A.Blake

29. Sebastian Thrun Stanford University CS223B Computer Vision More Particle Filter Tracking David Stavens, Andrew Lookingbill, David Lieb, CS223b Winter 2004

30. Sebastian Thrun Stanford University CS223B Computer Vision Nonlinearity in the Particle Filter

31. Sebastian Thrun Stanford University CS223B Computer Vision Data Association in Particle Filters n Suppose: k features in image, k state variables. Which ones to marry? n Particle filter: Each particle selects its own data association n Probabilistic interpretation: Particles are posteriors over continuous state and discrete data associations. (KF can only to continuous state)

32. Sebastian Thrun Stanford University CS223B Computer Vision Summary Kalman Filter n Estimates state of a system –Position –Velocity –Many other continuous state variables possible n KF maintains –Mean vector for the state –Covariance matrix of state uncertainty n Implements –Time update = prediction –Measurement update n Standard Kalman filter is linear-Gaussian –Linear system dynamics, linear sensor model –Additive Gaussian noise (independent) –Nonlinear extensions: extended KF, unscented KF: linearize n More info: –CS226 –Probabilistic Robotics (Thrun/Burgard/Fox, MIT Press) Particle and discrete set of particles (example states) = predictive sampling = resampling, importance weights fully nonlinear easy to implement