1. Computer Science Department Distribution Fields A Unifying Representation for Low-Level Vision Problems Erik Learned-Miller with Laura Sevilla Lara, Manju Narayana, Ben Mears

2. 2 Distribution Fields Unifying Low-Level Vision  Tracking  Optical Flow  Affine invariant matching  Stereo  Backgrounding  Registration  Image Stitching What kind of representation serves all these problems well?

3. 3 Distribution Fields Unifying Descriptors and Representations  HOG, SIFT, Shape Contexts  Geometric blur  Multi-scale methods  Mixture of Gaussian backgrounding  Bilateral filter  Joint alignment (congealing)

4. 4 Distribution Fields Tracking

5. 5 Distribution Fields Basics of Tracking Frame TFrame T+d

6. 6 Distribution Fields Basics of Tracking Frame TFrame T+d

7. 7 Distribution Fields Basics of Tracking Frame TFrame T+d

8. 8 Distribution Fields Basics of Tracking Frame TFrame T+d

9. 9 Distribution Fields The Core Matching Problem patch I image J Find best match of patch I to image J, for some set of transformations.

10. 10 Distribution Fields Key issues  Large “basin of attraction” in gradient descent  Robustness to appearance changes of target

11. 11 Distribution Fields Local optimum problem in alignment Unaligned Stuck in a local optimum

12. 12 Distribution Fields Common solution to gradient descent matching  Blur images? “Spreads information” Also destroys information through averaging

13. 13 Distribution Fields Exploding an image One plane for each feature value.

14. 14 Distribution Fields Spatial Blur: 3d convolution with 2d Gaussian

15. 15 Distribution Fields Spatial Blur: 3d convolution with 2d Gaussian KEY PROPERTY: doesn't destroy information through averaging

16. 16 Distribution Fields Properties of Distribution Field Representation  Much larger basin of attraction than other descriptors.  A useful probability model from a single image.  Inherently multi-scale.  Extension and unification of many popular descriptors.  State of the art performance from extremely simple algorithms.

17. 17 Distribution Fields Properties of Distribution Field Representation  Much larger basin of attraction than other descriptors.  A useful probability model from a single image.  Inherently multi-scale.  Extension and unification of many popular descriptors.  State of the art performance with extremely simple algorithms.  THANKS!

18. 18 Distribution Fields The likelihood match

19. 19 Distribution Fields Sharpening match

20. 20 Distribution Fields Understanding the sharpening match What standard deviation maximizes the likelihood of a given point under a zero-mean Gaussian?

21. 21 Distribution Fields Intuition behind sharpening match  Increase standard deviation until it matches “average distance” to matching points.

22. 22 Distribution Fields Properties of the sharpening match  A patch has probability of 1.0 under its own distribution field.  Probability of an image patch degrades gracefully as it is translated away from best position.  Optimum “sigma” value gives a very intuitive notion of the quality of the image match.

23. 23 Distribution Fields Basin of attraction studies

24. 24 Distribution Fields Basin of attraction studies GIVEN A RANDOM PATCH...

25. 25 Distribution Fields Basin of attraction studies AND A RANDOM DISPLACEMENT...

26. 26 Distribution Fields Basin of attraction studies

27. 27 Distribution Fields Basin of attraction studies

28. 28 Distribution Fields Basin of attraction results

29. 29 Distribution Fields Tracking results  State of the art results on tracking with standard sequences Very simple code Trivial motion model Simple memory model

30. 30 Distribution Fields Closely Related work  Mixture of Gaussian backgrounding (Stauffer...)  Shape contexts (Belongie and Malik)  Congealing (me)  Bilateral filter  SIFT (Lowe), HOG (Dalal and Triggs)  Geometric Blur (Berg)  Rectified flow techniques (Efros, Mori)  Mean-shift tracking  Kernel tracking  and many others...