1. Optical Snow and the Aperture Problem Michael Langer Richard Mann School of Computer Science McGill U. U. Waterloo

2. Optical snow e.g. falling snow

3. Optical snow

4. Egomotion in a 3D cluttered scene

5. Optical snow

6. Relevance to “Heading” Human observers can judge their direction of heading through a “3D cloud of dots”. (W. Warren and others) However, it is unclear how. Computational models assume accurate velocity estimates can be computed. (see Langer and Mann, ECVP ’01, ICCV ’01)

7. Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V1) 3.“Aperture problem”

8. Fourier model of image translation v f + v f + f = 0 x x y y t f y t (Watson & Ahumada ’85) f x f t

9. Image velocities are (α v, α v ) x y Optical Snow

10. α v f + α v f + f = 0 x x y y t f t f x f t Fourier model of optical snow “bowtie”

11. Example of bowtie in power spectrum bush image sequence

12. Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V1) 3.“Aperture problem”

13. f t f y f x Oriented, directionally tuned cells in V1. What about the brain ?

14. f t f y f x Oriented, directionally tuned cells in V1.

15. Image translation (v, v ) (see Heeger ’87, Yuille and Grzywacz ’90, Simoncelli and Heeger ‘97) f y t f x f t f t f y f x x y

16. f t f y f x t Optical snow (α v, α v ) f x y f y f x

17. Overview of Talk 1. Fourier model of image motion 2. Distributed codes in the brain (V1) 3.“Aperture problem”

18. The Aperture Problem 1. Perceived motion of a line is in “normal” direction. 2. Problem can be resolved if endpoints or corners are visible.

19. Aperture problem ? yes

20. Aperture problem ? no

21. Falling ellipsoids with random phase spectrum. Aperture problem ? ?

22. Open problems How do dominant spatial orientations affect perceived direction(s) of motion ? How do T-junctions, perspective, etc. affect perception of optical snow ? Computational models ?

23. f t f y f x Aperture problem t f f y f x