1. 1 Computational Vision CSCI 363, Fall 2012 Lecture 29 Structure from motion, Heading

2. 2 Depth Estimate from 2D velocities We can compare the 2D velocities that would be generated by the estimated depths of each point with the measured 2D velocities. => Estimate depths, Z i, and velocities: Minimize:

3. 3 Combine with Incremental rigidity The full model uses the 2D velocity estimates and the rigidity constraint: Minimize: E D + E R Where E D is from previous slide. E R is rigidity error: Temporal integration: Information from multiple frames is combined using Kalman filters. Velocity computed at one instant is used to predict depths at the next instant. This tends to smooth out errors.

4. 4 Results Estimated dot positions Surface reconstruction. Surface reconstruction uses smoothness and position of image points to interpolate a surface between point positions.

5. 5 Cylinder Perceptions Ramachandran showed that people do not see cylinder structure accurately in some conditions: Cylinders with different speeds. Cylinders with different radii. Perception

6. 6 Model results for illusions Illusion 1: Different speeds Illusion 2: Different radii The model generates structures similar to those perceived by people.

7. 7 Motion Information for a Moving observer When a person moves through a stationary world, the images of objects move across the retina. This image motion carries information about: 1)Relative depth of objects (from Motion parallax) 2)The direction of motion of the observer

8. 8 Motion Parallax For an observer moving in a straight line, the images of nearby objects move more quickly than those of distant objects. object 1 object 2

9. 9 Sources of Information For a moving observer, there are many sources of information that can help determine direction of motion: 1)Visual information 2)Proprioception (information about how we are moving our legs) 3)Vestibular information (information about acceleration). It is clear that under many conditions, vision alone is sufficient. (Video)

10. 10 Questions about Heading 1.How accurately can humans judge heading in different conditions? 2.What visual information do people use to judge heading? 3.What are the neural mechanisms that are used for computing heading?

11. 11 Image motion for a Moving observer: Translation

12. 12 Observer motion toward a surface

13. 13 Translation leads to a Radial pattern of image motion

14. 14 Image Velocities Suppose the observer has velocity T = (T X, T Y, T Z ) We can compute the image velocity, v = (v x, v y ): In the 1950's, Gibson noted that one can find the observer's motion direction (or heading) by finding the intersection of lines through any two of the image velocity vectors.

15. 15 Rotations

16. 16 Translation + Rotation

17. 17 Motion over a Ground Plane

18. 18 Image motion for Translation

19. 19 Image motion for Translation plus Rotation

20. 20 Image velocities for translation plus rotation

21. 21 Models for Dealing with Eye movements

22. 22

23. 23

24. 24 Data from Psychophysics Real eye movements Simulated eye movements True Heading

25. 25 Eye movements vs. Curved Paths Paths traversed by image points.

26. 26 Can we tell them apart?