1. From local motion estimates to global ones - physiology:

2. Motion fields for more complex patterns: Hildreth (1985): Smoothness of velocity field along the contour True motion field Local motion estimates Smoothest Velocity field

3. Motion fields for more complex patterns (contd.):

4. Ellipse demo

5. Recovering 3D structure from motion: Kinetic Depth Effect [Wallach, 1953] Percept Another possible percept Inference: The human visual system has a preference for rigid interpretations

6. Ullman’s model for recovering 3D structure from motion: 1.Establish correspondence between features in different frames 2.Recover transformation matrix and z values of points Key result: For a rigid structure, 4 non-coplanar points in 3 frames are sufficient to solve for all the unknowns [Ullman, 1979] Open questions: 1. Do these bounds apply to human observers too? 2. Does the rigidity assumption always hold? 3. How do we recover the 3D structure of non-rigid dynamic objects? Video 1: NR rotating object Video 2: Johansson

7. Ullman’s incremental rigidity scheme: Allows structure recovery even for gradually deforming objects. However: 1. Humans are able to recover 3D structures even with just 2 frames. It is unclear how this is accomplished. 2. Correspondence is not an easy problem. Errors in correspondence lead to errors In structure recovery.

8. Processing Framework Proposed by Marr Recognition Shape From stereo Motion flow Shape From motion Color estimation Color estimation Shape From contour Shape From shading Shape From texture 3D structure; motion characteristics; surface properties Edge extraction Image

9. Color

11. Color Estimation: Goal: To recover the intrinsic surface reflectance of an object. And yet, we have good lightness constancy!

12. Lightness Constancy: The constancy in perceived surface reflectance regardless of differences in illumination. Goal: Given L, recover R. Clearly underconstrained. Assumptions are needed for unique solutions. Luminance (L) = Reflectance (R) * Illumination (I) Helmholtz’s theory: Observer ‘knows’ I through past experience. Hering, Wallach, Land & McCann: Observer computes luminance ratios across edges. (some important hidden assumptions here) Explain fig above

13. The perceptual importance of luminance ratios at edges: Cornsweet Illusion

14. Explaining simultaneous contrast illusions via edge ratios:

15. Are ratios taken with actual or perceived luminances? TANGENT ALERT!

19. Land and McCann’s Retinex theory: * I R L Given L, recover R

20. Land and McCann’s Retinex theory - Assumptions: 1.The world is flat and all sharp luminance variations are due to changes in reflectance. Reflectance always changes abruptly. 2.Illumination changes gradually across a scene. Basic idea: Preserve luminance ratios at edges and discard slow variations.