1. MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence

2. MASKS © 2004 Invitation to 3D vision Given an image point in left image, what is the (corresponding) point in the right image, which is the projection of the same 3-D point Image Primitives and Correspondence

3. MASKS © 2004 Invitation to 3D vision Correspondence Lambertian assumption Rigid body motion Matching - Correspondence

4. MASKS © 2004 Invitation to 3D vision Translational model Affine model Transformation of the intensity values and occlusions Local Deformation Models

5. MASKS © 2004 Invitation to 3D vision Translational model RHS approx. by first two terms of Taylor series Small baseline Brightness constancy constraint Feature Tracking and Optical Flow

6. MASKS © 2004 Invitation to 3D vision Normal flow Aperture Problem

7. MASKS © 2004 Invitation to 3D vision Integrate around over image patch Solve Optical Flow

8. MASKS © 2004 Invitation to 3D vision rank(G) = 0 blank wall problem rank(G) = 1 aperture problem rank(G) = 2 enough texture – good feature candidates Conceptually: In reality: choice of threshold is involved Optical Flow, Feature Tracking

9. MASKS © 2004 Invitation to 3D vision Qualitative properties of the motion fields Previous method - assumption locally constant flow Alternative regularization techniques (locally smooth flow fields, integration along contours) Optical Flow

10. MASKS © 2004 Invitation to 3D vision Feature Tracking

11. MASKS © 2004 Invitation to 3D vision Compute eigenvalues of G If smallest eigenvalue  of G is bigger than  - mark pixel as candidate feature point Alternatively feature quality function (Harris Corner Detector) Point Feature Extraction

12. MASKS © 2004 Invitation to 3D vision Harris Corner Detector - Example

13. MASKS © 2004 Invitation to 3D vision Wide Baseline Matching

14. MASKS © 2004 Invitation to 3D vision Sum of squared differences Normalize cross-correlation Sum of absolute differences Region based Similarity Metric

15. MASKS © 2004 Invitation to 3D vision Compute image derivatives if gradient magnitude >  and the value is a local maximum along gradient direction – pixel is an edge candidate Canny edge detector gradient magnitude original image Edge Detection

16. MASKS © 2004 Invitation to 3D vision   x y Edge detection, non-maximum suppression (traditionally Hough Transform – issues of resolution, threshold selection and search for peaks in Hough space) Connected components on edge pixels with similar orientation - group pixels with common orientation Non-max suppressed gradient magnitude Line fitting

17. MASKS © 2004 Invitation to 3D vision Line fitting Lines determined from eigenvalues and eigenvectors of A Candidate line segments - associated line quality second moment matrix associated with each connected component Line Fitting

18. MASKS © 2004 Invitation to 3D vision Examples

19. MASKS © 2004 Invitation to 3D vision Feature Selection Compute Image Gradient Compute Feature Quality measure for each pixel Search for local maxima Feature Quality FunctionLocal maxima of feature quality function

20. MASKS © 2004 Invitation to 3D vision Feature Tracking Translational motion model Closed form solution 1. Build an image pyramid 2. Start from coarsest level 3. Estimate the displacement at the coarsest level 4. Iterate until finest level

21. Coarse to fine feature tracking 1. compute 2. warp the window in the second image by 3. update the displacement 4. go to finer level 5. At the finest level repeat for several iterations 0 2 1

22. MASKS © 2004 Invitation to 3D vision Integrate around over image patch Solve Optical Flow

23. MASKS © 2004 Invitation to 3D vision

24. MASKS © 2004 Invitation to 3D vision

25. MASKS © 2004 Invitation to 3D vision

26. MASKS © 2004 Invitation to 3D vision

27. MASKS © 2004 Invitation to 3D vision Affine feature tracking Intensity offset Contrast change

28. MASKS © 2004 Invitation to 3D vision Tracked Features

29. MASKS © 2004 Invitation to 3D vision Wide baseline matching Point features detected by Harris Corner detector

30. MASKS © 2004 Invitation to 3D vision Wide baseline Feature Matching 1. Select the features in two views 2. For each feature in the first view 3. Find the feature in the second view that maximizes 4. Normalized cross-correlation measure Select the candidate with the similarity above selected threshold