1. What we didn’t have time for CS664 Lecture 26 Thursday 12/02/04 Some slides c/o Dan Huttenlocher, Stefano Soatto, Sebastian Thrun

2. Administrivia Final project is due at noon on Friday 12/17 Write-up only (5MB max) Be sure to include some pictures Send me email if you missed any quiz for a good reason

3. Outline Geometry Graph-based segmentation Statistics

4. Geometry

5. Homogeneous coordinates Identify a point in the image plane with ray passing through that point (pixel) (x,y) ´ (  x,  y,  ) for non-zero  (X,Y,Z) ´ (X/Z,Y/Z,1) for non-zero Z

6. Advantages Many non-linear operations become linear in homogeneous coordinates Example: ( X, Y, Z ) projects to ( fX / Z, fY / Z ) 2D point 3D point 3x4 camera projection

7. Camera projection matrix

8. epipole Epipolar geometry epipolar plane epipolar line Stefano Soatto (c) 2002

9. Pencil of planes Different epipolar planes for different scene points x Plane defined by camera origins + x

10. Epipolar lines are important For pixel p in I there is a corresponding epipolar line in I’ This allows us to limit the search! Generalization of stereo to arbitrary camera positions Classical stereo has parallel cameras p

11. Example: verged stereo

12. Examples: motion Parallel to Image Plane Forward

13. Essential matrix E Ex is perpendicular to x ’s epipolar line in the other image So if x ’ corresponds to x then x ’ T Ex = 0 Captures the scene geometry We assume the cameras are calibrated Otherwise we get the fundamental matrix

14. Estimating the geometry The essential matrix has 5 parameters Can estimate from 5 corresponding points Fundamental matrix has 7 The question of “how few perfect correspondences do you need” has spawned an unfortunately large literature

15. Yet more optimization We can estimate the essential matrix from a bunch of point matches A similar technique can be used to compute structure from motion Bundle adjustment

16. RANSAC (line fitting) Variant of generate-and-test Pick a small set of points at random Fit them via least squares Points “far” from this line are outliers Repeat until you find a line with very few outliers

17. RANSAC (camera geometry) Pick a small set of corresponding pixels At least 5 (essential) or 7 (fundamental) Compute the matrix from these See how many corresponding pixels this matrix explains

18. Graph-based Segmentation

19. Segmentation by min cut Image Pixels w Similarity Measure Minimum Cut * From Khurram Hassan-Shafique CAP5415 Computer Vision 2003

20. Min cuts don’t segment well Ideal Cut Cuts with lesser weight than the ideal cut * Slide from Khurram Hassan-Shafique CAP5415 Computer Vision 2003

21. Normalized cuts Instead of the min cut, minimize Measure of dis-similarity between the sets A and B NP-hard to minimize Rely on continuous approximation

22. Normalized cuts examples

23. Limitations of normalized cuts Works by binary partitioning Slow and memory-intensive Textured backgrounds are problems

24. Other graph-based methods Many other variants on min cuts Typical cuts, nested cuts, etc. No clear winner for segmentation Perhaps mean shift?

25. MST-based segmentation Minimum spanning tree is the cheapest way to connect all pixels into a single component (or “region”) Merge two components when the cheapest edge between them is cheap compared to a measure of the internal variation Provably good segmentation under a fairly natural definition Neither too coarse nor too fine

26. Example output Solves many problems with normalized cuts

27. More statistics

28. Dimensionality reduction We can represent orange points only by their v 1 coordinate

29. Eigenfaces An n -pixel image is a point in < n Find low-dimensional representation of face images (from a training set) Recognition by finding the closest face in face space

30. Markov Random Fields MRF defining property: Hammersley-Clifford Theorem: neighborhood relationships ( n-links ) image pixels ( vertices ) - disparity at pixel p - configuration

31. MAP estimation of an MRF Observed data Likelihood function (sensor noise) Prior (MRF model) Bayes rule

32. Energy minimization Data term (sensor noise) Smoothness term (MRF prior)