1. Feature Matching

2. Feature Space Outlier Rejection

3. After Outlier Rejection

4. RANdom SAmple Consensus

5. RANSAC for Homography

6. Example: Panorama

7. Homography Transform,Warping

8. Image Warping

9. Forward Warping

11. Inverse Warping

13. Forward vs. Inverse Warping

14. Two View Geometry  When a camera changes position and orientation, the scene moves rigidly relative to the camera 3-D Scene u u’u’ Rotation + Translation

15. Two View Geometry (simple cases)  In two cases this results in homography: 1. Camera rotates around its focal point 2. The scene is planar Then: Point correspondence forms 1:1mapping depth cannot be recovered

16. The General Case: Epipolar Lines epipolar line

17. Epipolar Plane epipolar plane epipolar line Baseline P O O’

18. Epipole  Every plane through the baseline is an epipolar plane  It determines a pair of epipolar lines (one in each image)  Two systems of epipolar lines are obtained  Each system intersects in a point, the epipole  The epipole is the projection of the center of the other camera epipolar lines Baseline O O’

19. Epipolar Lines epipolar plane epipolar line Baseline P O O’ To define an epipolar plane, we define the plane through the two camera centers O and O’ and some point P. This can be written algebraically (in some world coordinates) as follows:

20. Essential Matrix (algebraic constraint between corresponding image points)  Set world coordinates around the first camera  What to do with O’P? Every rotation changes the observed coordinate in the second image  We need to de-rotate to make the second image plane parallel to the first  Replacing by image points

21. Essential Matrix (cont.)  Denote this by:  Then  Define  E is called the “essential matrix”

22. Properties of the Essential Matrix  E is homogeneous  9 parameters  E can be recovered up to scale using 8 points.  The constraint det E=0  7 points suffices  In fact, there are only 5 degrees of freedom in E, 3 for rotation 2 for translation (up to scale), determined by epipole

23.  Background The lens optical axis does not coincide with the sensor  We model this using a 3x3 matrix the Calibration matrix Camera Internal Parameters or Calibration matrix

24. Camera Calibration matrix  The difference between ideal sensor and the real one is modeled by a 3x3 matrix K:  (c x,c y ) camera center, (a x,a y ) pixel dimensions, b skew  We end with

25. Fundamental Matrix  F, is the fundamental matrix.

26. Properties of the Fundamental Matrix  F is homogeneous  9 parameters  F can be recovered up to scale using 8 points.  The constraint det F=0  7 points suffices

27. Epipolar Plane l’ l Baseline P O O’ Other derivations Hartley & Zisserman p. 223 x X’ e e’ e’

28. HomographyEpipolar Form ShapeOne-to-one mapConcentric epipolar lines D.o.f.88/5 F/E Eqs/pnt21 Minimal configuration 45+ (8, linear) Depth NoYes, up to scale Scene Planar (or no translation) 3D scene Two-views geometry Summary:

29. Stereo Vision  Objective: 3D reconstruction  Input: 2 (or more) images taken with calibrated cameras  Output: 3D structure of scene  Steps: Rectification Matching Depth estimation

30. Rectification Image Reprojection  reproject image planes onto common plane parallel to baseline Notice, only focal point of camera really matters (Seitz)

31. Rectification  Any stereo pair can be rectified by rotating and scaling the two image planes (=homography)  Images have to be rectified so that Image planes of cameras are parallel. Focal points are at same height. Focal lengths same.  Then, epipolar lines fall along the horizontal scan lines of the images

32. References  http://web.me.com/dellaert/07F-Vision/Schedule.html http://web.me.com/dellaert/07F-Vision/Schedule.html  http://cseweb.ucsd.edu/classes/wi07/cse252a/homography_estimation/ho mography_estimation.pdf http://cseweb.ucsd.edu/classes/wi07/cse252a/homography_estimation/ho mography_estimation.pdf  http://en.wikipedia.org/wiki/Homography http://en.wikipedia.org/wiki/Homography  http://www.andrew.cmu.edu/course/16-720/lectures/figs1.pdf http://www.andrew.cmu.edu/course/16-720/lectures/figs1.pdf  http://www.cs.utoronto.ca/~strider/vis-notes/tutHomography04.pdf http://www.cs.utoronto.ca/~strider/vis-notes/tutHomography04.pdf  http://people.scs.carleton.ca/~c_shu/Courses/comp4900d/notes/ http://people.scs.carleton.ca/~c_shu/Courses/comp4900d/notes/  http://graphics.cs.cmu.edu/courses/15- 463/2005_fall/www/Lectures/RANSAC.pdf http://graphics.cs.cmu.edu/courses/15- 463/2005_fall/www/Lectures/RANSAC.pdf  http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/lec/warping.pdf http://www.ics.uci.edu/~dramanan/teaching/cs116_fall08/lec/warping.pdf  http://www.wisdom.weizmann.ac.il/~bagon/CVSpring08/files/2ViewsPart 2.ppt http://www.wisdom.weizmann.ac.il/~bagon/CVSpring08/files/2ViewsPart 2.ppt