1. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 1 The plan for today Leftovers and from last time Camera matrix Part A) Notation, preprocessing, and concepts. Part B) 4 Stereo Algorithms

2. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 2 The first “photograph” www.hrc.utexas.edu/exhibitions/permanent/wfp/ Joseph Nicéphore Niépce. View from the Window at Le Gras. 1826.

3. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 3 Perspectivities Projectivities Perspectivities are not a group L l1 l2

4. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 4 Following slides are mainly Courtesy of Reinhard Koch and Jan-Michael Frahm From their Tutorial at DAGM 2001, München

5. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 5 Focal length f aperture image object View direction lens Image sensor Center (c x, c y ) Pinhole camera model

6. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 6 Camera matrix X Y O  (2)(2) Canonic camera matrix P 0 : Perspective projection P 0 from P 3 onto P 2 : Hartley & Zisserman Ch 6.

7. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 7 Perspective projection Perspective projection in  3 models pinhole camera: –scene geometry is affine  3 space with coordinates M=(X,Y,Z,1) T –camera focal point in O=(0,0,0,1) T, camera viewing direction along Z –image plane (x,y) in  (  2 ) aligned with (X,Y) at Z= Z 0 –Scene point M projects onto point M p on plane surface X Y Z0Z0 33 O  (2)(2) Image plane Camera center Z (Optical axis) x y

8. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 8 A Transformation Projective Transformation maps M onto M p (image plane in  3 space) X Y O

9. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 9 Dimension reduction Dimension reduction from  3 into  2 by projection onto  (  2 ) X Y O  (2)(2) Combined transformation+ reduction yelids : The “canonic camera” P 0 from  3 onto  2 :

10. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 10 Projection in general pose Rotation [R] Projection center C M World coordinates Projection: mpmp

11. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 11 X Y Image center c= (c x, c y ) T Projection center Z (Optical axis) Focal length Z 0 Pixel scale f= (f x,f y ) T x y Pixel coordinates m = (y,x) T Image plane and image sensor A sensor with picture elements (Pixel) is added onto the image plane Image sensor Image-sensor mapping: Pixel coordinates are related to image coordinates by affine transformation K with five parameters: –Image center c at optical axis –distance Z p (focal length) and Pixel size scales pixel resolution f –image skew s to model angle between pixel rows and columns

12. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 12 General Projection matrix P Camera projection matrix P combines: –inverse transformation T a -1 to canonic pose (origin + orientation) –Canonic Perspective projection P 0 –affine mapping K from image to sensor coordinates

13. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 13 Calibrated Camera: K known, Pose (R,C) unknown (metric camera ) Uncalibrated camera: K,R,C unknown (projective camera ) Summary projective cameras A 3X4 matrix of rank 3

14. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 14 Reconstruction from projective cameras unknown: Scene points M, projection matrices P known: image projections m i of scene points M i problem: reconstruction is ambiguous in projective space! scene is defined only up to a projective Transformation T camera is skewed by inverse Transformation T -1

15. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 15 Ambiguity in projective reconstruction T Euclidean scene Projective reconstruction Valid projective reconstructions

16. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 16 Self-Calibration T -1 Metric reconstruction Projective reconstruction Apply self-calibration to recover T -1 Recover metric structure from projective reconstruction Use constraints on the calibration matrix K Utilize invariants in the scene to estimate K Hartley & Zisserman Ch. 19 Not part of this class

17. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 17 Two views geometry Given two views: X Y O Z C1C1 P0P0 P1P1 Epipolar line

18. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 18 The Fundamental Matrix F The projective points e 1 and (H  m 0 ) define an epipolar line l e = e 1 x (H  m 0 ) The corresponding point m 1 lies on line: m 1 T l e = 0 Fundamental Matrix F Epipolar constraint

19. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 19 Two views geometry Given two views: X Y O Z C1C1 P0P0 P1P1 Epipolar line Epipolar constraint

20. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 20 The Fundamental Matrix F P0P0 m0m0 L l1l1 M m1m1 MM P1P1 m 1  =H  m 0 Epipole e1e1 F = [e] x H  = Fundamental Matrix

21. CAU Kiel DAGM 2001-Tutorial on Visual-Geometric 3-D Scene Reconstruction 21 Estimation of F from image correspondences Given a set of corresponding points, solve since defined up to scale, only eight elements are independent since the operator [e] x and hence F have rank 2, F has only 7 independent parameters (all epipolar lines intersect at e) each correspondence gives 1 collinearity constraint => solve F with minimum of 7 correspondences for N>7 correspondences minimize distance point-line: