1. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction from uncalibrated images 2003. 12. 2 Young Ki Baik Visual Information Processing Lab. School of Radio Science & Communication Engineering. Hong-Ik University

2. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Contents Introduction Introduction  Overview Feature Extraction and Matching Feature Extraction and Matching Relating Image Relating Image  Camera model / Projection matrix  Fundamental matrix Initial reconstruction Initial reconstruction  Initial projection matrix

3. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Contents Auto-calibration Auto-calibration  Motivation  The Absolute Dual Quadric method  Result Future works Future works  Critical motion sequence  Dense matching  3D model building

4. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction from uncalibrated images Image Sequence Feature Extraction/ Matching Relating Image Projective Reconstruction Auto-Calibration Dense Matching 3D Model Building Overview Overview

5. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Feature Extraction / Matching Feature Extraction Feature Extraction  Harris corner detector OpenCVOpenCV C.Harris and M.Stephens, “ A combined corner and edge detector ”, Fourth Alvey Vision conference, 1988

6. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Feature Extraction / Matching Matching Matching  Optical flow Initial matchingInitial matching OpenCVOpenCV  Template matching ( SAD ) Near point matchingNear point matching

7. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Camera model Camera model

8. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Camera model Camera model

9. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Camera model Camera model

10. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Camera model Camera model  Non-square pixels and skew If m x and m y pixels per unit distance in x and y directions and skew, Pixel

11. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Fundamental matrix F Fundamental matrix F

12. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Relating images Fundamental matrix F Fundamental matrix F  RANSAC Normalized 8-Point algorithm (MVG p.265)Normalized 8-Point algorithm (MVG p.265)

13. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Initial reconstruction Initial reconstruction Initial reconstruction  Triangulation method Multiple View Geometry in computer visionMultiple View Geometry in computer vision –Hartly and Zisserman (p.297) World center

14. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Initial reconstruction Initial projection matrix Initial projection matrix  Projection matrix Get P matrix fromGet P matrix from initial world point X initial world point X  Bundle adjustment Sparse LMSparse LM F X x

15. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction Affine reconstruction(Calibration) Affine reconstruction(Calibration)

16. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction Affine reconstruction(Calibration) Affine reconstruction(Calibration)

17. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction Metric reconstruction Metric reconstruction Cholesky decomposition

18. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering 3D reconstruction Metric reconstruction Metric reconstruction

19. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Motivation Motivation  Avoid explicit calibration procedure Complex procedureComplex procedure Need for calibration objectNeed for calibration object Need to maintain calibrationNeed to maintain calibration  Allow flexible acquisition No prior calibration necessaryNo prior calibration necessary Possibility to vary intrinsic parametersPossibility to vary intrinsic parameters

20. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Projection matrix Projection matrix

21. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Projection matrix Projection matrix

22. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Constraints on intrinsic parameters Constraints on intrinsic parameters  Constant parameter Fixed cameraFixed camera K 1 = K 2 = …  Known parameter Rectangular pixel : s = 0Rectangular pixel : s = 0 Square pixel : s = 0, f x = f ySquare pixel : s = 0, f x = f y Principle point known : ( u x, u y ) = image centerPrinciple point known : ( u x, u y ) = image center

23. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Constraints on intrinsic parameters Constraints on intrinsic parameters  Constant parameter Faugeras et al. ECCV’92, Hartley’93, Triggs’97Faugeras et al. ECCV’92, Hartley’93, Triggs’97 –Kruppa equations, A stratified solution, Quadric Method Pollefeys et al. PAMI’98 …Pollefeys et al. PAMI’98 … –Absolute dual quadric method  Known parameter Heyden&Astrom CVPR’97Heyden&Astrom CVPR’97 Pollefeys et al. ICCV’98 …Pollefeys et al. ICCV’98 …

24. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration The Absolute Dual Quadric method The Absolute Dual Quadric method   **

25. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration The Absolute Dual Quadric method The Absolute Dual Quadric method  Constant parameter

26. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration The Absolute Dual Quadric method The Absolute Dual Quadric method  Constant parameter

27. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Result Result  Affine reconstruction

28. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Result Result  Metric reconstruction

29. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Auto-calibration Result ( Internal parameter K ) Result ( Internal parameter K )  Rig  Calibration (Vanishing point)  Quadric method

30. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Future works Critical Motion Sequence(CMS) Critical Motion Sequence(CMS)  Critical Motion Sequences have more than one potenti al absolute conic satisfying all constraints Pure translation / Pure rotationPure translation / Pure rotation Planar motion / Obital motionPlanar motion / Obital motion  Possible to derive classification of CMS  Sturm(CVPR´97), Kahl(ICCV´99)

31. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Future works Dense matching (Stereo vision) Dense matching (Stereo vision)  Dense matching through the epipolar line Conventional stereoConventional stereo Rectangulr subregioning / Sensus stereoRectangulr subregioning / Sensus stereo Cooperative stereo algorithmCooperative stereo algorithm Graph cut / PDE-based dense matchingGraph cut / PDE-based dense matching

32. HONGIK UNIVERSITY School of Radio Science & Communication Engineering Visual Information Processing Lab Hong-Ik University School of Radio Science & Communication Engineering Future works 3D model building 3D model building  Volumetric model  Super resolution  Plenoptic modeling and rendering