Self-Calibration and Metric Reconstruction from Single Images Ruisheng Wang Frank P. Ferrie Centre for Intelligent Machines, McGill University

Outline Contributions Existing Methods The Idea Our Method Comparison Conclusion

Contributions We developed a direct solution to 3D reconstruction from single images which is model based – No model-to-image projection and readjustment procedure We made it possible to perform accurate 3D measurement using an uncalibrated* camera based on single images only We quantitatively evaluated our model-based approach with vanishing point based method, and the results indicate our approach is better than vanishing point based method * Intrinsic parameters known/estimated

Three vanishing points Vanishing Points (VPs) Based Method Determine three orthogonal vanishing points – Manual detection – Search over Gaussian sphere – Hough transform – Projective geometry Determine camera focal length and rotation Determine camera translation and model dimensions

Problems in VPs Based Method Three orthogonal VPs may not be always available – One or two vanishing point only Hard to accurately determine VPs – Need many lines The accuracy of VPs affects the accuracy of the 3D reconstruction 1 Point Perspective2 Points Perspective

Methods with Ground Control Points/Lines Point-based methods – Collinearity Equations Line-based methods – Model-to-Image Fitting From Debevec et al.1996

The Idea Use model to estimate camera exterior orientation – Need 6 parameters X3, Y3, L, W, H, If an object-centered coordinate system selected – Need three parameters L, W, H Divide camera parameters into two groups: rotation and translation It’s possible to estimate relative camera exterior orientation without using GCPs and Vanishing Points L H 3 6 WX Y Z H W Z 3Y X O L X3, Y3

Overview of Our Approach Self-Calibration – Recover camera rotation Initial estimate of camera rotation Refinement of camera rotation – Determine camera translation and building dimensions Simultaneous estimates of camera translation and the first building dimensions Metric Reconstruction – Roughly estimate the second building orientation – Refine the second building orientation – Determine the second building dimensions and location

Initial estimate of camera rotation Refinement of camera rotation Recover Camera Rotation Imaging Geometry Relationship

Form objective function Solve a constrained quadratic form minimization problem Determine Camera Translation and the First Building Dimensions Imaging Geometry Relationship

Building 2 Model edge 56 (v, u) Camera Coordinate System Image edge 56{(x 1, y 1, -f), (x 2, y 2, -f)} Object Coordinate System Building 1 Model edge 67(v, u) 5 m (m x, m y, m z ) C 3( X 3, Y 3 ) z Y t(X 0, Y 0,Z 0 ) R 1 Assuming both buildings lie on the same ground plane Initial estimate of the second building orientation Refinement of the second building orientation Recover the Second Building Orientation Imaging Geometry Relationship x

Building 2 Model edge 56 (v, u) Camera Coordinate System Image edge 56{(x 1, y 1, -f), (x 2, y 2, -f)} Object Coordinate System Building 1 Model edge 67(v, u) 5 m (m x, m y, m z ) C 3( X 3, Y 3 ) z Y t(X 0, Y 0,Z 0 ) R 1 Determine the Second Building Dimensions and Locations Unknown parameters – Building dimensions L, W, H – Building location X 3, Y 3 Known parameters – Camera pose – Building orientation Solution – Solve a set of linear equations Imaging Geometry Relationship x

Comparison with VP based Methods Using Identical Simulation Data Using same error for two methods – Additive random noise in endpoints of image segments – Principle points offsets Impact on the outputs from two methods – Camera pose – Geometry of the reconstructed buildings – Topology of the reconstructed buildings

Random Errors in Image Segments

Principle Point Offsets

Comparison Using Identical Real Data Digital Camera: Canon PowerShot SD750 Image size (3072x 2304 pixels) Image 2: Burnside Hall, McGill universityImage 1: Two boxes

Results from the Image 1 Unit: mmDimensions measured using a ruler Dimensions computed from the image Absolute errors Left Box Width V M Height V M Right Box Length V M Width V M49.61 Height V M

Results from the Image 2 Unit: meterDimensions from DWG fileDimensions computed from the image Length Width Height D model of Burnside Hall in Google Earth Visualization of the recovered camera pose and building

Conclusions Interactive solution to metric reconstruction from single images – Model-based approach but without model-to-image projection and readjustment procedure – better than vanishing point based methods Using an off-the-shelf camera, taking a picture, one can get object dimensions

Thank You!

Experimental Design Camera Parameters Building Parameters Focal Length (m) X 0 (m)Y 0 (m)Z 0 (m)Omg (degree) Phi ( degree) Kap ( degree) Building 1Building 2 Length (m) Width (m) Height (m) Orientation along X axis (degree) Location of a building model vertex (m)X3X Y3Y