Interactively Modeling with Photogrammetry Pierre Poulin Mathieu Ouimet Marie-Claude Frasson Dép. Informatique et recherche opérationnelle Université de Montréal

Motivation Photo-realism is difficult to achieve Important recent progress in rendering Acquiring realistic 3D models is still a major hurtle Important needs for realism, special effects in movies, CAR, etc. Extracting 3D models from photographs

Computer Vision / Robotics 3D models do not satisfy most of the visual accuracy necessary in graphics Fully automatic systems are challenging : –false correspondences –missed edge detections –noise –textures Provide much inspiration in our system

Our Interactive Reconstruction System User knows the 3D models / textures User is responsible for everything User interactions : –User draws 2D primitives –User puts the 2D primitives in correspondences –User adds 3D constraints –User extracts a unified texture

Drawing 2D Primitives

Correspondences

3D Constraints Perpendicularity Parallelism Co-planarity

Extracted Textures

Reconstruction Process Incremental Robust Intuitive Provides good graphics models Labor-intensive

The Camera Our camera is a transformation matrix No explicit need for real camera parameters

Reconstructing a Camera 6 or more 2D-to-3D point correspondences (0,1,0) (0,1,1) (1,1,0) (0,0,1) (1,0,0) (1,0,1)

Reconstructing a Camera Least-squares to compute all T i Solution with SVD –Fast –Robust –Always provides a solution –Conditions for accuracy similar to non-linear

Reconstructing a 3D Point Incidence of 3D point on planes Least-squares to compute each (x,y,z) Polygons as set of 3D points

Reconstructing a 3D Line Plücker coordinates of a 3D line

Additional 3D Constraints Co-planarity Parallelism Perpendicularity Weights can be used to alter the importance of certain constraints Weights

Iterating Better cameras give better 3D geometry Better 3D geometry give better cameras Iterations between the two improve both

Convergence

Recovering Texel Colors u v u v t s Texture map3D Polygon 2D Images t t s s

Occlusion Testing Zones of Occlusion 3D Model 2D Image

Linear Fit Misalignments due to imprecisions in the 3D model and its cameras 2D transformation matrix using least-squares

Unifying Texel Criteria Clustering to discriminate view-dependent colors for a texel Other metrics used to weight valid texels : –Projected area (adaptive sampling) –Texture quality

Two Scenes with Cubes

Desktop

Lego Tower

Coffee Pot

Conclusions User knows best Satisfying 3D models and extracted textures Labor-intensive

Future Work Better user interface Error detection Radiances, reflectances, and global illumination Displacement maps on 3D primitives Bounds on reconstructed information