Marc Levoy 1 Szymon Rusinkiewicz 1 Matt Ginzton 1 Jeremy Ginsberg 1 Kari Pulli 1 David Koller 1 Sean Anderson 1 Jonathan Shade 2 Brian Curless 2 Lucas Pereira 1 James Davis 1 Duane Fulk 3 1 Computer Science Department Stanford University 2 Department of Computer Science and Engineering University of Washington 3 Cyberware Inc. Presented by : Abhinav Dayal

Overview Motivations push 3D scanning technology push 3D scanning technology tool for art historians tool for art historians lasting archive lasting archive Technical goals scan a big statue scan a big statue capture chisel marks capture chisel marks capture reflectance capture reflectance 5 meters 1/4 mm 20,000:1 20,000 2  1 billion

Why capture chisel marks? Atlas (Accademia) ugnetto ?

Day (Medici Chapel) 2 mm

single scan of St. Matthew 1 mm

Issues Involved Scanning Scanning Scanner design Scanner design Scanning Procedure Scanning Procedure Post Processing Post Processing Range Processing Range Processing Color Processing Color Processing Handling large data sets Handling large data sets

Scanner Design Laser stripe triangulation system Laser stripe triangulation system Resolution and field of view Resolution and field of view Should capture chisel marks Should capture chisel marks Reasonable size of resulting dataset Reasonable size of resulting dataset Standoff and baseline Standoff and baseline Longer standoff  access to deeper recesses + safe distance from statue Longer standoff  access to deeper recesses + safe distance from statue Longer standoff  longer baseline  prone to miscalibration Longer standoff  longer baseline  prone to miscalibration

Color Acquisition Single pass Single pass 1D luminaire and 1D color sensor 1D luminaire and 1D color sensor Cross talk b/w luminaire & sensor  poor fidelity Cross talk b/w luminaire & sensor  poor fidelity RGB lasers RGB lasers Large and complex Large and complex Separate pass Separate pass They used broadband luminaire and separate sensor (Digital Camera – 1520 x 1144 pix res) They used broadband luminaire and separate sensor (Digital Camera – 1520 x 1144 pix res)

Camera Resolution and field For color to range calibration match respective standoffs For color to range calibration match respective standoffs One color per pixel resolution (per range data) One color per pixel resolution (per range data) Controlled illumination Controlled illumination Depth of field Depth of field DOF > FOV(Z) of range-camera DOF > FOV(Z) of range-camera

Gantry:Geometric design 4 motorized axes laser, range camera, white light, and color camera truss extensions for tall statues

Working volume of the scanner calibrated motions calibrated motions pitch (yellow) pitch (yellow) pan (blue) pan (blue) horizontal translation (orange) horizontal translation (orange) uncalibrated motions –vertical translation –remounting the scan head –moving the entire gantry

Scanning the David height of gantry: 7.5 meters weight of gantry: 800 kilograms

Calibration Range Calibration Color Calibration  Their calibration was complex and did moderately well Used a planar target with feature points to calculate camera’s intrinsic parameters and to build a per pixel intensity correction tableUsed a planar target with feature points to calculate camera’s intrinsic parameters and to build a per pixel intensity correction table

Scanning procedure Range Scanning Range Scanning Typical range involves several horizontally adjacent vertical sweeps or vice versa Typical range involves several horizontally adjacent vertical sweeps or vice versa Overlap adjacent sweeps by 40% Overlap adjacent sweeps by 40% Overlap adjacent shells by 15% Overlap adjacent shells by 15% Color Scanning Color Scanning Image with spotlight – Image without spotlight = Image with only spotlight Image with spotlight – Image without spotlight = Image with only spotlight

Safety Concerns energy deposition energy deposition Low and not a problem Low and not a problem avoiding collisions avoiding collisions manual motion controls manual motion controls automatic cutoff switches automatic cutoff switches one person serves as spotter one person serves as spotter surviving collisions surviving collisions pad the scan head pad the scan head

Range processing pipeline  steps  manual initial alignment  ICP to one existing scan  automatic ICP of all overlapping pairs  global relaxation to spread out error  merging using volumetric method (space carving)  problems  should have tracked the gantry location  ICP is unstable on smooth surfaces

Color processing pipeline  steps  compensate for ambient illumination  discard shadowed or specular pixels  map onto vertices – one color per vertex  correct for irradiance  diffuse reflectance  limitations  ignored interreflections  ignored subsurface scattering  treated diffuse as Lambertian  used aggregate surface normals

Handling large datasets  range images instead of polygon meshes  r(u,v) (special case of displacement map) yields 18:1 lossless compression (run- length encoding)  multiresolution using (range) image pyramid  lazy evaluation  viewer based on point rendering (Qsplat)

Range image pyramids Range image at 1x resolution Range image at 1/2x resolution Range image at 1/4x resolution Samples with any of its four children at the next finer pyramid level missing deleted Redden parents of missing children in proportion to the fraction of its missing children Can see holes at coarser resolutions

Problems faced Approximated marble as truly lambertian Approximated marble as truly lambertian Many unavoidable holes in the scan Many unavoidable holes in the scan Expensive and bulky gantry Expensive and bulky gantry Inadequate calibration Inadequate calibration Manual view planning  prone to errors Manual view planning  prone to errors Manual alignment of successive scans Manual alignment of successive scans

How optically cooperative is marble? systematic bias of 40 micronssystematic bias of 40 microns noise of 150 – 250 micronsnoise of 150 – 250 microns –worse at oblique angles of incidence –worse for polished statues

Removing the holes Space carving technique Space carving technique Create a maximum region of space consistent with the scans Create a maximum region of space consistent with the scans Creates a watertight surface Creates a watertight surface may lead to surfaces that are less plausible than smoothly extending the observed surfaces may lead to surfaces that are less plausible than smoothly extending the observed surfaces.  Recent work based upon Volumetric diffusion (Video) Video Reference: Filling holes in complex surfaces using volumetric diffusion: James Davis, Steven R. Marschner, Matt Garr, Marc Levoy, Computer Science Department, Stanford University (August, 2001)

Conclusion provides remarkable background for future developments provides remarkable background for future developments Provides massive mesh models Provides massive mesh models Discusses basic issues involved in 3D scanning Discusses basic issues involved in 3D scanning Many lessons learnt Many lessons learnt Encouraging many new applications (in areas such as: reverse engineering; industrial design; repair, reproduction, and improvement of machinery; medical diagnostics, analysis and simulation; 3D photography; and building rich virtual environments) Encouraging many new applications (in areas such as: reverse engineering; industrial design; repair, reproduction, and improvement of machinery; medical diagnostics, analysis and simulation; 3D photography; and building rich virtual environments) This This is just a beginning QUESTIONS?