1. 3D Scanning

2. Computer Graphics Pipeline Human time = expensiveHuman time = expensive Sensors = cheapSensors = cheap – Computer graphics increasingly relies on measurements of the real world RenderingRendering ShapeShape LightingandReflectanceLightingandReflectance MotionMotion Shape 3D Scanning

3. 3D Scanning Applications Computer graphicsComputer graphics Product inspectionProduct inspection Robot navigationRobot navigation As-built floorplansAs-built floorplans Product design Archaeology Clothes fitting Art history

4. Industrial Inspection Determine whether manufactured parts are within tolerancesDetermine whether manufactured parts are within tolerances

5. Medicine Plan surgery on computer model, visualize in real timePlan surgery on computer model, visualize in real time

6. Medicine

7. Medicine

8. Medicine

9. Scanning Buildings Quality control during constructionQuality control during construction As-built modelsAs-built models

10. Scanning Buildings Quality control during constructionQuality control during construction As-built modelsAs-built models

11. Clothing Scan a person, custom-fit clothingScan a person, custom-fit clothing U.S. Army; booths in mallsU.S. Army; booths in malls

12. The Digital Michelangelo Project

13. Why Scan Sculptures? Sculptures interesting objects to look atSculptures interesting objects to look at Introduce scanning to new disciplinesIntroduce scanning to new disciplines – Art: studying working techniques – Art history – Cultural heritage preservation – Archeology High-visibility projectHigh-visibility project

14. Goals Scan 10 sculptures by MichelangeloScan 10 sculptures by Michelangelo High-resolution (“quarter-millimeter”) geometryHigh-resolution (“quarter-millimeter”) geometry Side projects: architectural scanning (Accademia and Medici chapel), scanning fragments of Forma Urbis RomaeSide projects: architectural scanning (Accademia and Medici chapel), scanning fragments of Forma Urbis Romae

15. Why Capture Chisel Marks? Atlas (Accademia) ugnettougnetto ? ?

16. Why Capture Chisel Marks as Geometry? Why Capture Chisel Marks as Geometry? Day (Medici Chapel) 2 mm

17. Side project: The Forma Urbis Romae

18. Forma Urbis Romae Fragment Forma Urbis Romae Fragment side face

20. Range Acquisition Taxonomy Range acquisition Contact Transmissive Reflective Non-optical Optical Industrial CT Mechanical (CMM, jointed arm) Radar Sonar Ultrasound MRI Ultrasonic trackers Magnetic trackers Inertial (gyroscope, accelerometer)

21. Range Acquisition Taxonomy Optical methods Passive Active Shape from X: stereomotionshadingtexturefocusdefocus Active variants of passive methods Stereo w. projected texture Active depth from defocus Photometric stereo Time of flight Triangulation

22. Touch Probes Jointed arms with angular encodersJointed arms with angular encoders Return position, orientation of tipReturn position, orientation of tip Faro Arm – Faro Technologies, Inc.

23. Stereo Find feature in one image, search along epipolar line in other image for correspondenceFind feature in one image, search along epipolar line in other image for correspondence

24. Why More Than 2 Views? BaselineBaseline – Too short – low accuracy – Too long – matching becomes hard

25. Why More Than 2 Views? Ambiguity with 2 viewsAmbiguity with 2 views Camera 1 Camera 2 Camera 3

26. Multibaseline Stereo [ Okutomi & Kanade]

27. Shape from Motion “Limiting case” of multibaseline stereo“Limiting case” of multibaseline stereo Track a feature in a video sequenceTrack a feature in a video sequence For n frames and f features, have 2  n  f knowns, 6  n +3  f unknownsFor n frames and f features, have 2  n  f knowns, 6  n +3  f unknowns

28. Shape from Shading Given: image of surface with known, constant reflectance under known point lightGiven: image of surface with known, constant reflectance under known point light Estimate normals, integrate to find surfaceEstimate normals, integrate to find surface Problem: ambiguityProblem: ambiguity

29. Shape from Shading Advantages:Advantages: – Single image – No correspondences – Analogue in human vision Disadvantages:Disadvantages: – Mathematically unstable – Can’t have texture “Photometric stereo” (active method) more practical than passive version“Photometric stereo” (active method) more practical than passive version

30. Shape from Texture Mathematically similar to shape from shading, but uses stretch and shrink of a (regular) textureMathematically similar to shape from shading, but uses stretch and shrink of a (regular) texture

31. Shape from Focus and Defocus Shape from focus: at which focus setting is a given image region sharpest?Shape from focus: at which focus setting is a given image region sharpest? Shape from defocus: how out-of-focus is each image region?Shape from defocus: how out-of-focus is each image region? Passive versions rarely usedPassive versions rarely used Active depth from defocus can be made practicalActive depth from defocus can be made practical

32. Active Variants of Passive Techniques Regular stereo with projected textureRegular stereo with projected texture – Provides features for correspondence Active depth from defocusActive depth from defocus – Known pattern helps to estimate defocus Photometric stereoPhotometric stereo – Shape from shading with multiple known lights

33. Pulsed Time of Flight Basic idea: send out pulse of light (usually laser), time how long it takes to returnBasic idea: send out pulse of light (usually laser), time how long it takes to return

34. Pulsed Time of Flight Advantages:Advantages: – Large working volume (up to 100 m.) Disadvantages:Disadvantages: – Not-so-great accuracy (at best ~5 mm.) Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds Does not scale with working volumeDoes not scale with working volume Often used for scanning buildings, rooms, archeological sites, etc.Often used for scanning buildings, rooms, archeological sites, etc.

35. Triangulation Project laser stripe onto objectProject laser stripe onto object Object Laser CameraCamera

36. CameraCamera Triangulation Depth from ray-plane triangulationDepth from ray-plane triangulation Laser (x,y) Object

37. Triangulation: Moving the Camera and Illumination Moving independently leads to problems with focus, resolutionMoving independently leads to problems with focus, resolution Most scanners mount camera and light source rigidly, move them as a unitMost scanners mount camera and light source rigidly, move them as a unit

38. Triangulation: Moving the Camera and Illumination

40. Scanning a Large Object Calibrated motionsCalibrated motions – pitch (yellow) – pan (blue) – horizontal translation (orange) Uncalibrated motions – vertical translation – rolling the gantry – remounting the scan head

41. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method

42. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method

43. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method

44. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method

45. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method +

46. Range Processing Pipeline StepsSteps 1. manual initial alignment 2. ICP to one existing scan 3. automatic ICP of all overlapping pairs 4. global relaxation to spread out error 5. merging using volumetric method

47. Statistics About the Scan of David 480 individually aimed scans480 individually aimed scans 0.3 mm sample spacing0.3 mm sample spacing 2 billion polygons2 billion polygons 7,000 color images7,000 color images 32 gigabytes32 gigabytes 30 nights of scanning30 nights of scanning 22 people22 people

48. Head of Michelangelo’s David Photograph 1.0 mm computer model