1. Overview of 3D Scanners Acknowledgement: some content and figures by Brian Curless

2. 3D Data Types Point DataPoint Data Volumetric DataVolumetric Data Surface DataSurface Data

3. 3D Data Types: Point Data “Point clouds”“Point clouds” Advantage: simplest data typeAdvantage: simplest data type Disadvantage: no information on adjacency / connectivityDisadvantage: no information on adjacency / connectivity

4. 3D Data Types: Volumetric Data Regularly-spaced grid in (x,y,z): “voxels”Regularly-spaced grid in (x,y,z): “voxels” For each grid cell, storeFor each grid cell, store – Occupancy (binary: occupied / empty) – Density – Other properties Popular in medical imagingPopular in medical imaging – CAT scans – MRI

5. 3D Data Types: Volumetric Data Advantages:Advantages: – Can “see inside” an object – Uniform sampling: simpler algorithms Disadvantages:Disadvantages: – Lots of data – Wastes space if only storing a surface – Most “vision” sensors / algorithms return point or surface data

6. 3D Data Types: Surface Data PolyhedralPolyhedral – Piecewise planar – Polygons connected together – Most popular: “triangle meshes” SmoothSmooth – Higher-order (quadratic, cubic, etc.) curves – Bézier patches, splines, NURBS, subdivision surfaces, etc.

7. 3D Data Types: Surface Data Advantages:Advantages: – Usually corresponds to what we see – Usually returned by vision sensors / algorithms Disadvantages:Disadvantages: – How to find “surface” for translucent objects? – Parameterization often non-uniform – Non-topology-preserving algorithms difficult

8. 3D Data Types: Surface Data Implicit surfaces (cf. parametric)Implicit surfaces (cf. parametric) – Zero set of a 3D function – Usually regularly sampled (voxel grid) Advantage: easy to write algorithms that change topologyAdvantage: easy to write algorithms that change topology Disadvantage: wasted space, timeDisadvantage: wasted space, time

9. 2½-D Data Image: stores an intensity / color along each of a set of regularly-spaced rays in spaceImage: stores an intensity / color along each of a set of regularly-spaced rays in space Range image: stores a depth along each of a set of regularly-spaced rays in spaceRange image: stores a depth along each of a set of regularly-spaced rays in space Not a complete 3D description: does not store objects occluded (from some viewpoint)Not a complete 3D description: does not store objects occluded (from some viewpoint) View-dependent scene descriptionView-dependent scene description

10. 2½-D Data This is what most sensors / algorithms really returnThis is what most sensors / algorithms really return AdvantagesAdvantages – Uniform parameterization – Adjacency / connectivity information DisadvantagesDisadvantages – Does not represent entire object – View dependent

11. 2½-D Data Range imagesRange images Range surfacesRange surfaces Depth imagesDepth images Depth mapsDepth maps Height fieldsHeight fields 2½-D images2½-D images Surface profilesSurface profiles xyz maps xyz maps …

12. Related Fields Computer VisionComputer Vision – Passive range sensing – Rarely construct complete, accurate models – Application: recognition MetrologyMetrology – Main goal: absolute accuracy – High precision, provable errors more important than scanning speed, complete coverage – Applications: industrial inspection, quality control, as-built models

13. Related Fields Computer GraphicsComputer Graphics – Often want complete model – Low noise, geometrically consistent model more important than absolute accuracy – Application: animated CG characters

14. Terminology Range acquisition, shape acquisition, rangefinding, range scanning, 3D scanningRange acquisition, shape acquisition, rangefinding, range scanning, 3D scanning Alignment, registrationAlignment, registration Surface reconstruction, 3D scan merging, scan integration, surface extractionSurface reconstruction, 3D scan merging, scan integration, surface extraction 3D model acquisition3D model acquisition

15. Range Acquisition Taxonomy Range acquisition Contact Transmissive Reflective Non-optical Optical Industrial CT Mechanical (CMM, jointed arm) Radar Sonar Ultrasound MRI Ultrasonic trackers Magnetic trackers Inertial (gyroscope, accelerometer)

16. Range Acquisition Taxonomy Optical methods Passive Active Shape from X: stereomotionshadingtexturefocusdefocus Active variants of passive methods Stereo w. projected texture Active depth from defocus Photometric stereo Time of flight Triangulation

17. Touch Probes Jointed arms with angular encodersJointed arms with angular encoders Return position, orientation of tipReturn position, orientation of tip Faro Arm – Faro Technologies, Inc.

18. Optical Range Acquisition Methods Advantages:Advantages: – Non-contact – Safe – Usually inexpensive – Usually fast Disadvantages:Disadvantages: – Sensitive to transparency – Confused by specularity and interreflection – Texture (helps some methods, hurts others)

19. Stereo Find feature in one image, search along epipolar line in other image for correspondenceFind feature in one image, search along epipolar line in other image for correspondence

20. Stereo Advantages:Advantages: – Passive – Cheap hardware (2 cameras) – Easy to accommodate motion – Intuitive analogue to human vision Disadvantages:Disadvantages: – Only acquire good data at “features” – Sparse, relatively noisy data (correspondence is hard) – Bad around silhouettes – Confused by non-diffuse surfaces Variant: multibaseline stereo to reduce ambiguityVariant: multibaseline stereo to reduce ambiguity

21. Why More Than 2 Views? BaselineBaseline – Too short – low accuracy – Too long – matching becomes hard

22. Why More Than 2 Views? Ambiguity with 2 viewsAmbiguity with 2 views Camera 1 Camera 2 Camera 3

23. Multibaseline Stereo [ Okutami & Kanade]

24. Shape from Motion “Limiting case” of multibaseline stereo“Limiting case” of multibaseline stereo Track a feature in a video sequenceTrack a feature in a video sequence For n frames and f features, have 2  n  f knowns, 6  n +3  f unknownsFor n frames and f features, have 2  n  f knowns, 6  n +3  f unknowns

25. Shape from Motion Advantages:Advantages: – Feature tracking easier than correspondence in far- away views – Mathematically more stable (large baseline) Disadvantages:Disadvantages: – Does not accommodate object motion – Still problems in areas of low texture, in non-diffuse regions, and around silhouettes

26. Shape from Shading Given: image of surface with known, constant reflectance under known point lightGiven: image of surface with known, constant reflectance under known point light Estimate normals, integrate to find surfaceEstimate normals, integrate to find surface Problem: ambiguityProblem: ambiguity

27. Shape from Shading Advantages:Advantages: – Single image – No correspondences – Analogue in human vision Disadvantages:Disadvantages: – Mathematically unstable – Can’t have texture “Photometric stereo” (active method) more practical than passive version“Photometric stereo” (active method) more practical than passive version

28. Shape from Texture Mathematically similar to shape from shading, but uses stretch and shrink of a (regular) textureMathematically similar to shape from shading, but uses stretch and shrink of a (regular) texture

29. Shape from Texture Analogue to human visionAnalogue to human vision Same disadvantages as shape from shadingSame disadvantages as shape from shading

30. Shape from Focus and Defocus Shape from focus: at which focus setting is a given image region sharpest?Shape from focus: at which focus setting is a given image region sharpest? Shape from defocus: how out-of-focus is each image region?Shape from defocus: how out-of-focus is each image region? Passive versions rarely usedPassive versions rarely used Active depth from defocus can be made practicalActive depth from defocus can be made practical

31. Active Optical Methods Advantages:Advantages: – Usually can get dense data – Usually much more robust and accurate than passive techniques Disadvantages:Disadvantages: – Introduces light into scene (distracting, etc.) – Not motivated by human vision

32. Active Variants of Passive Techniques Regular stereo with projected textureRegular stereo with projected texture – Provides features for correspondence Active depth from defocusActive depth from defocus – Known pattern helps to estimate defocus Photometric stereoPhotometric stereo – Shape from shading with multiple known lights

33. Pulsed Time of Flight Basic idea: send out pulse of light (usually laser), time how long it takes to returnBasic idea: send out pulse of light (usually laser), time how long it takes to return

34. Pulsed Time of Flight Advantages:Advantages: – Large working volume (up to 100 m.) Disadvantages:Disadvantages: – Not-so-great accuracy (at best ~5 mm.) Requires getting timing to ~30 picosecondsRequires getting timing to ~30 picoseconds Does not scale with working volumeDoes not scale with working volume Often used for scanning buildings, rooms, archeological sites, etc.Often used for scanning buildings, rooms, archeological sites, etc.

35. AM Modulation Time of Flight Modulate a laser at frequency m, it returns with a phase shift Modulate a laser at frequency m, it returns with a phase shift  Note the ambiguity in the measured phase!  Range ambiguity of 1 / 2 m nNote the ambiguity in the measured phase!  Range ambiguity of 1 / 2 m n

36. AM Modulation Time of Flight Accuracy / working volume tradeoff (e.g., noise ~ 1 / 500 working volume)Accuracy / working volume tradeoff (e.g., noise ~ 1 / 500 working volume) In practice, often used for room-sized environments (cheaper, more accurate than pulsed time of flight)In practice, often used for room-sized environments (cheaper, more accurate than pulsed time of flight)

37. Triangulation

38. Triangulation: Moving the Camera and Illumination Moving independently leads to problems with focus, resolutionMoving independently leads to problems with focus, resolution Most scanners mount camera and light source rigidly, move them as a unitMost scanners mount camera and light source rigidly, move them as a unit

39. Triangulation: Moving the Camera and Illumination

41. Triangulation: Extending to 3D Possibility #1: add another mirror (flying spot)Possibility #1: add another mirror (flying spot) Possibility #2: project a stripe, not a dotPossibility #2: project a stripe, not a dot Object Laser CameraCamera

42. Triangulation Scanner Issues Accuracy proportional to working volume (typical is ~1000:1)Accuracy proportional to working volume (typical is ~1000:1) Scales down to small working volume (e.g. 5 cm. working volume, 50  m. accuracy)Scales down to small working volume (e.g. 5 cm. working volume, 50  m. accuracy) Does not scale up (baseline too large…)Does not scale up (baseline too large…) Two-line-of-sight problem (shadowing from either camera or laser)Two-line-of-sight problem (shadowing from either camera or laser) Triangulation angle: non-uniform resolution if too small, shadowing if too big (useful range: 15  -30  )Triangulation angle: non-uniform resolution if too small, shadowing if too big (useful range: 15  -30  )

43. Triangulation Scanner Issues Material properties (dark, specular)Material properties (dark, specular) Subsurface scatteringSubsurface scattering Laser speckleLaser speckle Edge curlEdge curl Texture embossingTexture embossing

44. Multi-Stripe Triangulation To go faster, project multiple stripesTo go faster, project multiple stripes But which stripe is which?But which stripe is which? Answer #1: assume surface continuityAnswer #1: assume surface continuity

45. Multi-Stripe Triangulation To go faster, project multiple stripesTo go faster, project multiple stripes But which stripe is which?But which stripe is which? Answer #2: colored stripes (or dots)Answer #2: colored stripes (or dots)

46. Multi-Stripe Triangulation To go faster, project multiple stripesTo go faster, project multiple stripes But which stripe is which?But which stripe is which? Answer #3: time-coded stripesAnswer #3: time-coded stripes

47. Time-Coded Light Patterns Assign each stripe a unique illumination code over time [Posdamer 82]Assign each stripe a unique illumination code over time [Posdamer 82] Space Time