Triangulation Scanner Design Options

Triangulation System Options Single-stripe systems most robust, but slowest To go faster, project multiple stripes But which stripe is which? In limit, project 2D pattern, determine projector/camera correspondence

Time-Coded Light Patterns Assign each stripe a unique illumination code over time [Posdamer 82] Time Just as an example, here’s a very simple code based on binary numbers. If you look at a single position in space (i.e., a single pixel), you see a certain on/off pattern over these four frames. This conveys a code that tells you which stripe you are looking at, which gives you a plane in space with which you can then triangulate to find depths. Space

Gray-Code Patterns To minimize effects of quantization error: each point may be a boundary only once Time Space

Accounting for Reflectance Because of surface reflectance and ambient light, distinguishing between black and white not always easy Solution: project all-white (and sometimes all-black) frame Permits multiple shades of gray

Multiple Shades of Gray

Multiple Shades of Gray

Intensity Wedges Limiting case: intensity wedges

Temporal vs. Spatial Continuity Structured-light systems make certain assumptions about the scene: Temporal continuity assumption: Assume scene is static Assign stripes a code over time Spatial continuity assumption: Assume scene is one object Project a grid, pattern of dots, etc.

Grid Methods Assume exactly one continuous surface Count dots or grid lines Occlusions cause problems Some methods use dynamic programming [Maas]

Codes Assuming Local Spatial Continuity Codeword

Codes Assuming Local Spatial Continuity [Zhang] [Ozturk]

Spatio-Temporal Continuity Another possible assumption: Object may move Velocity low enough to permit tracking “Spatio-temporal” continuity

Designing a Code for Moving Scenes Projector Camera

Designing a Code for Moving Scenes t

Designing a Code for Moving Scenes Codeword t

Codes for Moving Scenes Assign time codes to stripe boundaries Perform frame-to-frame tracking of corresponding boundaries Propagate illumination history [Hall-Holt & Rusinkiewicz, ICCV 2001] In our system, though, the objects are moving from frame to frame, so if you just used the simple algorithm you might get half of one code and half of another, and come up with the wrong code. So, we need to do some tracking from frame to frame. In fact, we do this based on looking at the boundaries between stripes. The stripes are tracked from frame to frame, and at the end of the day the illumination on both sides of the boundary is what conveys the code. Illumination history = (WB),(BW),(WB) Code

Designing a Code Want many “features” to track: lots of black/white edges at each frame Try to minimize ghosts – WW or BB “boundaries” that can’t be seen directly There’s an extra little wrinkle, though. If you’re looking for the boundaries between stripes, sometimes you have a “boundary” between two stripes of the same color, which we’ll call a ghost. So, some stripes are easy to track, because you can see them in both frames, but in some cases you have to infer the presence of a stripe you can’t see directly. In order to make this at all feasible, we have to design a code that tries to minimize these ghosts and ensures, for example, that a ghost in one frame becomes visible in the next.

Designing a Code Design a code to make tracking possible: Do not allow two spatially adjacent ghosts Do not allow two temporally adjacent ghosts t

Designing a Code Graph (for 4 frames): Nodes: stripes (over time) 0011 1110 1011 0110 0100 1001 0001 1100 0000 1101 1010 0111 1000 0101 1111 0010 Edges: boundaries (over time) Nodes: stripes (over time) Space Time

Designing a Code Graph (for 4 frames): 0011 1110 1011 0110 0100 1001 0001 1100 0000 1101 1010 0111 1000 0101 1111 0010 Nodes: stripes (over time) Boundary visible at even times Boundary visible at odd times Edges: boundaries (over time) Path with alternating colors: 55 edges in graph  maximal-length traversal has 110 boundaries (111 stripes)

Designing a Code Many solutions to the graph problem as stated, so we can add more constraints Maximize effect of errors No static boundaries Even distribution of stripes of width 1 and 2

Implementation Pipeline: DLP projector illuminates scene @ 60 Hz. Synchronized NTSC camera captures video Pipeline returns range images @ 60 Hz. Project Code Capture Images Find Boundaries Match Boundaries Decode Compute Range So here’s how the range scanner looks. We project stripes at 60 Hz., capture video frames at the same rate, and process them to get range images at 60 Hz.

Results Video frames Stripe boundaries unknown known ghosts

More Options: Shadow Scanning Variant of single-stripe triangulation Use a simple lamp, stick to create a shadow [Bouguet]

More Options: Active Stereo Benefit: only camera/camera calibration, not camera/projector Projection possibilities: Random texture Single stripe Others (space-time codes?) [Davis]