1. Depth from Structured Light II: Error Analysis
Michael H. Rosenthal
April 19th, 2000

2. Outline Review of Structured Light Projection Error in Ideal Model
Outline
Review of Structured Light Projection
Error in Ideal Model
Lighting and Surface Property Errors
Stripe Assignment Errors
Motion Errors
Structured Light in Practice

3. Structured Light Projection
Structured Light Projection
Camera and projector are calibrated
Light stripes are projected onto scene
Depth for each pixel is measured by intersecting pixel ray with stripe plane

4. Binary Stripe Encoding
Binary Stripe Encoding
Pixels are “labeled” using binary encoding: light = 1, dark = 0
High-order bits encode broad regions, low-order bits encode fine location

5.
Error in Ideal Model

6. Error in Ideal Model Pixels and stripes have finite size
Error in Ideal Model
Pixels and stripes have finite size
All visible surfaces within the pixel/stripe intersection volume are sampled and lumped together

7.
Error in Ideal Model
Size of sampled volume depends upon stripe width, pixel width and projector orientation
Corollary: depth resolution is controlled by the same factors

8. Error from Lighting and Reflection
Error from Lighting and Reflection

9. Lighting and Reflection
Lighting and Reflection
Prior assumptions:
Bright Lambertian surface
Path from projector to surface to camera
Real world
Dark or shiny materials (apples, metal, cloth)
Things in your way
Specular highlights interfere with structured light patterns

10. Lighting and Reflection
Lighting and Reflection
Preventive Methods:
Capture images with projector fully on and fully off
Use difference to measure intensity range for each pixel
Threshold difference to find to find trouble spots
Images from structured light projected into a simple scene

11. Lighting and Reflection
Lighting and Reflection
Minimum intensity image may have bright spots - specular highlights
Maximum intensity image may have dark spots - shadows or dark objects

12. Lighting and Reflection
Lighting and Reflection
Difference image shows range of intensity between on and off
Narrow range corresponds to non-ideal behavior
Threshold to identify trouble spots
Exclude trouble spots from final depth image

13. Lighting and Reflection
Lighting and Reflection
Other practical solutions:
Lambertian coatings and sprays (talc)
High dynamic range cameras
Variable exposure times

14. Error in Stripe Assignment
Error in Stripe Assignment

15. Stripe Assignment Errors
Stripe Assignment Errors
Stripe encoding requires binary decision for each pixel
What happens if a pixel is misassigned to the wrong stripe (i.e. gets a 0 rather than a 1 when illuminated)?
Images from structured light projected into a simple scene

16. Stripe Assignment Errors
Stripe Assignment Errors
Caused by random noise, low contrast, motion, edge uncertainty, etc.
Magnitude depends upon significance of bit!
1st bit yields error N/2, 2nd bit yields error N/4, kth bit yields error N/2k
Binary stripe encoding (phase shifted method)

17. Stripe Assignment Errors
Stripe Assignment Errors
Assume that probability of misassignment is p
Expected error from kth bit is p*N/2k
Total expected error E = p*N/2 + p * N/4 +… + p * N/2k + … + p*N/2logN
E = p*N*(1- 1/2logN)
For large N, E ~ p*N
For < 1% expected error, we must have < 1% chance of misassignment

18.
Error from Motion

19. Moving object in a structured light system
Motion Errors
What errors can motion cause in a structured light system?
Two components of motion:
Along a pixel ray
Pixel to pixel
Moving object in a structured light system

20. Moving object in a structured light system
Pixel to Pixel Motion
Similar to stripe assignment error - “random” bits are flipped in affected pixels
Moving object in a structured light system

21. Motion along a pixel ray traverses projected stripes over time
Motion on a Pixel Ray
Motion along a pixel ray causes different stripes to be reflected over time
Final pixel code is composed of bits from different positions
Motion along a pixel ray traverses projected stripes over time

22.
Motion on a Pixel Ray
Magnitude of error depends upon initial position and velocity
Very large errors near edges in broad stripes (high order bits)

23.
Motion on a Pixel Ray
Absolute error relative to final object position - note the chaotic dependence upon position and velocity

24. Structured Light in Practice
Structured Light in Practice
Office of the Future uses it for surface measurement
Augmented Reality Biopsy project has tested it for skin surface measurement and for internal surgery
Videos!

25. Conclusions Four major classes of error in structured light:
Conclusions
Four major classes of error in structured light:
Error intrinsic to ideal system - affects spatial resolution
Lighting and reflection error - some regions of the scene will be unmeasurable
Stripe assignment error - need reliable methods for binary categorization
Motion error - yields large, unpredictable errors
Despite these problems, structured light is useful and widely available