1. Rogerio Feris 1, Ramesh Raskar 2, Matthew Turk 1
Dealing with Multiscale Depth Changes and Motion in Depth Edge Detection
Rogerio Feris 1, Ramesh Raskar 2, Matthew Turk 1
1 University of California, Santa Barbara
2 Mitsubishi Electric Research Labs

2. Motivation Intensity Discontinuities (Edge Methods)
- Limited in their ability to reveal scene structure
- Shape boundaries are not captured in low-contrast scenes

3. Motivation Why this is an important problem?
Ideally, we want to detect edges according to their physical origin (discontinuities in depth, surface normal, albedo, illumination and motion)
Our work is focused on the detection and modeling of depth discontinuities, also known as depth edges
Why this is an important problem?

4. Motivation
Input Image
Canny Edges
Depth Edges
Depth edges correspond to sharp discontinuities in depth. They outline shape boundaries and are directly tied to the 3D scene geometry.
Useful for many computer vision applications such as segmentation, stereo, and recognition.

5. Motivation Can we find depth edges without full 3D reconstruction?
Obvious way to detect depth edges: acquire depth map and then find discontinuities
Depth recovery methods are noisy at discontinuities
Can we find depth edges without full 3D reconstruction?

6. Depth Edges with MultiFlash
Raskar, Tan, Feris, Yu, Turk – ACM SIGGRAPH 2004

11. Shadow-Free Depth Edges Shadow-Free Depth Edges
Bottom Flash Top Flash Left Flash Right Flash
Shadow-Free
Ratio images and directions of epipolar traversal
Depth Edges
Shadow-Free
Depth Edges

12. Limitations Outdoor Scenes Lack of Background Specular Reflections
Transparent or Low albedo surfaces
Thin narrow objects
Lack of Background
Dynamic Scenes

13. Varying Illumination Parameters

14. Multiscale Depth Changes
Small Baseline
Shadow Missed
Large Baseline
Shadow Detaches
Baseline Tradeoff

15. A MultiBaseline Approach

16. Multiscale Depth Changes
Small Baseline
Shadow Missed
Large Baseline
Shadow Detaches
Minimum Image

17. Small Baseline
Our Final Result
Large Baseline

18. Input Image
Canny Edges

19. Single Baseline
Our MultiBaseline Approach

20. Multiscale Depth Changes
Drawbacks:
Slow acquisition time
Detached shadows caused by small baseline flash
Solution: Linear Light Sources
Linear Light

21. Multiscale Depth Changes
Image Capture
Ratio Right Flash
Ratio Bottom Flash
Depth Edges

22. Multiscale Depth Changes
Linear Light Drawbacks:
More sensitive to noise
Depth edges lying in shadowed regions

23. Flash
NoFlash
No Background

24. Flash
NoFlash
Flash NoFlash

25. Varying Wavelength Light sources are triggered at the same time!
Colored shadows are explored to detect depth edges

26. Varying Wavelength
Colored Light
White Light

27. Varying Wavelength
Colored Light
Colored Light / White Light

28. Varying Wavelength Depth edges from a single image
Learning Shadow Color Transitions
Depth Edges with variable Wavelength Canny Edges

30. Conclusions
Variation of illumination parameters (spatial position, number, type, and wavelength of light sources) for robust depth edge detection
- A novel multibaseline approach for multiscale depth edge detection
- A novel method to detect depth edges in motion with variable wavelength light sources

31. Limitations Future Work Limited to indoor scenes
Depth edges in motion with one-shot photography only for specific applications, not for general scenes.
Future Work
Extend this framework to detect other physical discontinuities (surface normal, albedo, illumination, and motion)

32. Thank you !