1. A Bayesian Approach For 3D Reconstruction From a Single Image
Presented By: Erick Delage
Supervisor: Prof. Andrew Y. Ng
AI Laboratory, Stanford University

2. Autonomous Monocular Vision Depth Reconstruction for Indoor Image
Main problem presentation
Can a robot reconstruct 3D from a single image?
Erick Delage, Stanford University, 2005

3. Review of Publications
Popular 3d reconstruction
Stereo Vision (Trucco et Verri, 1998)
Structure from Motion
Single View 3d reconstruction
Shape from Shading (Zhang et al., 1999)
3d Metrology (Criminisi et al., 2000)
Our Goal
To develop an autonomous algorithm that recovers 3D information from a single image in a complex environment
Shape from stereovision (dependent on base line)
State of the art in single view (shape from shading, Single view metrology)
Limitations
Compare to us leads to idea of learning prior knowledge for depth reconstruction
Erick Delage, Stanford University, 2005

4. Simplification of the Problem
Assumptions
Image contains flat floor and walls
Camera is parallel to the ground plane
The camera is at a known height above the ground
The image is obtained by perspective projection
Our Theory:
Given the floor boundary position, the 3D coordinates in an image of all points can be recovered
Erick Delage, Stanford University, 2005

5. Erick Delage, Stanford University, 2005
General Approach
Show images that present the whole approach
Orig -> detected boundary -> 3d recon
Erick Delage, Stanford University, 2005

6. Erick Delage, Stanford University, 2005
General Approach (2)
Prior Knowledge about Indoor + Machine Learning
Image Analysis
Floor Boundary detection (Machine Learning)
3D reconstruction
Show images that present the whole approach
Orig -> detected boundary -> 3d recon
Erick Delage, Stanford University, 2005

7. Floor Boundary Detection
Magnitude of Image gradient
Difference in chromatic space
Input Image
Difference from the floor color
How can we combine these image features for floor boundary detection ?
Erick Delage, Stanford University, 2005

8. Floor Boundary Detection
Input Image
Using Logistic Regression :
(Martin, D. R., et al., 2002)
Training Mask
The model was trained using 25 labeled images of a diverse range of indoor environments on Stanford’s campus
Develop and train a logistic model of the probability that a point in the image is part of the edge of the floor based on the evaluation of local feature functions
Martin, D. R., Fowlkes, C. C., & Malik, J. (2002). Learning to Detect Natural Image Boundaries using Brightness and Texture. NIPS.
Erick Delage, Stanford University, 2005

9. Floor Boundary Detection : Results
Trade-off between accuracy and noise as detector threshold varies. Precision is the fraction of detections which are true positives, while recall is the fraction of positives that are detected
Erick Delage, Stanford University, 2005

10. Floor Boundary Detection : Results
Trade-off between accuracy and noise as detector threshold varies. Precision is the fraction of detections which are true positives, while recall is the fraction of positives that are detected
Precision = “true positives” / “all positives”
Recall = “true positives” / “all true’s”
Erick Delage, Stanford University, 2005

11. Bayesian Inference on Floor Boundary
Can we use prior knowledge about the structure of floors and their boundaries?
Present image with little squares that explains how boundary evolve
Erick Delage, Stanford University, 2005

12. Erick Delage, Stanford University, 2005
Bayesian Inference Y1 D1 C X1 Y3 D3 X3 YN DN XN …
Di : Direction of the floor boundary in column i
Yi : Position of floor boundary in column i
Xi : Local image features
C : Color of the floor
Erick Delage, Stanford University, 2005

13. Erick Delage, Stanford University, 2005
Bayesian Inference D1 D1 D3 … DN Y1 Y1 Y3 … YN X1 X1 X3 … XN … C
: initial distribution of variables
Erick Delage, Stanford University, 2005

14. Erick Delage, Stanford University, 2005
Bayesian Inference D1 D1 D3 … DN Y1 Y1 Y3 … YN X1 X1 X3 … XN … C
Erick Delage, Stanford University, 2005

15. Erick Delage, Stanford University, 2005
Bayesian Inference D1 D1 D3 … DN Y1 Y1 Y3 … YN X1 X1 X3 … XN … C
Erick Delage, Stanford University, 2005

16. Erick Delage, Stanford University, 2005
Bayesian Inference D1 D1 D3 … DN Y1 Y1 Y3 … YN X1 X1 X3 … XN … C
from the detection algorithm
Erick Delage, Stanford University, 2005

17. Erick Delage, Stanford University, 2005
Bayesian Inference D1 D1 D3 … DN Y1 Y1 Y3 … YN X1 X1 X3 … XN … C
Erick Delage, Stanford University, 2005

18. Training / Bayesian Inference
60 images of indoor environment in 8 different buildings of Stanford’s campus
Leave-one-out cross-validation: train on 7 buildings, test on 1
Parameters for density models estimated from training data using Maximum Likelihood
Exact inference on graph done using Viterbi-like algorithm
Erick Delage, Stanford University, 2005

19. Results – Floor Boundary Detection
Erick Delage, Stanford University, 2005

20. Results – Floor Boundary Detection
Erick Delage, Stanford University, 2005

21. Erick Delage, Stanford University, 2005
3D Reconstruction
Erick Delage, Stanford University, 2005

22. Erick Delage, Stanford University, 2005
3D Reconstruction
Erick Delage, Stanford University, 2005

23. Erick Delage, Stanford University, 2005
3D Reconstruction
Extra Material:
Exemples #1, #2, #3
Or at:
Erick Delage, Stanford University, 2005

24. Erick Delage, Stanford University, 2005
Performance
Precision of floor boundary in segmentation
Precision of floor boundary in 3d localization
Erick Delage, Stanford University, 2005

25. Erick Delage, Stanford University, 2005
Conclusion
Monocular 3d reconstruction is a good example of an ambiguous problem that can be resolved using prior knowledge about the domain
The presented Bayesian network proves high efficiency in learning prior knowledge necessary for this application
This is the first autonomous algorithm for depth recovery in a rich, textured indoor scene.
Erick Delage, Stanford University, 2005

26. Erick Delage, Stanford University, 2005
Future Work
Apply graphical modeling for more complex geometry.
Formulate the problem in a form that scales precision performance with depth of objects.
Embed this approach in real robot navigation problem (ex. RC car, night indoor navigation)
Erick Delage, Stanford University, 2005

27. Erick Delage, Stanford University, 2005
Questions ?
Erick Delage, Stanford University, 2005

28. Erick Delage, Stanford University, 2005
References
Criminisi, A., Reid, I., & Zisserman, A. (2000). Single View Metrology. IJCV, 40,
Martin, D. R., Fowlkes, C. C., & Malik, J. (2002). Learning to Detect Natural Image Boundaries using Brightness and Texture. NIPS.
Trucco, E., & Verri, A. (1998). Introductory techniques for 3d computer vision. Prentice Hall.
Zhang, R., Tsai, P.-S., Cryer, J. E., & M. Shah (1999). Shape from shading: a survey. IEEE Trans. On PAMI, 21,
Erick Delage, Stanford University, 2005