1. 3-D Depth Reconstruction from a Single Still Image 何開暘 2010.6.11

2. Visual Cues for Depth Perception Monocular Cues  Texture variations, texture gradients, interposition, occlusion, known object sizes, light and shading, haze, defocus Stereo Cues Motion Parallax and Focus Cues

3. image → feature → depth Chose features that capture 3 types of cues: texture variations, texture gradients, and color Model conditional distribution of depths given monocular image features p(d|x) Estimate parameters by maximizing conditional log likelihood of training data Given an image, find MAP estimate of depths

4. Outline Introduction Feature Vector Probabilistic Model Experiments Reference

5. Feature vectors Two types of features  Absolute depth features―used to estimate absolute depth at a particular patch  Relative features―used to estimate relative depths Capture three types of cues  Texture variation―apply Law ’ s masks to intensity channel  Haze―apply a local averaging filter to color channels  Texture gradient―apply six oriented edge filters to intensity channel

6. Features for Absolute Depth Compute summary statistics of a patch i in the image I(x,y) as follows  Use the output of each of the 17 (9 Law’s masks, 2 color channels and 6 texture gradients) filters Fn, n=1,…,17 as: (dimension 34) To estimate absolute depth at a patch, local image features centered on the patch are insufficient Use more global properties

7. More Global Properties Use image features extracted at multiple spatial scales (three scale) Features used to predict depth of a particular patch are computed from that patch as well as 4 neighboring patches (Repeated at each of the three scales) Add to features of a patch additional summary features of the column it lies in (5*3+4)*34=636 dimensional

9. Features for Relative Depth To learn the dependencies between two neighboring patches Compute a 10-bin histogram of each of the 17 filter outputs, giving a total of 170 features y is for each patch i at scale s Relative depth features y ijs for two neighboring patches i and j at scale s will be the differences between their histogram, i.e., y ijs =y is -y js

10. Outline Introduction Feature Vector Probabilistic Model Experiments Reference

11. Gaussian Model

12. Laplacian Model

13. Outline Introduction Feature Vector Probabilistic Model Experiments Reference

15. Result

16. Improving Performance of Stereovision using Monocular Cues

18. The average errors as a function of the distance from the camera

19. Reference A.Y. Ng A. Saxena, S.H. Chung. 3-d depth reconstruction from a single still image. In International Journal of Computer Vision (IJCV), 2007. Michels, J., Saxena, A., & Ng, A. Y. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In 22 nd international conference on machine learning (ICML).