1. Depthmap Reconstruction Based on Monocular cues
第九章 单幅图像深度重建
Depthmap Reconstruction Based on Monocular cues

2. 深度图

3. 章节安排 基于单眼线索的深度重建 Shape From Shading Shape From Vanishing Point
Shape From Defocus
Shape From Texture

4. Shape From Shading

5. What is Shading? Well… not shadow…
We can’t reconstruct shape from one shadow…

6. What is Shading? Variable levels of darkness
Gives a cue for the actual 3D shape
There is a relation between intensity and shape

7. Shading Examples These circles differ only in grayscale intensity
Intensities give a strong “feeling” of scene structure

8. What determines scene radiance?n

10. Surface Normal Convenient notation for surface orientation
A smooth surface has a tangent plane at every point
We can model the surface using the normal at every point

11. The Shape From Shading Problem
Given a grayscale image
And albedo
And light source direction
Reconstruct scene geometry
Can be modeled by surface normals

12. Lambertian Surface Appears equally bright from all viewing directions
Reflects all light without absorbing
Matte surface, no “shiny” spots
Brightness of the surface as seen from camera is linearly correlated to the amount of light falling on the surface
Here we will discuss only Lambertian surfaces under point-source illumination n

13. Some Notations: Surface Orientation

14. Some Notations: Surface Orientation

15. Reflectance Map

16. Reflectance Map Lambertian case Reflectance Map (Lambertian)
Iso-brightness contour
cone of constant

17. Reflectance Map Lambertian case Note: is maximum when iso-brightness
contour
Note: is maximum when

18. Reflectance Map Example
Brightness as a function of surface orientation
Lambertian surface
iso-brightness
contour

19. Reflectance Map of a Glossy Surface
Brightness as a function of surface orientation
Surface with diffuse and glossy components

20. Reflectance Map Examples
Brightness as a function of surface orientation

21. Graphics with a 3D Feel

22. Shape From Shading?

23. Shape From Shading! Use more images Shape from shading
Photometric stereo
Shape from shading
Introduce constraints
Solve locally
Linearize problem

24. Photometric Stereo
Take several pictures of same object under same viewpoint with different lighting

25. Photometric Stereo
Take several pictures of same object under same viewpoint with different lighting

26. Photometric Stereo
Take several pictures of same object under same viewpoint with different lighting

27. Photometric Stereo We can write this in matrix form: Lambertian case:
Image irradiance:
We can write this in matrix form:

28. 改变光源所获得的同一个球的五幅图像

32. Shape From Shading! Use more images Shape from shading
Photometric stereo
Shape from shading
Introduce constraints
Solve locally
Linearize problem

33. Human Perception Our brain often perceives shape from shading.
Mostly, it makes many assumptions to do so.
For example:
Light is coming from above (sun).
Biased by occluding contours.
by V. Ramachandran

34. Main Approaches

35. Main Approaches

36. Main Approaches

37. Main Approaches

38. Basic MINimizatION Solution

39. Stereographic Projection
(p,q)-space (gradient space)
(f,g)-space
Problem
(p,q) can be infinite when
Redefine reflectance map as

40. Occluding Boundaries
and are known
The values on the occluding boundary can be used as
the boundary condition for shape-from-shading

41. Image Irradiance Constraint
Image irradiance should match the reflectance map
Minimize
(minimize errors in image irradiance in the image)

42. Smoothness Constraint
Used to constrain shape-from-shading
Relates orientations (f,g) of neighboring surface points
Minimize
: surface orientation under stereographic projection
(penalize rapid changes in surface orientation f and g over the image)

43. Basic Propagation Solution
Horn [85]
Solution by Characteristic Curves
Basic Propagation Solution

44. Propagating Solution

45. Propagating Solution

46. Propagating Solution

47. Propagating Solution

48. Propagating Characteristic Curve
Need to initialize every curve at some known point
Singular points
Occluding boundaries

49. Propagating Characteristic Curve
Need to initialize every curve at some known point
Singular points
Occluding boundaries
Curves are “grown” independently, very instable

50. Basic LINEARIZED Solution
Pentland, 1988
Basic LINEARIZED Solution

51. Linearized Solution
Describe reflection map as a function and linearized it.
S1. Calculate the Taylor series expansion and keep the low-order items：

52. Linearized Solution
2. Apply Fourier transform to both side of equation：
Calculate
Then make inverse Fourier transform to obtain the surface normal

53. Input image
Ground truth
Minimization based method
Propagation based method
Improved propagation based method
Linearized method