Depthmap Reconstruction Based on Monocular cues 第九章 单幅图像深度重建 Depthmap Reconstruction Based on Monocular cues

深度图

章节安排 基于单眼线索的深度重建 Shape From Shading Shape From Vanishing Point Shape From Defocus Shape From Texture

Shape From Shading

What is Shading? Well… not shadow… We can’t reconstruct shape from one shadow…

What is Shading? Variable levels of darkness Gives a cue for the actual 3D shape There is a relation between intensity and shape

Shading Examples These circles differ only in grayscale intensity Intensities give a strong “feeling” of scene structure

What determines scene radiance?       n  

   

Surface Normal Convenient notation for surface orientation A smooth surface has a tangent plane at every point We can model the surface using the normal at every point

The Shape From Shading Problem Given a grayscale image And albedo And light source direction Reconstruct scene geometry Can be modeled by surface normals

Lambertian Surface Appears equally bright from all viewing directions Reflects all light without absorbing Matte surface, no “shiny” spots Brightness of the surface as seen from camera is linearly correlated to the amount of light falling on the surface Here we will discuss only Lambertian surfaces under point-source illumination n    

Some Notations: Surface Orientation    

Some Notations: Surface Orientation            

Reflectance Map  

Reflectance Map Lambertian case Reflectance Map (Lambertian) Iso-brightness contour cone of constant

Reflectance Map Lambertian case Note: is maximum when iso-brightness contour Note: is maximum when

Reflectance Map Example Brightness as a function of surface orientation Lambertian surface iso-brightness contour  

Reflectance Map of a Glossy Surface Brightness as a function of surface orientation Surface with diffuse and glossy components

Reflectance Map Examples Brightness as a function of surface orientation

Graphics with a 3D Feel  

Shape From Shading?  

Shape From Shading! Use more images Shape from shading Photometric stereo Shape from shading Introduce constraints Solve locally Linearize problem

Photometric Stereo Take several pictures of same object under same viewpoint with different lighting

Photometric Stereo Take several pictures of same object under same viewpoint with different lighting

Photometric Stereo Take several pictures of same object under same viewpoint with different lighting

Photometric Stereo We can write this in matrix form: Lambertian case: Image irradiance: We can write this in matrix form:

改变光源所获得的同一个球的五幅图像

Shape From Shading! Use more images Shape from shading Photometric stereo Shape from shading Introduce constraints Solve locally Linearize problem

Human Perception Our brain often perceives shape from shading. Mostly, it makes many assumptions to do so. For example: Light is coming from above (sun). Biased by occluding contours. by V. Ramachandran

Main Approaches  

Main Approaches  

Main Approaches  

Main Approaches  

Basic MINimizatION Solution

Stereographic Projection (p,q)-space (gradient space) (f,g)-space Problem (p,q) can be infinite when Redefine reflectance map as

Occluding Boundaries and are known The values on the occluding boundary can be used as the boundary condition for shape-from-shading

Image Irradiance Constraint Image irradiance should match the reflectance map Minimize (minimize errors in image irradiance in the image)

Smoothness Constraint Used to constrain shape-from-shading Relates orientations (f,g) of neighboring surface points Minimize : surface orientation under stereographic projection (penalize rapid changes in surface orientation f and g over the image)

Basic Propagation Solution Horn [85] Solution by Characteristic Curves Basic Propagation Solution

Propagating Solution    

Propagating Solution  

Propagating Solution  

Propagating Solution    

Propagating Characteristic Curve Need to initialize every curve at some known point Singular points Occluding boundaries      

Propagating Characteristic Curve Need to initialize every curve at some known point Singular points Occluding boundaries Curves are “grown” independently, very instable      

Basic LINEARIZED Solution Pentland, 1988 Basic LINEARIZED Solution

Linearized Solution Describe reflection map as a function and linearized it. S1. Calculate the Taylor series expansion and keep the low-order items：

Linearized Solution 2. Apply Fourier transform to both side of equation： Calculate Then make inverse Fourier transform to obtain the surface normal

Input image Ground truth Minimization based method Propagation based method Improved propagation based method Linearized method