Photometric Stereo & Shape from Shading 3-D Computer Vision CSc 83029 Photometric Stereo & Shape from Shading 3-D Computer Vision CSc83029 / Ioannis Stamos

Photometric Stereo & Shape from Shading Technique for recovering 3-D shape information from image intensity (brightness) We will discuss: Reflectance maps. Photometric stereo. Shape from shading. 3-D Computer Vision CSc83029 / Ioannis Stamos

Radiometry and Reflectance Image irradiance Brightness falloff 1 / F-number of lens Scene radiance We assume (calibration is needed) that: 3-D Computer Vision CSc83029 / Ioannis Stamos

Lambertian Reflectance Model I(x,y) p θi v n s P A Lambertian sphere k : Source brightness ρ’: Surface albedo (reflectance) ρ : Effective albedo (absorbs source brightness) or: : REFLECTANCE MAP 3-D Computer Vision CSc83029 / Ioannis Stamos

Lambertian Reflectance Model I(x,y) p θi v n s P A Lambertian sphere : REFLECTANCE MAP Relates Image Irradiance E to surface orientation for given source direction and surface reflectance 3-D Computer Vision CSc83029 / Ioannis Stamos

Representation of surface normal Unit vector ry Z Z=Z(x,y) rx y Appendix A.5 (Trucco) x 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Gradient Space (p,q) Source z 1.0 q y p x Surface normal can be represented by a point (p,q) on a plane! Source direction can be represented by a point (ps,qs)! **We want to calculate (p,q) from intensity I(x,y) 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Gradient Space (p,q) Source z 1.0 q y p x Surface normal Source direction Assumption: SOURCE DIR. IS CONSTANT FOR ENTIRE SCENE. 3-D Computer Vision CSc83029 / Ioannis Stamos

Reflectance Map (Lambertian) OR: Source z ISO-BRIGHTNESS CONTOUR q Constant θi p y x 3-D Computer Vision CSc83029 / Ioannis Stamos

Reflectance Map (Lambertian) ISO-BRIGHTNESS CONTOURS NOTE: R(p,q) is maximum when (p,q)=(ps,qs) 3-D Computer Vision CSc83029 / Ioannis Stamos

Reflectance Map (Lambertian) Examples. Where is the source with respect to the sphere? 3-D Computer Vision CSc83029 / Ioannis Stamos

Reflectance Map (Glossy Surfaces) 3-D Computer Vision CSc83029 / Ioannis Stamos

Shape from Shading PROBLEM: Given 1) source direction ISO-BRIGHTNESS CONTOURS PROBLEM: Given 1) source direction 2) surface reflectance (ρ) 3) one intensity image I(x,y) Can we find unique surface orientation (p,q)?

3-D Computer Vision CSc83029 / Ioannis Stamos Two reflectance maps? 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Two reflectance maps? Intersections: 2 solutions for p and q. What if we don’t know the albedo? 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Photometric Stereo Use multiple light sources to resolve ambiguity In surface orientation. Note: Scene does not move – Correspondence between points in different images is easy! Notation: Direction of source i: or Image intensity produced by source i: 3-D Computer Vision CSc83029 / Ioannis Stamos

Lambertian Surfaces (special case) Use THREE sources in directions Image Intensities measured at point (x,y): orientation albedo 3-D Computer Vision CSc83029 / Ioannis Stamos

Photometric Stereo: RESULT INPUT orientation albedo

From Surface Orientations to Shape Integrate needle map 3-D Computer Vision CSc83029 / Ioannis Stamos

Calibration Objects & Look-up Tables Calibration: Useful when Reflectance Map Equations are uknown. Use Calibration sphere of known size and same reflectance as scene objects. Each point on the sphere has a unique known orientation. 3-D Computer Vision CSc83029 / Ioannis Stamos

Calibration Objects & Look-up Tables Illuminate the sphere with one source at a time and obtain an image. Each surface point with orientation (p,q) produces three images (I1, I2, I3) Generate a LOOK-UP TABLE (I1, I2, I3) -> (p,q) I3 (p,q) ARRAY I1 I2 For an object of uknown shape but same reflectance, obtain 3 images using same sources. For each image point use LUT to map (I1, I2, I3) -> (p,q)

Photometric Stereo: Remarks Reflectance & illumination must be known a-priori. Local Method. Major Assumption: No interreflections. Concave surfaces exhibit interreflections. 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Shape from Shading Can we recover shape from a Single Image? 3-D Computer Vision CSc83029 / Ioannis Stamos

Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3-D Computer Vision CSc83029 / Ioannis Stamos

Human Perception of Shape from Shading We assume light source is above us. Ramachandran 88 3-D Computer Vision CSc83029 / Ioannis Stamos

Human Perception of Shape from Shading Surface boundaries have strong influence on perceived shape 3-D Computer Vision CSc83029 / Ioannis Stamos

Finding the Illumination Direction Assumption: The surface normals of the scene are distributed uniformly in 3-D space Illumination direction Mean intensity Mean square intensity Mean image gradient

3-D Computer Vision CSc83029 / Ioannis Stamos Shape from Shading Smoothness constraint 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Shape from Shading Calculus of Variations -> Discrete Case: Is the average of the 4 neighboring values Iterative solution 3-D Computer Vision CSc83029 / Ioannis Stamos

3-D Computer Vision CSc83029 / Ioannis Stamos Shape from Shading We know the normal at the contour. This provides boundary conditions. OCCLUDING BOUNDARY 3-D Computer Vision CSc83029 / Ioannis Stamos