1. Photometric Stereo & Shape from Shading
3-D Computer Vision CSc 83029
Photometric Stereo & Shape from Shading
3-D Computer Vision CSc83029 / Ioannis Stamos

2. Photometric Stereo & Shape from Shading
Technique for recovering 3-D shape information from image intensity (brightness)
We will discuss:
Reflectance maps.
Photometric stereo.
Shape from shading.
3-D Computer Vision CSc83029 / Ioannis Stamos

3. Radiometry and Reflectance
Image irradiance
Brightness falloff
1 / F-number of lens
Scene radiance
We assume (calibration is needed) that:
3-D Computer Vision CSc83029 / Ioannis Stamos

4. Lambertian Reflectance Model
I(x,y) p θi v n s P
A Lambertian sphere
k : Source brightness
ρ’: Surface albedo (reflectance)
ρ : Effective albedo (absorbs
source
brightness)
or:
: REFLECTANCE MAP
3-D Computer Vision CSc83029 / Ioannis Stamos

5. Lambertian Reflectance Model
I(x,y) p θi v n s P
A Lambertian sphere
: REFLECTANCE MAP
Relates Image Irradiance E to surface
orientation for given source direction
and surface reflectance
3-D Computer Vision CSc83029 / Ioannis Stamos

6. Representation of surface normal
Unit vector ry Z
Z=Z(x,y) rx y
Appendix A.5 (Trucco) x
3-D Computer Vision CSc83029 / Ioannis Stamos

7. 3-D Computer Vision CSc83029 / Ioannis Stamos
Gradient Space (p,q)
Source z
1.0 q y p x
Surface normal can be represented by a point (p,q) on a plane!
Source direction can be represented by a point (ps,qs)!
**We want to calculate (p,q) from intensity I(x,y)
3-D Computer Vision CSc83029 / Ioannis Stamos

8. 3-D Computer Vision CSc83029 / Ioannis Stamos
Gradient Space (p,q)
Source z
1.0 q y p x
Surface normal
Source direction
Assumption: SOURCE DIR. IS CONSTANT FOR ENTIRE SCENE.
3-D Computer Vision CSc83029 / Ioannis Stamos

9. Reflectance Map (Lambertian)
OR:
Source z
ISO-BRIGHTNESS
CONTOUR q
Constant θi p y x
3-D Computer Vision CSc83029 / Ioannis Stamos

10. Reflectance Map (Lambertian)
ISO-BRIGHTNESS
CONTOURS
NOTE: R(p,q) is maximum when (p,q)=(ps,qs)
3-D Computer Vision CSc83029 / Ioannis Stamos

11. Reflectance Map (Lambertian)
Examples.
Where is
the source
with respect
to the sphere?
3-D Computer Vision CSc83029 / Ioannis Stamos

12. Reflectance Map (Glossy Surfaces)
3-D Computer Vision CSc83029 / Ioannis Stamos

13. Shape from Shading PROBLEM: Given 1) source direction
ISO-BRIGHTNESS
CONTOURS
PROBLEM: Given 1) source direction
2) surface reflectance (ρ)
3) one intensity image I(x,y)
Can we find unique surface orientation (p,q)?

14. 3-D Computer Vision CSc83029 / Ioannis Stamos
Two reflectance maps?
3-D Computer Vision CSc83029 / Ioannis Stamos

15. 3-D Computer Vision CSc83029 / Ioannis Stamos
Two reflectance maps?
Intersections:
2 solutions for
p and q.
What if we don’t know the albedo?
3-D Computer Vision CSc83029 / Ioannis Stamos

16. 3-D Computer Vision CSc83029 / Ioannis Stamos
Photometric Stereo
Use multiple light sources to resolve ambiguity
In surface orientation.
Note: Scene does not move – Correspondence
between points in different images is easy!
Notation: Direction of source i: or
Image intensity produced by source i:
3-D Computer Vision CSc83029 / Ioannis Stamos

17. Lambertian Surfaces (special case)
Use THREE sources in directions
Image Intensities measured at point (x,y):
orientation
albedo
3-D Computer Vision CSc83029 / Ioannis Stamos

18. Photometric Stereo: RESULT
INPUT
orientation
albedo

19. From Surface Orientations to Shape
Integrate needle map
3-D Computer Vision CSc83029 / Ioannis Stamos

20. Calibration Objects & Look-up Tables
Calibration: Useful when Reflectance Map Equations
are uknown.
Use Calibration sphere of known size and same reflectance
as scene objects.
Each point on the sphere
has a unique known
orientation.
3-D Computer Vision CSc83029 / Ioannis Stamos

21. Calibration Objects & Look-up Tables
Illuminate the sphere with one source at a time and
obtain an image.
Each surface point with orientation (p,q) produces
three images (I1, I2, I3)
Generate a LOOK-UP TABLE (I1, I2, I3) -> (p,q) I3
(p,q)
ARRAY I1 I2
For an object of uknown shape but same reflectance,
obtain 3 images using same sources. For each image
point use LUT to map (I1, I2, I3) -> (p,q)

22. Photometric Stereo: Remarks
Reflectance & illumination must be known a-priori.
Local Method.
Major Assumption: No interreflections.
Concave surfaces exhibit interreflections.
3-D Computer Vision CSc83029 / Ioannis Stamos

23. 3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
Can we recover shape from a Single Image?
3-D Computer Vision CSc83029 / Ioannis Stamos

24. Human Perception of Shape from Shading
We assume light source is above us.
Ramachandran 88
3-D Computer Vision CSc83029 / Ioannis Stamos

25. Human Perception of Shape from Shading
We assume light source is above us.
Ramachandran 88
3-D Computer Vision CSc83029 / Ioannis Stamos

26. Human Perception of Shape from Shading
Surface boundaries have strong influence on perceived shape
3-D Computer Vision CSc83029 / Ioannis Stamos

27. Finding the Illumination Direction
Assumption: The surface normals of the scene are distributed uniformly in 3-D space
Illumination direction
Mean intensity
Mean square intensity
Mean image gradient

28. 3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
Smoothness constraint
3-D Computer Vision CSc83029 / Ioannis Stamos

29. 3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
Calculus of Variations -> Discrete Case:
Is the average of the 4 neighboring values
Iterative solution
3-D Computer Vision CSc83029 / Ioannis Stamos

30. 3-D Computer Vision CSc83029 / Ioannis Stamos
Shape from Shading
We know the normal at the contour.
This provides boundary conditions.
OCCLUDING BOUNDARY
3-D Computer Vision CSc83029 / Ioannis Stamos