1. Lambertian Reflectance and Linear Subspaces
Ronen Basri David Jacobs
Weizmann NEC

2. Lighting affects appearance

3. How Complicated is Lighting?
Lighting => infinite DOFs.
Set of possible images infinite dimensional (Belhumeur and Kriegman)
But, in many cases, lighting => 9 DOFs.

4. Prior Empirical Study 90.7 97.2 96.3 99.5 #9 88.5 95.3 99.1 #7 84.7
(Epstein, Hallinan and Yuille;
see also Hallinan; Belhumeur and Kriegman)
90.7
97.2
96.3
99.5 #9
88.5
95.3
99.1 #7
84.7
94.1
93.5
97.9 #5
76.3
88.2
90.2
94.4 #3
42.8
67.9
53.7
48.2 #1
Parrot
Phone
Face
Ball
Dimension:

5. Our Results
Convex, Lambertian objects: 9D linear space captures >98% of reflectance.
Explains previous empirical results (Epstein, Hallinan and Yuille; Hallinan; Belhumeur and Kriegman)
For lighting, justifies low-dim methods.
Simple, analytic form => New recognition algorithms.

6. Domain Domain llmax (cosq, 0) Lambertian No cast shadows
Lights far and isotropic n l q
llmax (cosq, 0)

7. Reflectance
Lighting
Images
...

8. Lighting to Reflectance: Intuition

9. +
(See D’Zmura, ‘91; Ramamoorthi and Hanrahan ‘00)

10. Spherical Harmonics Orthonormal basis for functions on the sphere
Funk-Hecke convolution theorem
Rotation = Phase Shift
n’th order harmonic has 2n+1 components.

11. Amplitudes of Kernel n

12. Reflectance functions near low-dimensional linear subspace
Yields 9D linear subspace.

13. How accurate is approximation?
Accuracy depends on lighting.
For point source: 9D space captures 99.2% of energy
For any lighting: 9D space captures >98% of energy.

14. Forming Harmonic imagesl lZ lX lY
lXY
lXZ
lYZ

15. Accuracy of Approximation of Images
Normals present to varying amounts.
Albedo makes some pixels more important.
Worst case approximation arbitrarily bad.
“Average” case approximation should be good.

16. Models
Find Pose
Harmonic Images
Query
Compare
Vector: I
Matrix: B

17. Comparison Methods Linear: Non-negative light:
(See Georghides, Belhumeur and Kriegman)
Non-negative light, first order approximation:

18. Previous Linear Methods
Shashua. With no shadows, i=lln
with B = [lX,lY,lZ].
First harmonic, no DC
Koenderink & van Doorn heuristically suggest using l too.

19. PCA on many images
Amano, Hiura, Yamaguti, and Inokuchi; Atick and Redlich; Bakry, Abo-Elsoud, and Kamel; Belhumeur, Hespanha, and Kriegman; Bhatnagar, Shaw, and Williams; Black and Jepson; Brennan and Principe; Campbell and Flynn; Casasent, Sipe and Talukder; Chan, Nasrabadi and Torrieri; Chung, Kee and Kim; Cootes, Taylor, Cooper and Graham; Covell; Cui and Weng; Daily and Cottrell; Demir, Akarun, and Alpaydin; Duta, Jain and Dubuisson-Jolly; Hallinan; Han and Tewfik; Jebara and Pentland; Kagesawa, Ueno, Kasushi, and Kashiwagi; King and Xu; Kalocsai, Zhao, and Elagin; Lee, Jung, Kwon and Hong; Liu and Wechsler; Menser and Muller; Moghaddam; Moon and Philips; Murase and Nayar; Nishino, Sato, and Ikeuchi; Novak, and Owirka; Nishino, Sato, and Ikeuchi; Ohta, Kohtaro and Ikeuchi; Ong and Gong; Penev and Atick; Penev and Sirivitch; Lorente and Torres; Pentland, Moghaddam, and Starner; Ramanathan, Sum, and Soon; Reiter and Matas; Romdhani, Gong and Psarrou; Shan, Gao, Chen, and Ma; Shen, Fu, Xu, Hsu, Chang, and Meng; Sirivitch and Kirby; Song, Chang, and Shaowei; Torres, Reutter, and Lorente; Turk and Pentland; Watta, Gandhi, and Lakshmanan; Weng and Chen; Yuela, Dai, and Feng; Yuille, Snow, Epstein, and Belhumeur; Zhao, Chellappa, and Krishnaswamy; Zhao and Yang.

20. Comparison to PCA Space built analytically Size and accuracy known
More efficient
time,
When pose unknown, rendering and PCA
done at run time.

21. Experiments 3-D Models of 42 faces acquired with scanner.
30 query images for each of 10 faces (300 images).
Pose automatically computed using manually selected features (Blicher and Roy).
Best lighting found for each model; best fitting model wins.

23. Results 9D Linear Method: 90% correct.
9D Non-negative light: 88% correct.
Ongoing work: Most errors seem due to pose problems. With better poses, results seem near 100%.

26. Summary We characterize images object produces.
Useful for recognition with 3D model.
Also tells us how to generalize from images.