1. Specularity, the Zeta-image, and Information-Theoretic Illuminant
Estimation
Mark S. Drew1, Hamid Reza Vaezi Joze1, and Graham D. Finlayson2
2School of Computing Sciences,
The University of East Anglia
Norwich, England
1School of Computing Science
Simon Fraser University
Vancouver, BC, Canada

2. Zeta-Image: Goal= Discover the light color (yet another!??)
Relative Chromaticity:
The main idea is that we can get at a good solution for the chromaticity of the light by dividing image chromaticity 3-vector  by the candidate light chromaticity e — the “relative chromaticity”  =    e
[where   is component-wise division].

3. Algorithm:
Then we show that, over pixels that are specular or white, the log of the relative chromaticity log() is perpendicular to the light chromaticity e in color space. This gives a useful hint for recovering e .

4. Proof; Background: Simple image formation model -- k = R,G,B surface
“Neutral interface”: Lee, JOSA 1986, “Method for computing the scene-illuminant hromaticity”
k = R,G,B
pixel
color
surface
light
specularity
Light is “white enough”: Borges, JOSA 1991, “Trichromatic approximation method for surface illumination”

5. Proof…
 = {R,G,B}/{R+G+B)
Now chromaticity:
Relative chrom.:

6. Proof…
…Relative chrom.:
simple!

7. Proof…
Now let’s head for a Planar Constraint:

8. Proof…

9. Therefore, let the Zeta-image† be
Definition:
where e is the chromaticity of illuminant and (x) is the chromaticity value of pixel at position x .
Properties
It has the structure of the
Kullback-Leibler Divergence from Information Theory;
Zeta is low (near-zero) at specularities:
† Patent applied for

10. Zeta-image for illumination estimation
Planar constraint: For near-specular pixels or white surfaces, Log-Relative-Chromaticity values are orthogonal to the light chromaticity. Or, equally, the zeta image is near zero.
So the best light chromaticity e for an image is that which minimizes the zeta-image for near-specular pixels (or white surfaces) => guess a domain  ; then ,
== Search

11. Search — explained:
assume candidate e; form  ; the lowest 10-percentile, say, of dot-product values could be near-specular pixels.
Over a grid of possible light-chromaticities e, minimize dot-product values
over candidate illuminants for those lowest 10- percentile pixels.

12. Or, Thm: Analytic solution is the geometric mean of
where  is a set of bright pixels
Search is better than Analytic but slower.

13. Domain  for Analytic Solution:
We start by approximating  as top-5% brightness pixels.
Could be any other method to indicate near- specular pixels and white surface regions.
Detecting Failure  i.e., detecting images not having specularity or white surfaces in top brightness pixels:
 can stem from areas of images belonging to the brightest surface which happens to tend to be some particular surface color.
we can simply check if these pixels are in the possible chromaticity gamut of illuminants.
 can be a bag-of-pixels from all over the image.
we can investigate the distribution of  in chromaticity space.

14. Does this work?...
divide by correct light, bottom 20% of Zeta: yes!

15. ...Does this work?...
incorrect light, bottom 20% of Zeta: no!
correct light, float (inverted) Zeta 

16. Details of Analytic:
How did we come to the geometric mean?
Solve:  

17. …Details of Analytic… How do we know  is positive?
If components were probabilities then
has structure of Kullback-Leibler Divergence:
extra bits to code samples from ek when using
codebook based on ik , so positive!

18. …Details of Analytic… Does this work?
Final step: form  using the geomean chromaticity
for bright pixels, and then trim to least-10% values of
Zeta, and recalculate the geomean.
Does this work?

19. float (inverted) Zeta
bright
lowzeta = bright &
(zeta<quantile(zeta)),0.10);

20. plots of  for correct light e (O) and for
analytic solution e (x) :

21. In expts.
mask off the
ColorChecker

22. Algorithm 2: Planar Constraint Applied as Post-Processing
so far: Algorithm 1:
Use either analytic answer, or a
simple hierarchical grid search over light-
chromaticity
Algorithm 2: Planar Constraint Applied as Post-Processing
Use any alg’s answer for e; take SVD of lowest-10% dot-product pixels: improves light estimate!

23. fallback position:
If either of two chromaticity checks fails, use Grey-Edge.

24. Post-processing: Results.

25. Illumination Estimation: Results.
(Analytic)

26. Funding: Natural Sciences and Engineering Research Council of Canada
Thanks!
Funding: Natural Sciences and
Engineering Research Council of Canada