1. Light Mixture Estimation for Spatially Varying White Balance
SIGGRAPH 2008
Eugene Hsu, (MIT, CSAIL)
Tom Mertens (Hasselt Univ. EDM)
Sylvain Paris (Adobe System)
Shai Avidan (Adobe System)
Frédo Durand (MIT, CSAIL)

2. Light Mixture Estimation for Spatially Varying White Balance
Outline
Overview
White Balance
Material Color Estimation
Mixture Interpolation
Result
Relighting
Light Mixture Estimation for Spatially Varying White Balance

3. Light Mixture Estimation for Spatially Varying White Balance
Overview
Goal:
Perform a white balance technique for scenes with two light types.
Input:
Original image
Two light colors (user specified)
Light Mixture Estimation for Spatially Varying White Balance

4. Light Mixture Estimation for Spatially Varying White Balance
Assumption
The interaction of light can be described using RGB channels only, instead of requiring full spectra
Surfaces are Lambertian and non-fluorescent.
Color bleeding due to indirect illumination can be ignored.
There are two illuminant types present in the scene and their colors are known beforehand.
Scenes are dominated by only a small number of material colors.
Light Mixture Estimation for Spatially Varying White Balance

5. Light Mixture Estimation for Spatially Varying White Balance

6. Light Mixture Estimation for Spatially Varying White Balance

7. Material Color Estimation
Natural scenes are dominated by a small set of material colors
[Omer and Werman 2004]
Sampling
Voting
Set Estimation
Light Mixture Estimation for Spatially Varying White Balance

8. Light Mixture Estimation for Spatially Varying White Balance

9. Material Color Estimation
Light Mixture Estimation for Spatially Varying White Balance

10. Material Color Estimation
Light Mixture Estimation for Spatially Varying White Balance

11. Mixture Interpolation
Extends light mixture values over the entire image using interpolation.
Image chromaticities are a linear blend of the material chromaticity multiplied by the light chromaticities.
matting Laplacian [Levin et al. 2006]
Light Mixture Estimation for Spatially Varying White Balance

12. Mixture Interpolation
Light Mixture Estimation for Spatially Varying White Balance

13. Mixture Interpolation
Color line model
Colors lie on a line in RGB space.
Idea
Obtain the pixel opacities β by minimizing the quadratic βT Mβ, where M is the matting Laplacian.
Light Mixture Estimation for Spatially Varying White Balance

14. Mixture Interpolation
Define Mij
wk : window.
Ii, Ij : the colors at pixels i and j.
δij is 1 if i= j, 0 otherwise.
μk, Σk : the mean and variance of pixel colors in wk.
E3 is the 3×3 identity matrix.
ε: regularizing constant.
Light Mixture Estimation for Spatially Varying White Balance

15. Mixture Interpolation
Objective function
α∗ : vector contains mixture constraints.
D : a diagonal matrix that selects the marked pixels from the voting step.
λ : smoothing constant.
Light Mixture Estimation for Spatially Varying White Balance

16. Mixture Interpolation
Light Mixture Estimation for Spatially Varying White Balance

17. Light Mixture Estimation for Spatially Varying White Balance
Result
Synthetic inputs from multiple exposures
Single exposures with real mixed lighting
Light Mixture Estimation for Spatially Varying White Balance

18. Light Mixture Estimation for Spatially Varying White Balance
Result
Light Mixture Estimation for Spatially Varying White Balance

19. Light Mixture Estimation for Spatially Varying White Balance
Result
Light Mixture Estimation for Spatially Varying White Balance

20. Light Mixture Estimation for Spatially Varying White Balance
Relighting
Input
White balanced image.
Light mixture α.
Output
Image that light colors are changed.
Light Mixture Estimation for Spatially Varying White Balance

21. Light Mixture Estimation for Spatially Varying White Balance

22. Light Mixture Estimation for Spatially Varying White Balance
Relighting
Light Mixture Estimation for Spatially Varying White Balance

23. Light Mixture Estimation for Spatially Varying White Balancek + k1 WR WG WB + ÷ IR IG IB RR RG RB • = LR LG LB LR LG LB
Light Mixture Estimation for Spatially Varying White Balance