Light Mixture Estimation for Spatially Varying White Balance SIGGRAPH 2008 Eugene Hsu, (MIT, CSAIL) Tom Mertens (Hasselt Univ. EDM) Sylvain Paris (Adobe System) Shai Avidan (Adobe System) Frédo Durand (MIT, CSAIL)
Light Mixture Estimation for Spatially Varying White Balance Outline Overview White Balance Material Color Estimation Mixture Interpolation Result Relighting Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Overview Goal: Perform a white balance technique for scenes with two light types. Input: Original image Two light colors (user specified) Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Assumption The interaction of light can be described using RGB channels only, instead of requiring full spectra Surfaces are Lambertian and non-fluorescent. Color bleeding due to indirect illumination can be ignored. There are two illuminant types present in the scene and their colors are known beforehand. Scenes are dominated by only a small number of material colors. Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance
Material Color Estimation Natural scenes are dominated by a small set of material colors [Omer and Werman 2004] Sampling Voting Set Estimation Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance
Material Color Estimation Light Mixture Estimation for Spatially Varying White Balance
Material Color Estimation Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Extends light mixture values over the entire image using interpolation. Image chromaticities are a linear blend of the material chromaticity multiplied by the light chromaticities. matting Laplacian [Levin et al. 2006] Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Color line model Colors lie on a line in RGB space. Idea Obtain the pixel opacities β by minimizing the quadratic βT Mβ, where M is the matting Laplacian. Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Define Mij wk : window. Ii, Ij : the colors at pixels i and j. δij is 1 if i= j, 0 otherwise. μk, Σk : the mean and variance of pixel colors in wk. E3 is the 3×3 identity matrix. ε: regularizing constant. Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Objective function α∗ : vector contains mixture constraints. D : a diagonal matrix that selects the marked pixels from the voting step. λ : smoothing constant. Light Mixture Estimation for Spatially Varying White Balance
Mixture Interpolation Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Result Synthetic inputs from multiple exposures Single exposures with real mixed lighting Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Result Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Result Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Relighting Input White balanced image. Light mixture α. Output Image that light colors are changed. Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance Relighting Light Mixture Estimation for Spatially Varying White Balance
Light Mixture Estimation for Spatially Varying White Balance k + k1 WR WG WB + ÷ IR IG IB RR RG RB • = LR LG LB LR LG LB Light Mixture Estimation for Spatially Varying White Balance