Illumination Estimation via Non- Negative Matrix Factorization By Lilong Shi, Brian Funt, Weihua Xiong, ( Simon Fraser University, Canada) Sung-Su Kim, Byoung-Ho Kang, Sung-Duk Lee, and Chang-Yeong Kim (Samsung Advanced Institute of Technology, Korea) Presented by: Lilong Shi

Automatic White Balance Problem AWB Colour constancy accounting for differences in illumination colour

Overview N sub-windows Take log and apply NMFsc Illumination component (low sparseness) M Reflectance basis (high sparseness) Illumination image by anti-log Reflectance images by anti-log With this we can do AWB

The Model of Illumination and Feature Reflectances RGB sensor response is defined by E(λ) : illumination spectral power distribution S(λ) : matte surface reflectance function R k (λ) : sensor sensitivity function of channel k Assuming narrowband sensors:

The Model of Illumination and Feature Reflectances In logarithm space Linear combination of illumination and reflectance For an entire colour image I, with E and S the illumination and reflectance

Linear Reflectance Features Illumination log E Changes slowly cross an image Reflectance log S Linear combination of M “features” F i weights h i

7 Linear Reflectance Features “Feature” Reflectances “building blocks” e.g. basis images derived from the ORL face image database following Li et al. (2001) Independent  No non-zero pixels in common  Dot product of 2 blocks is zero The complete model

Non-Negative Matrix Factorization NMF Input data matrix Basis vectorsWeights Factored result A data instance v is a weighted combination of basis

Constraints on the Factorization Illumination & reflectance non-negative => NMF basis non-negative E smooth, R non-smooth Sparseness vs. Smoothness 1D example Increasing smoothness Increasing sparseness

Sparseness Constraint Sparseness implies most entries zero 2D example Increasing sparseness

Sparseness Measure Sparseness s(x) of x= Sparseness constraint is enforced during matrix factorization L-1 norm L-2 norm

NMFsc Using Non-Negative Matrix Factorization with sparseness constraint Calling it NMFsc

NMFsc for Auto White Balancing The Illumination-Reflectance model NMFsc form In combination

Incorporating Sparseness Finding M+1 basis vectors Set low sparseness for 1 st basis vector (illumination) Set high sparseness for 2 nd -(M+1) th basis (feature reflectance)

The Algorithm N sub-windows Take log and apply NMFsc Illumination basis (low sparseness) M Reflectance basis (high sparseness) Illumination image by anti-log Reflectance images by anti-log

Experiment on MNFsc (M=4) Input Ground Truth NMFsc result

Experiment on MNFsc (M=4) Illumination Image Reflectance Images

More Experiment on NMFsc (M=4) Input Ground Truth NMFsc result

Experiment on MNFsc (M=4) Illumination Image Reflectance Images

Experiment on MNFsc (M=1) Ground Truth Input Illumination Image NMFsc Result Reflectance Image

More Experiments (M=1) NMFsc Result Reflectance Image Ground Truth Input Illumination Image

Tests on Large Dataset (M=4) 16 sub-windows (16x16) Take log and apply NMFsc Illumination basis (sparseness=0.001) 4 Reflectance basis (sparseness = 0.45) Illumination image by anti-log Reflectance images by anti-log 7661 images (64x64) Average to estimate illumination

Tests on Large Dataset (M=1) Single sub-window (64x64) Take log and apply NMFsc Illumination basis (sparseness=0.001) One reflectance basis (sparseness = 0.45) Illumination image by anti-log Reflectance images by anti-log 7661 images (64x64) Average to estimate illumination

Results Processing Time: 0.83 sec/image for M = 4; 2.43 sec/image for M = 1; Method Angular DegreesL-2 Distance (x10 2 ) MeanMaxMeanMax GW SoG MAX RGB NMFsc (M = 4) NMFsc (M = 1)

Algorithm Comparison via Wilcoxon MethodGWSoGMAX NMFsc (M=4) NMFsc (M=1) GW =+=- SoG =++- MAX ---- NMFsc (M=4) =-+- NMFsc (M=1) ++++ NMFsc better than Greyworld, Shades of Gray, Max RGB

Conclusions New AWB method using NMF NMF ‘factors’ illumination from reflectance Provides separate estimate for each pixel Globally minimizes objective function across all three colour channels Incorporates both colour and spatial (sparseness) information Assumptions spatially smooth illumination variation non-smooth reflectance variation

Conclusions Insensitive to sparseness setting NMFsc converges quickly iterations Good AWB results Tested on large data set of natural images

Financial support provided by Samsung Advanced Institute of Technology

Thank you! Yoho National Park British Columbia, Canada