Multispectral Image Invariant to Illumination Colour, Strength, and Shading Mark S. Drew and Amin Yazdani Salekdeh School of Computing Science, Simon Fraser University, Vancouver, BC, Canada {mark/ayazdani}@cs.sfu.ca

Table of Contents Introduction RGB Illumination Invariant Multispectral Image Formation Synthetic Multispectral Images Measured Multispectral Images Conclusion

New  Introduction Invariant Images – RGB: Information from one pixel, with calibration Information from all pixels – use entropy New  Multispectral data: Information from one pixel without calibration, but knowledge of narrowband sensors peak wavelengths

RGB Illumination Invariant Removing Shadows from Images, ECCV 2002 Graham Finlayson, Steven Hordley, and Mark Drew 4

An example, with delta function sensitivities RGB… An example, with delta function sensitivities B W R Y G P Narrow-band (delta-function sensitivities) Log-opponent chromaticities for 6 surfaces under 9 lights

Deriving the Illuminant Invariant RGB… Deriving the Illuminant Invariant Log-opponent chromaticities for 6 surfaces under 9 lights Rotate chromaticities This axis is invariant to illuminant colour

An example with real camera data RGB… An example with real camera data Normalized sensitivities of a SONY DXC-930 video camera Log-opponent chromaticities for 6 surfaces under 9 different lights

Deriving the invariant RGB… Deriving the invariant Log-opponent chromaticities Rotate chromaticities The invariant axis is now only approximately illuminant invariant (but hopefully good enough)

Image Formation Multispectral Illumination : motivate using theoretical assumptions, then test in practice Planck’s Law in Wien’s approximation: Lambertian surface S(), shading is , intensity is I Narrowband sensors qk(), k=1..31, qk()=(-k) Specular: colour is same as colour of light (dielectric):

Multispectral Image Formation … To equalize confidence in 31 channels, use a geometric-mean chromaticity: Geometric Mean Chromaticity:  with

Multispectral Image Formation … surface-dependent sensor-dependent illumination-dependent So take a log to linearize in (1/T) ! 11

Multispectral Image Formation … Logarithm: known because, in special case of multispectral, *know* k !  Only sensor-unknown is ! ( spectral-channel gains)

Multispectral Image Formation … If we could identify at least one specularity, we could recover log k ?? Nope, no pixel is free enough of surface colour . So (without a calibration) we won’t get log k, but instead it will be the origin in the invariant space. Note: Effect of light intensity and shading removed: 31D  30-D Now let’s remove lighting colour too: we know 31- vector (ek – eM)  (-c2/k - c2/M) Projection  to (ek – eM) removes effect of light, 1/T : 30D  29-D

Algorithm: Form 31-D chromaticity k Take log Project  to (ek – eM) using projector Pe

What’s different from RGB?  Algorithm: What’s different from RGB?  For RGB have to get “lighting-change direction” (ek – eM) either from calibration, or internal evidence (entropy) in the image. For multispectral, we know (ek – eM) !

First, consider synthetic images, for understanding: Surfaces: 3 spheres, reflectances from Macbeth ColorChecker Camera: Kodak DSC 420 31 sensor gains qk() Carry out all in 31-D, but show as camera would see it.

Synthetic Images shading, for light 1, for light 2 Under blue light, P10500 Under red light, P2800

Synthetic Images Original: not invariant Spectral invariant

Measured Multispectral Images Under D75 Under D48 Invt. #1 Invt. #2

Measured Multispectral Images After invt. processing In-shadow, In-light

Measured Multispectral Images

Measured Multispectral Images

Measured Multispectral Images

Next: removing shadows from Conclusion A novel method for producing illumination invariant, multispectral image Successful in removing effects of Illuminant strength, colour, and shading Next: removing shadows from remote-sensing data.

Funding: Natural Sciences and Engineering Research Council of Canada Thanks! Funding: Natural Sciences and Engineering Research Council of Canada Multispectral Images Invariant to Illumination Colour, Strength and Shading