Simon Fraser University Computational Vision Lab Lilong Shi, Brian Funt and Tim Lee

 Studies of factors affecting skin colour  Simple and linear model of skin  Modelling Skin appearance under lights  Applications:  Estimate melanin and hemoglobin concentrations  Correct imaged skin tones for lighting conditions

Tone correction Preserve melanin Skin tone correction Melanin/Hemoglobin separation

 Appearance of human skin determined by  Biological factors ▪ pigmentation, blood microcirculation, roughness, etc..  Viewing conditions ▪ Inducing lights  Acquisition devices ▪ Cones in retina, RGB sensors of CCD digital cameras

 Two-layered Skin Model [2]  Epidermis Layer: melanin absorbance  Dermis Layer: hemoglobin absorbance  A layer has properties of an optical filter

 Various skin colour <= melanin + hemoglobin  Genetic: Race  Temporary: ▪ Exposure to UV ▪ Hot bath  Mixture varying by 2 independent factors  Analyse melanin and hemoglobin factors

 Estimate melanin and hemoglobin concentration  Independent Component Analysis (ICA) – Statistical technique for revealing “hidden” factors – To “unmix” or “separate” signals composed of multiple sources – Independent and linear mixing – Related to Eigen-vector analysis

Original Source SignalsObserved SignalsMixing s1 s2 70% 30% v1 s × A = v 20% 80% v2 0% 100% v3

Melanin Hemoglobin Skin samples Melanin Hemoglobin

 Typical skin spectrum  Visible wavelength 400nm – 700nm  Extract skin bases from observed spectrum by ICA ICA (left) 33 skin spectrum after normalization; (right) two independent basis spectrum – the melanin and hemoglobin, and the spectrum of chromophores other than melanin and hemoglobin pigments.

 Arbitrary skin spectrum can be approximated constru are variables

 Human vision ▪ 3 types of Photoreceptors  L, M and S Cones  Digital Cameras ▪ 3 sensors  Red, Green, and Blue  Reflectance spectrum recorded by 3 sensors => three values (R, G, B) for a skin colour

Possible skin colours lie within plane Given a pixel from skin, compute by projecting log(R,G,B) onto

Input Image [3] Melanin Image Hemoglobin Image

- Inverse melanin concentration - Inverse hemoglobin concentration

 Skin appearance greatly affected by lights  Reveal true skin colour by removing illum.  Common lights  blackbody radiation  e.g. tungsten/halogen lamps, sunrise/sunset, etc  Varying colour temperature T ▪ Redish -> white -> bluish

 Colour: illumination times reflectance  In log space, multiplication => addition: Illum. basis

 In practice  Drop hemoglobin basis ▪ Small angle between Illum and hemoglobin axes  Ignore brightness  Skin colour varying by T and

 384 real skin reflectances times  67 real light sources  => samples

 Skin tone correction example ( UOPB DB [4] ) 20 Tone correction Preserve melanin 16 different illumination + camera settings

Skin tone correction example ( UOPB DB [4] )

 Skin colour modelling:  Melanin and Hemoglobin concentration  Linear model in logarithm space  Estimation by Independent Component Analysis  Skin appearance + Light modelling:  Estimates light source  Preserves skin colour by melanin value  Applied to digital images from CCD cameras

 [1] Shi, L., and Funt, B., "Skin Colour Imaging That Is Insensitive to Lighting," Proc. AIC (Association Internationale de la Couleur) Conference on Colour Effects & Affects, Stockholm, June 2008  [2] Angelopoulou, E., Molana, R., and Daniilidis, K. “Multispectral skin color modeling,” In IEEE Conf. on Computer Vision and Pattern Recognition, volume 2, pages , Kauai, Hawaii, Dec  [3] Shimizu, H., Uetsuki, K., Tsumura, N., and Miyake, Y. Analyzing the effect of cosmetic essence by independent component analysis for skin color images. In 3 rd Int. Conf. on Multispectral Color Science, pages 65-68, Joensuu, Finland, June  [4] Martinkauppi, B. “Face color under varying illumination-analysis and applications,” Ph.D. Dissertation, University of Oulu, 2002.