Light, Surface and Feature in Color Images Lilong Shi Postdoc at Caltech Computational Vision Lab, Simon Fraser University

Topics  Color Constancy  Surface Reflectance Model  Feature Analysis

Color Formation reflectance spectral Illum. power distribution camera response sensor sensitivity

Color Constancy

Automatic White Balance AWB Canonical

Color Constancy Methods  Retinex Theory (McCann64)  MaxRGB/White-Patch (Land77): max(R)  Gray-World (Buchsbaum80): mean(R)  Shades-of-Gray (Finlayson04): [mean(R p )] 1/p  Gray-Edge Hypothesis (Weijer07): mean(edge(R))  Non-Negative Matrix Factorization (Shi07) =

Color Constancy Methods  Gamut Mapping (Forsyth90)  Color by Correlation (Finlayson01)  Neural Network (Cardei02)  Support Vector Regression (Xiong06)  Thin Plate Spline (Shi11)

Color Constancy Methods  Classification-based (Bianco09)  Scene-based (Gijsenij11)

Color Constancy Evaluation MethodInputTrainspeedPara. Relative Performance Assumptions Max-RGB imgnovery fastnonepoorwhite surface Gray-World imgnovery fastnonepooraverage gray Shades-of-Gray imgnomoderateonemoderate/goodaverage gray Edge-based Hyp. imgnofastnonemoderateaverage gray Color-by-Corre. histyesfasta fewmoderatecandidate illums Neural-Network histyesmoderatesomegoodnone Sup. Vector Reg. histyesdep. trainsomemoderate/goodnone Thin-Plate-Spline thumyesdep. traina fewgoodnone

Blackbody Radiator Lights  Tungsten lamps, sunrise/sunset, sky light  Planckian locus  Narrowband sensors

Surface Reflectance Model  LIS Coordinate (Finlayson 01)

Achromatic Surface  Detection in LIS Gray Surface

Skin Color Model  Skin: melanin + hemoglobin  Skin Reflectance (Hiraoka et al 93)  Under blackbody illumination pigment density absorbance length in epidermis/dermis absorbance of other material

Skin Color Locus  Linear model   m is melanin basis,   h is hemoglobin basis,   is blackbody radiator basis,  c is a constant vector

Skin Tone Correction  Even simpler model: Tone correction Preserve melanin 16 different illum + camera calibrations

Features  Textures, edges, corner, blobs, etc..  Colors  Integrated by Quaternion

Quaternion  Real, complex, quaternion (q = a + b  i + c  j + d  k)  Non-commutative (pq ≠ qp)  Convolution, Correlation, Fourier, Wavelet, etc  SVD, EVD, PCA

Texture Feature Extraction QPCA Image-specific quaternion texture basis Sampled sub-windows

Texture Feature Extraction Single quaternion A texture patch 1 st QPCA Basis T 

Texture Feature  1 st Feature

Segmentation Quaternion Hoang(05)

Segmentation

Color Curvature

Iso-luminance  Color -> Gray  Cancellation in combining +/- derivatives

Hessian Descriptor  2 nd order local shape  Principle Curvature eigenvectors: (e 1, e 2 ) eigenvalues: | 1 |<| 2 | e1e1 e2e2 1 λ2λ2 e1e1 e2e2 λ2λ2 1

Curvature  Tubular, vessel-like structures [Frangi98]  With eigen-values  blobness:  backgroundness:  vesselness:  R and  S  Gray image, 2 λ’s; RGB image, 6λ’s

Color Curvature  Quaternion-valued Hessian  QSVD on H  2 real singular values

Curvature Detection Frangi Quaternion

Future Works  Content-based color constancy  Color blob/points detection  Possibilities …