1. Light, Surface and Feature in Color Images Lilong Shi Postdoc at Caltech Computational Vision Lab, Simon Fraser University

2. Topics  Color Constancy  Surface Reflectance Model  Feature Analysis

3. Color Formation reflectance spectral Illum. power distribution camera response sensor sensitivity

4. Color Constancy

5. Automatic White Balance AWB Canonical

6. Color Constancy Methods  Retinex Theory (McCann64)  MaxRGB/White-Patch (Land77): max(R)  Gray-World (Buchsbaum80): mean(R)  Shades-of-Gray (Finlayson04): [mean(R p )] 1/p  Gray-Edge Hypothesis (Weijer07): mean(edge(R))  Non-Negative Matrix Factorization (Shi07) =

7. Color Constancy Methods  Gamut Mapping (Forsyth90)  Color by Correlation (Finlayson01)  Neural Network (Cardei02)  Support Vector Regression (Xiong06)  Thin Plate Spline (Shi11)

8. Color Constancy Methods  Classification-based (Bianco09)  Scene-based (Gijsenij11)

9. Color Constancy Evaluation MethodInputTrainspeedPara. Relative Performance Assumptions Max-RGB imgnovery fastnonepoorwhite surface Gray-World imgnovery fastnonepooraverage gray Shades-of-Gray imgnomoderateonemoderate/goodaverage gray Edge-based Hyp. imgnofastnonemoderateaverage gray Color-by-Corre. histyesfasta fewmoderatecandidate illums Neural-Network histyesmoderatesomegoodnone Sup. Vector Reg. histyesdep. trainsomemoderate/goodnone Thin-Plate-Spline thumyesdep. traina fewgoodnone

10. Blackbody Radiator Lights  Tungsten lamps, sunrise/sunset, sky light  Planckian locus  Narrowband sensors

11. Surface Reflectance Model  LIS Coordinate (Finlayson 01)

12. Achromatic Surface  Detection in LIS Gray Surface

13. Skin Color Model  Skin: melanin + hemoglobin  Skin Reflectance (Hiraoka et al 93)  Under blackbody illumination pigment density absorbance length in epidermis/dermis absorbance of other material

14. Skin Color Locus  Linear model   m is melanin basis,   h is hemoglobin basis,   is blackbody radiator basis,  c is a constant vector

15. Skin Tone Correction  Even simpler model: Tone correction Preserve melanin 16 different illum + camera calibrations

16. Features  Textures, edges, corner, blobs, etc..  Colors  Integrated by Quaternion

17. Quaternion  Real, complex, quaternion (q = a + b  i + c  j + d  k)  Non-commutative (pq ≠ qp)  Convolution, Correlation, Fourier, Wavelet, etc  SVD, EVD, PCA

18. Texture Feature Extraction QPCA Image-specific quaternion texture basis Sampled sub-windows

19. Texture Feature Extraction Single quaternion A texture patch 1 st QPCA Basis T 

20. Texture Feature  1 st Feature

21. Segmentation Quaternion Hoang(05)

22. Segmentation

23. Color Curvature

24. Iso-luminance  Color -> Gray  Cancellation in combining +/- derivatives

25. Hessian Descriptor  2 nd order local shape  Principle Curvature eigenvectors: (e 1, e 2 ) eigenvalues: | 1 |<| 2 | e1e1 e2e2 1 λ2λ2 e1e1 e2e2 λ2λ2 1

26. Curvature  Tubular, vessel-like structures [Frangi98]  With eigen-values  blobness:  backgroundness:  vesselness:  R and  S  Gray image, 2 λ’s; RGB image, 6λ’s

27. Color Curvature  Quaternion-valued Hessian  QSVD on H  2 real singular values

28. Curvature Detection Frangi Quaternion

29. Future Works  Content-based color constancy  Color blob/points detection  Possibilities …