Human Color Perception in the HSV Space and its Application in Histogram Generation for Image Retrieval A. Vadivel Department of Computer Science & Engineering Indian Institute of Technology Kharagpur India.

CBIR – An Introduction Content Based Image Retrieval involves retrieval of images from a database which are “similar” to a query image. The similarity metric is usually based on image features such as color, texture, shape, etc.,. Some existing CBIR systems:- QBIC, VisualSeek, NeTra, MARS, Blobworld, PicToSeek,SIMPLIcity. Color based feature are found to be most effective.

 Chose an appropriate Color Space such as RGB, HSV, etc.  To find NN with respect to Color Perception  HSV color representation separates Chrominance and luminance components Hue Saturation Intensity Developing color based CBIR Applications View Of HSV Color space

Typical Color Histogram RGB Color Histogram LSB is truncated Various BINs One update for one pixel A Pixel

When S=0, Hue is undefined When I=0, Saturation is undefined Pixel is “True Color”, when H is defined Pixel is “Gray Color”, when H is Undefined Updated HereOR Updated Here TRUE COLORGRAY COLOR Useful properties of HSV color space Histogram

S=0, No Color Perceived S is increased color perceived Saturation (S) I=0, Black Pixel I is increased for S= 1.0 Intensity (I) Saturation and Intensity values of a pixel determine “True color” pixel or a “Gray color” pixel. The variation in perceived color with changes in S and I min max min max

Fovea centralis(only cones) Very high – resolution color vision but low light sensitivity

Saturation Saturation(S) Color weight Near Saturation threshold Transition is gradual The color is perceived gradually min max Saturation Threshold to determine “True Color and Gray Color” Color weight = 1 if S>0.25 = 0 if S<=0.25 Color weight is added to the respective BIN’s value of the Color Histogram

w H (S,I) should satisfy the following conditions  (a) w H (S,I) is in the range [0,1]  (b) For S 1 > S 2, w H (S 1,I) > w H (S 2,I)  (c) For I 1 > I 2, w H (S,I 1 ) > w H (S,I 2 )  (d) w H (S,I) changes slowly with S for high values of I  (e) w H (S,I) changes sharply with S for low values of I A function W H (S,I)to capture the variation in perceived color with changes in S and I

Justification High I and low S –All types of cells are excited yet W H is low due to sharp vision, i.e. simultaneous excitation of different types of cones in the fovea centralis due to white light mix leads to loss in color. – captured by condition (d) Low I and high S– Visual perception is predominantly by rod cells very low contribution by cone cells. => loss of color perception. - captured by condition (e)

A Typical Choice of W H (S,I). W I (S,I) = 1 – W H (S,I) W H – True color degree of a pixel. W I - Gray color degree of a pixel. A Pixel WHWH WIWI Here r 1 and r 2 are constants with 0.2 and 1.5 as values + = 1

AND TRUE COLORGRAY COLOR BIN VALUE = BIN VALUE +W H BIN VALUE = BIN VALUE +W I A Pixel WIWI WHWH Histogram Generation Histogram

For each pixel in the image Read the RGB value Convert RGB to Hue (H), Saturation (S) and Intensity Value (I) Determine w H (S,I) and w I (S,I) Update histogram as follows: HCPH[Round(H.MULT_FCTR)]= HCPH[Round(H.MULT_FCTR)]+ w H (S,I) HCPH[N T +Round(I/DIV_FCTR)]= HCPH[N T + Round(I/DIV_FCTR)]+w I (S,I) Histogram Generation Procedure (HCPH) N T = Round (2  MULT_FCTR) +1 N g = Here N T is total number of True color BIN N g is total number of Gray color BINS

WEB BASED IMAGE RETRIEVAL SYSTEM Interested user can visit

HSVSN – HSV Normal HSVSD – HSV Hard Decision HCPH – Human color Perception Histogram Performance comparison with HSV color histograms. Variation of Perceived Precision with Nearest Neighbor based on different distance measures Observation : HCPH scheme leads to higher precision compared to other HSV color histogram for all types of distance measure (a) (b) (c) (d)

(a) (b) (c) (d) Performance of color histogram based schemes for different distance measures EU-Euclidean Distance MH – Manhattan Distance VCAD – Vector Cosine Angle Distance HI- Histogram Intersection JV- Jain & Vailaya’s scheme Precision variation with nearest neighbor Observation : Histogram Intersection distance measure leads to higher precision for most of the color histograms

Comparison of HCPH using Histogram Intersection based distance measure with some existing color histogram based schemes JV – Jain & Vailaya’s Method HCPH- Human color Perception Histogram Observation: (i) HCPH scheme leads to higher precision for most cases (ii ) HCPH scheme leads to uniformly high correct retrieval -> Lower standard deviation of perceived precision higher Precision

Precision(P) of retrieval of HCPH and some recently proposed CBIR schemes. Scheme usedN=10N=20N=50N=100 Local Fourier Transform (LFT) Quantization (YUV) Color Texture Moments (HSV) Color Texture Moments (SvcosH, SvsinH, V) Multimedia Retrieval Markup Language with Four-Level Relevance Feedback Systems combSUM Merge Color-Spatial Feature (36 Colors) Human Color Perception Histogram (HCPH) N= No. of images retrieved for query Average Precision (%)

Conclusion A Simple color weight function W H of S and I is proposed to estimate True color degree and Gray color degree of a pixel. HCPH scheme has lower histogram dimension (2-D). HCPH scheme tries to capture human visual perception of the color of a pixel for grouping similar pixels in the histogram. Histogram Intersection distance metric gives higher precision compared to other distance metrics. Among different color histogram based schemes HCPH leads to higher precision is most cases. Standard Deviation of observed Precision is smaller with HCPH scheme.

References [1] R. Brunelli and O. Mich, “Histograms Analysis for Image Retrieval”, Pattern Recognition 34, pp., , [2] Y. Deng, B. S. Manjunath, C. Kenney, M. S. Moore and H. Shin, “An Efficient Color Representation for Image Retrieval”, IEEE Transactions on Image Processing, 10, pp., , [3] J. C. French, J. V. S. Watson, X. Jin and W. N. Martin, “Integrating Multiple Multi-Channel CBIR Systems”, Inter. Workshop on Multimedia Information Systems (MIS 2003), pp., 85-95, [4] T. Gevers and H. M. G. Stokman, “Robust Histogram Construction from Color Invariants for Object Recognition”, IEEE Trans. on Pattern Analysis and Machine Intelligence (PAMI) 26(1), pp., , [5] R. C. Gonzalez and R. E. Woods, “Digital Image Processing “, II Ed. Pearson Education Asia, First Indian Reprint, [6] Z. Lei, L. Fuzong, and Z. Bo, “A CBIR Method Based Color-Spatial Feature”, Proc. IEEE Region 10th Annual International Conference, pp., , [7] H. Mueller, W. Mueller, S. Marchand-Maillet, D. Squire and T. Pun, “ A Web-Based Evaluation System for CBIR”, Third Intl. Workshop on Multimedia Information Retrieval (MIR2001), [8] T. Ojala, M. Rautiainen, E. Matinmikko and M. Aittola, “Semantic Image Retrieval with HSV Correlograms”, Scandinavian Conference on Image Analysis, pp., , [9] H.Yu, M. Li, H J. Zhang and J. Feng, “Color Texture Moments for Content-Based Image Retrieval”, Proc. Int. Conference on Image Processing, Volume III, pp., , [10] F. Zhou,J. Feng and Q. Shi, “Texture Feature Based on Local Fourier Transform”, Proc. Int. Conference on Image Processing, pp., 7-10, 2001.

Representation of colors in the histogram.  True Color Components   Gray Color Components  Circular representation of “true colors” and linear representation of “gray colors”.

Pixels when Hue is undefined Saturation Pixels when Hue is defined Weight = 1 if S>0.25 = 0 if S<=0.25 Saturation(S) Weight Weight is added to the respective BIN’s value of the Histogram Saturation Threshold to determine “True Color and Gray Color” min max