Image indexing and retrieving using histogram based methods 03/7/15資工研所陳慶鋒

Outline Histogram based methods Histogram based methods Image retrieval using the three methods Image retrieval using the three methods Experimental result Experimental result Library of Image formats Library of Image formats Future work Future work References References

Histogram based features Color Histogram Color Histogram Histogram Refinement Histogram Refinement Color Correlogram Color Correlogram

Color histogram For a n  n with m colors image I, For a n  n with m colors image I, the color histogram is the color histogram is where where p 為屬於 I 的 pixel, I(p) 為其顏色 p 為屬於 I 的 pixel, I(p) 為其顏色,,for,,for

Color histogram (cont.) Advantages Advantages -trivial to compute -trivial to compute -robust against small changes in camera -robust against small changes in camera viewpoint viewpoint Disadvantages Disadvantages -without any spatial information -without any spatial information

Histogram refinement The pixels of a given bucket are subdivided into classes based on local feature. Within a given bucket, only pixels in the same class are compared. The local feature which this paper used: Color Coherence Vectors(CCVs) Color Coherence Vectors(CCVs)

Histogram refinement (cont.) CCVs CCVs For the discretized color c i, the pixels with color c i are coherence if the number of connected component>= , indicated as  ci, otherwise are incoherence, indicated as  ci, and total pixel with color c i =  ci +  ci, a threshold  is defined as the condition of coherence or not For the discretized color c i, the pixels with color c i are coherence if the number of connected component>= , indicated as  ci, otherwise are incoherence, indicated as  ci, and total pixel with color c i =  ci +  ci, a threshold  is defined as the condition of coherence or not for color j, the coherence pair is (  ci,  ci ) for color j, the coherence pair is (  ci,  ci )

Histogram refinement (cont.)

Example Example

Histogram refinement (cont.) Example(cont.) Example(cont.)

Histogram refinement (cont.) Example(cont.) Example(cont.) LableABCDE Color12132 Size

Histogram refinement (cont.) Example(cont.) Example(cont.) Color123 α12200 β301

Color correlograms A table indexed by color pairs, where the k-th entry for color pair specifies the probability of finding a pixel of color j at a distance k from a pixel of color i in the image. A table indexed by color pairs, where the k-th entry for color pair specifies the probability of finding a pixel of color j at a distance k from a pixel of color i in the image. The correlogram is The correlogram is d………………… 1… (1,1)(1,2)…… (m,m)

Color correlograms(cont.) The autocorrelogram is d…………… 1… 1…m

Color correlograms (cont.) Example Example

Color correlograms (cont.) Example(cont.) Example(cont.)

Color correlograms (cont.) Example(cont.) Example(cont.)

Image retrieval using the three methods Similarity measure Similarity measure -L1 distance similarity -L1 distance similarity -relative distance -relative distance Performance measure Performance measure -ranking measure -ranking measure

Similarity measure L1 distance similarity Sim() L1 distance similarity Sim() Sim(I,I’) 愈大，兩張圖的相似度愈高 Sim(I,I’) 愈大，兩張圖的相似度愈高

Similarity measure(cont.) Relative distance Relative distance 愈小，兩張圖的相似度愈高 愈小，兩張圖的相似度愈高

Performance measure Ranking measures Ranking measures 令 為 query images 的集合， Q’ i 為 Q i 的 answer image answer image r-measure: r-measure: average r-measure: average r-measure: p 1 -measure: p 1 -measure: average p 1 -measure: average p 1 -measure:

Experimental result Experimental setup Experimental setup - Image database of 180 gray level images with size - Image database of 180 gray level images with size 192x x128 -Quantize gray level to 16 bins -Quantize gray level to 16 bins -Set  of CCV as Set  of CCV as Set d of autocorrelogram as 30 -Set d of autocorrelogram as 30 -A query set which consists 25 query images and 25 answer images -A query set which consists 25 query images and 25 answer images

Experimental result(cont.) Results Results similarity hist: 1 ccv: 1 auto: 1 similarity hist: 1 ccv: 1 auto: 1 relative distance hist: 1 ccv: 1 auto: 1 relative distance hist: 1 ccv: 1 auto: 1 similarity hist: 32 ccv: 26 auto: 44 similarity hist: 32 ccv: 26 auto: 44 relative distance hist: 33 ccv: 38 auto: 31 relative distance hist: 33 ccv: 38 auto: 31

Experimental result(cont.) Results(cont.) Results(cont.) similarity hist: 41 ccv: 11 auto: 77 similarity hist: 41 ccv: 11 auto: 77 relative distance hist: 10 ccv: 3 auto: 7 relative distance hist: 10 ccv: 3 auto: 7 similarity hist: 55 ccv: 26 auto: 80 similarity hist: 55 ccv: 26 auto: 80 relative distance hist: 2 ccv: 10 auto: 1 relative distance hist: 2 ccv: 10 auto: 1

Experimental result(cont.) Results(cont.) Results(cont.) performance measure in similarity and relative distance performance measure in similarity and relative distance SimilarityColor histogramccvauto r-measure avg r-measure p 1 -measure avg p 1 -measure Relative distanceColor histogramccv auto r-measure avg r-measure p 1 -measure avg p 1 -measure

Experimental result(cont.) Results(cont.) Results(cont.) performance measure in similarity and relative distance performance measure in similarity and relative distance SimilarityColor histogramccvauto r-measure avg r-measure p 1 -measure avg p 1 -measure Relative distanceColor histogramccv auto r-measure avg r-measure p 1 -measure avg p 1 -measure

Experimental result(cont.) Factors which affect performance Factors which affect performance - choice of image database - choice of image database - choices between query images and answer images - choices between query images and answer images -  of CCV -  of CCV - d of color autocorrelogram - d of color autocorrelogram

Library of Image formats Include: imgdata.h Include: imgdata.h Formats: pgm, jpg, png, bmp Formats: pgm, jpg, png, bmp We can get: width, height, and raw data We can get: width, height, and raw data

Library of Image formats(cont.) Functions Functions GetPGM(char, int*, int*, unsigned char**) GetPGM(char, int*, int*, unsigned char**) GetPNG(char, int*, int*, unsigned char**) GetPNG(char, int*, int*, unsigned char**) GetBMP(char, int*, int*, unsigned char**) GetBMP(char, int*, int*, unsigned char**) GetJPEG(char, int*, int*, unsigned char**) GetJPEG(char, int*, int*, unsigned char**)

Library of Image formats(cont.) Example Example int width, height int width, height unsigned char* data unsigned char* data GetJPEG(“1.jpg”, &width, &height, &data) GetJPEG(“1.jpg”, &width, &height, &data)

Future work Image indexing and retrieving of color images (debugging) Image indexing and retrieving of color images (debugging) Further study Further study

References [1] M. Swain and D. Ballard, “ Color indexing, ” International Journal of Computer Visioin, 7(1):11-32, 1991 [1] M. Swain and D. Ballard, “ Color indexing, ” International Journal of Computer Visioin, 7(1):11-32, 1991 [2] G. Pass and R.Zabih, “ Histogram refinement for content based image retrieval, ” IEEE Workshop on Applications of Computer Vision, pp , 1996 [2] G. Pass and R.Zabih, “ Histogram refinement for content based image retrieval, ” IEEE Workshop on Applications of Computer Vision, pp , 1996 [3] G Pass and R. Zabih, “ Compare images using color coherence vectors, ” Applications of Computer Vision, WACV '96., Proceedings 3rd IEEE Workshop on, 2-4 Dec 1996, Page(s): [3] G Pass and R. Zabih, “ Compare images using color coherence vectors, ” Applications of Computer Vision, WACV '96., Proceedings 3rd IEEE Workshop on, 2-4 Dec 1996, Page(s): [4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “ Image indexing using color correlograms, ” Conf. Computer Vision and Pattern Recognit., pp ,1997 [4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “ Image indexing using color correlograms, ” Conf. Computer Vision and Pattern Recognit., pp ,1997