1. Image indexing and retrieving using histogram based methods 03/7/15資工研所陳慶鋒

2. Outline Histogram based methods Histogram based methods Image retrieval using the three methods Image retrieval using the three methods Experimental result Experimental result Library of Image formats Library of Image formats Future work Future work References References

3. Histogram based features Color Histogram Color Histogram Histogram Refinement Histogram Refinement Color Correlogram Color Correlogram

4. Color histogram For a n  n with m colors image I, For a n  n with m colors image I, the color histogram is the color histogram is where where p 為屬於 I 的 pixel, I(p) 為其顏色 p 為屬於 I 的 pixel, I(p) 為其顏色,,for,,for

5. Color histogram (cont.) Advantages Advantages -trivial to compute -trivial to compute -robust against small changes in camera -robust against small changes in camera viewpoint viewpoint Disadvantages Disadvantages -without any spatial information -without any spatial information

6. Histogram refinement The pixels of a given bucket are subdivided into classes based on local feature. Within a given bucket, only pixels in the same class are compared. The local feature which this paper used: Color Coherence Vectors(CCVs) Color Coherence Vectors(CCVs)

7. Histogram refinement (cont.) CCVs CCVs For the discretized color c i, the pixels with color c i are coherence if the number of connected component>= , indicated as  ci, otherwise are incoherence, indicated as  ci, and total pixel with color c i =  ci +  ci, a threshold  is defined as the condition of coherence or not For the discretized color c i, the pixels with color c i are coherence if the number of connected component>= , indicated as  ci, otherwise are incoherence, indicated as  ci, and total pixel with color c i =  ci +  ci, a threshold  is defined as the condition of coherence or not for color j, the coherence pair is (  ci,  ci ) for color j, the coherence pair is (  ci,  ci )

8. Histogram refinement (cont.)

9. Example Example

10. Histogram refinement (cont.) Example(cont.) Example(cont.)

11. Histogram refinement (cont.) Example(cont.) Example(cont.) LableABCDE Color12132 Size1215315

12. Histogram refinement (cont.) Example(cont.) Example(cont.) Color123 α12200 β301

13. Color correlograms A table indexed by color pairs, where the k-th entry for color pair specifies the probability of finding a pixel of color j at a distance k from a pixel of color i in the image. A table indexed by color pairs, where the k-th entry for color pair specifies the probability of finding a pixel of color j at a distance k from a pixel of color i in the image. The correlogram is The correlogram is d………………… 1… (1,1)(1,2)…… (m,m) 1 0 0 1 1 1

14. Color correlograms(cont.) The autocorrelogram is d…………… 1… 1…m

15. Color correlograms (cont.) Example Example

16. Color correlograms (cont.) Example(cont.) Example(cont.)

17. Color correlograms (cont.) Example(cont.) Example(cont.)

18. Image retrieval using the three methods Similarity measure Similarity measure -L1 distance similarity -L1 distance similarity -relative distance -relative distance Performance measure Performance measure -ranking measure -ranking measure

19. Similarity measure L1 distance similarity Sim() L1 distance similarity Sim() Sim(I,I’) 愈大，兩張圖的相似度愈高 Sim(I,I’) 愈大，兩張圖的相似度愈高

20. Similarity measure(cont.) Relative distance Relative distance 愈小，兩張圖的相似度愈高 愈小，兩張圖的相似度愈高

21. Performance measure Ranking measures Ranking measures 令 為 query images 的集合， Q’ i 為 Q i 的 answer image answer image r-measure: r-measure: average r-measure: average r-measure: p 1 -measure: p 1 -measure: average p 1 -measure: average p 1 -measure:

22. Experimental result Experimental setup Experimental setup - Image database of 180 gray level images with size - Image database of 180 gray level images with size 192x128 192x128 -Quantize gray level to 16 bins -Quantize gray level to 16 bins -Set  of CCV as 1500 -Set  of CCV as 1500 -Set d of autocorrelogram as 30 -Set d of autocorrelogram as 30 -A query set which consists 25 query images and 25 answer images -A query set which consists 25 query images and 25 answer images

23. Experimental result(cont.) Results Results similarity hist: 1 ccv: 1 auto: 1 similarity hist: 1 ccv: 1 auto: 1 relative distance hist: 1 ccv: 1 auto: 1 relative distance hist: 1 ccv: 1 auto: 1 similarity hist: 32 ccv: 26 auto: 44 similarity hist: 32 ccv: 26 auto: 44 relative distance hist: 33 ccv: 38 auto: 31 relative distance hist: 33 ccv: 38 auto: 31

24. Experimental result(cont.) Results(cont.) Results(cont.) similarity hist: 41 ccv: 11 auto: 77 similarity hist: 41 ccv: 11 auto: 77 relative distance hist: 10 ccv: 3 auto: 7 relative distance hist: 10 ccv: 3 auto: 7 similarity hist: 55 ccv: 26 auto: 80 similarity hist: 55 ccv: 26 auto: 80 relative distance hist: 2 ccv: 10 auto: 1 relative distance hist: 2 ccv: 10 auto: 1

25. Experimental result(cont.) Results(cont.) Results(cont.) performance measure in similarity and relative distance performance measure in similarity and relative distance SimilarityColor histogramccvauto r-measure266203387 avg r-measure10.648.1215.48 p 1 -measure15.2714.5315.57 avg p 1 -measure0.610.580.62 Relative distanceColor histogramccv auto r-measure155185133 avg r-measure6.27.45.32 p 1 -measure15.7714.6816.54 avg p 1 -measure0.630.590.66

26. Experimental result(cont.) Results(cont.) Results(cont.) performance measure in similarity and relative distance performance measure in similarity and relative distance SimilarityColor histogramccvauto r-measure266203387 avg r-measure10.648.1215.48 p 1 -measure15.2714.5315.57 avg p 1 -measure0.610.580.62 Relative distanceColor histogramccv auto r-measure155185133 avg r-measure6.27.45.32 p 1 -measure15.7714.6816.54 avg p 1 -measure0.630.590.66

27. Experimental result(cont.) Factors which affect performance Factors which affect performance - choice of image database - choice of image database - choices between query images and answer images - choices between query images and answer images -  of CCV -  of CCV - d of color autocorrelogram - d of color autocorrelogram

28. Library of Image formats Include: imgdata.h Include: imgdata.h Formats: pgm, jpg, png, bmp Formats: pgm, jpg, png, bmp We can get: width, height, and raw data We can get: width, height, and raw data

29. Library of Image formats(cont.) Functions Functions GetPGM(char, int*, int*, unsigned char**) GetPGM(char, int*, int*, unsigned char**) GetPNG(char, int*, int*, unsigned char**) GetPNG(char, int*, int*, unsigned char**) GetBMP(char, int*, int*, unsigned char**) GetBMP(char, int*, int*, unsigned char**) GetJPEG(char, int*, int*, unsigned char**) GetJPEG(char, int*, int*, unsigned char**)

30. Library of Image formats(cont.) Example Example int width, height int width, height unsigned char* data unsigned char* data GetJPEG(“1.jpg”, &width, &height, &data) GetJPEG(“1.jpg”, &width, &height, &data)

31. Future work Image indexing and retrieving of color images (debugging) Image indexing and retrieving of color images (debugging) Further study Further study

32. References [1] M. Swain and D. Ballard, “ Color indexing, ” International Journal of Computer Visioin, 7(1):11-32, 1991 [1] M. Swain and D. Ballard, “ Color indexing, ” International Journal of Computer Visioin, 7(1):11-32, 1991 [2] G. Pass and R.Zabih, “ Histogram refinement for content based image retrieval, ” IEEE Workshop on Applications of Computer Vision, pp.96-102, 1996 [2] G. Pass and R.Zabih, “ Histogram refinement for content based image retrieval, ” IEEE Workshop on Applications of Computer Vision, pp.96-102, 1996 [3] G Pass and R. Zabih, “ Compare images using color coherence vectors, ” Applications of Computer Vision, 1996. WACV '96., Proceedings 3rd IEEE Workshop on, 2-4 Dec 1996, Page(s): 96 -102 [3] G Pass and R. Zabih, “ Compare images using color coherence vectors, ” Applications of Computer Vision, 1996. WACV '96., Proceedings 3rd IEEE Workshop on, 2-4 Dec 1996, Page(s): 96 -102 [4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “ Image indexing using color correlograms, ” Conf. Computer Vision and Pattern Recognit., pp.762-768,1997 [4] J. Huang, S. R. Kumar, M. Mitra, W. J. Zhu, and R.Zabih, “ Image indexing using color correlograms, ” Conf. Computer Vision and Pattern Recognit., pp.762-768,1997