1 A Compact Feature Representation and Image Indexing in Content- Based Image Retrieval A presentation by Gita Das PhD Candidate 29 Nov 2005 Supervisor: Dr. Sid Ray Clayton School of Information Technology Monash University, Australia

2 Overview Introduction to CBIR Research Issues Feature Representation Experimental Results Conclusion and Future Directions References

3 CBIR-what is it? Each image is described by it’s visual features e.g. colour, shape, texture Image content is extracted e.g. Colour Histogram, Colour Moments Each image is being represented by a M- dimensional feature vector A similarity measure is used to find the distance between a query image and the database image Images are ranked in order of closeness to query and top N r images are returned to the user

4 CBIR-how does it work? Image Database Feature Database Query Image Feature Extraction Similarity Measure Results

5 Research Issues Improvement of System Accuracy –Proper selection of features and their representation –Use of multiple features & how to integrate them Reduction of Semantic Gap –Human Intervention-Relevance Feedback –How to perceive user’s need, extract information and incorporate user’s feedback into the system Reduction in Retrieval time –Reduction in feature dimension –Efficient indexing

6 Feature Representation Which colour model? We use HSV model which is perceptually uniform. Which colour representation? We use Colour Co- occurrence Matrices (CCM) of H, S, V space to construct a feature vector. What is a CCM? In a CCM, P = [ p ij ], p ij indicates the no. of times a pixel having colour level i co-occurs with another pixel having colour level j, at a position d. Why CCM? It not only gives pixel information but also spatial information of an image.

7 Sum-Average of CCM Elements Haralick’s Sum-Average formula: If P is a LxL CCM, p 11 p 12 p 13 p 14 p 21 p 22 p 23 p 24 p 31 p 32 p 33 p 34 p 41 p 42 p 43 p 44 SA=2p 11 +3(p 21 +p 12 )+4(p 31 +p 22 +p 13 ) +……+8(p 44 ) Where i+j=k, k=2,3,….2L and ……(1)

8 A compact feature vector As we considered pixel pairs in both horizontal and vertical directions, H,S,V CCMs are symmetric. For H=16, S=3, V=3, Original dimension: 148-D ( ) Reduced dimension: 25-D ( ) We used all diagonal elements of CCM. And a single Sum-average value to represent all non-diagonal elements as per following formula: …….(2) where i,j are row and column no.

9 Image Indexing Each image is represented by a 25-D feature vector. Feature values are normalized to lie in the [0,1] range so that each component contributes equally in the distance metric. We start with equal weights to all components and then update them using Relevance Feedback.

10 Similarity Measure We used a weighted Minkowski distance to measure similarity between query image, Q and database image I: …….(3)

11 Relevance Feedback.….(5) RF is essential to reduce semantic gap. We updated both query vector and the weights in eqn. (3) as follows: …….(4)

12 Experimental Results No. of images in database: 2000 No. of categories: 10 (Flowers, Fruits and Vegetables, Nature, Leaves, Ships, Faces, Fishes, Cars, Animals, Aeroplanes) Query Image: all 2000 images chosen as query and then averaged to get final precision. Precision is used to measure performance.

13 Experimental Results Fig. 1 Effect of H-only and H,S,V together on precision at different feature dimensions

14 Experimental Results Fig 2: Precision is marginally worse at low scope and significantly better at higher scope

15 Experimental Results Scope148-D25-D Fig. 3 graphs showing improvement in precision with RF at different scopes and at different dimensions Table shows increase of precision(%) from 0rf to 5rf.

16 Conclusion and Future Directions What do we conclude? Addition of S and V-space with H-space improves information content of images and hence precision Less online computation time with our feature vector Better precision with dimension reduction Future work ? Compare our method with other existing ones Precision as a function of sample size and scope RF as a multiple class problem as opposed to binary

17 References 1.Young Rui, Thomas S. Huang, Shih-Fu Chang, Image Retrieval: Current Techniques, Promising Directions and Open Issues, Journal of Visual Communication and Image Presentation, Vol. 10, No. 4, April R. M. Haralick, K. Shanmugam, and I. Dinstein, “Textural features for image classification,” IEEE Transactions on Systems, Man, and Cybernetics, pp. 610–621, November S. Aksoy and R. M. Haralick, F.A. Cheikh and M. Gabbouj, “A weighted distance approach to relevance feedback,” in International Conference on Pattern Recognition, Barcelona, Spain, September S.-O. Shim and T.-S.Choi, “Image Indexing by modified colour co-occurrence matrix,” in International Conference on Image Processing, Vol. 3, September 2003.

18 THANK YOU! Any Questions?