1. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus IMAGE RETRIEVAL WITH RELEVANCE FEEDBACK Hayati CAM Ozge CAVUS

2. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Outline Question: What is Content Based Image Retrieval? Recent Work on CBIR Our Approach Evaluation Summary

3. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus CBIR Large quantities of multimedia data is used in archives Traditional way: Using keywords in IR(Image Retrieval) Problems: Annotation is very difficult Keywords may be insufficient to represent the contents of the images Keywords are user dependent

4. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus CBIR

5. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Recent Work Extracting global low-level features (texture or color) from images Problem: limited in capability of deriving higher semantic meanings of the images

6. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Recent Work Partitioning images into nonoverlapping grid cell Problem: Grids are not meaningful regions

7. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Our Approach

8. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Our Approach Image Segmentation Codebook Construction Image Representation by using Posterior Class Probability Values Content Based Image Retrieval with Relevance Feedback

9. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Dataset TRECVID 2005 dataset 29832 video shots Contain approximately 20 different classes exp: mountain, seaside, urban, sports …

10. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Segmentation

11. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Segmentation Cluster the RGB color values of the pixels by k-means

12. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Segmentation Smooth the regions by combined classifier approach

13. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Codebook Construction

14. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Representation Calculate region k=1000 bins histograms for each image

15. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Representation

16. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Image Representation

17. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Relevance Feedback At the first iteration images are ranked by distances to the query image After each iteration user labels the images as relevant and irrelevant The new result are retrieved according to the user feedback

18. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Content Based Image Retrieval

19. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Relevance Feedback Assign a weight value w to each class probability value The weights are assigned uniformly in the first iteration.

20. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Relevance Feedback Given two images: Distances between the corresponding probability terms are computed d i = distance between the i th probability values of two images where i=1, …, c These distances are combined as d = w i d i

21. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Relevance Feedback Given the positive and negative examples, for a probability term being significant for a particular query: Distances for the corresponding probability values for relevant images must usually be similar (hence, a small variance), Distances between the probability values for relevant images and irrelevant images must usually be different (hence, a large variance).

22. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Relevance Feedback Weights are computed as: std(distances of i th probability term between relevant and irrelevant images) W i = std(distances of i th probability term between relevant images)

23. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Evaluation Yaos formula for cluster validation n tr > n t Why do we need this? Better Clustering -> Better Probability Values -> Better Retrieval

24. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Evaluation Precision-Recall

25. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus Summary Steps of Our Approach Image Segmentation Codebook Construction Image Representation by probabilities CBIR with Relevance Feedback

26. Image Retrieval With Relevant Feedback Hayati Cam & Ozge Cavus THANK YOU