1. Content-Based Image Retrieval - Approaches and Trends of the New Age
Ritendra Datta, Jia Li, and James Z. Wang
The Pennsylvania State University
MIR2005

2. INTRODUCTION 為什麼image無法處理的像text一樣好
Text is man’s creation, images are a mere replica of what man has seen
Interpretation of what we see is hard to characterize
visual similarity != semantic similarity
CBIR has grown tremendously after 2000, not just in terms of size, but also in the number of new directions explored

3. INTRODUCTION

4. INTRODUCTION
The theoretical foundation behind how we humans interpret images is still an open problem
A brief scanning of about 300 relevant papers published in the last five years revealed that less than 20% were concerned with applications or real-world systems

5. CBIR領域研究方向 Feature Extraction Approaches to Retrieval
Annotation and Concept Detection
Relevance Feedback and Learning
Hardware and Interface Support

6. Feature Extraction 如何抽 Color Feature
“An Efficient Color Representation for Image Retrieval” (比傳統histograms好)
“Multiresolution Histograms and Their Use for Recognition” (用在textured image)
“Image retrieval using color histograms generated by Gauss mixture vector quantization” (利用GMVQ抽color histogram)

7. Feature Extraction Color + Texture 抽取 Shape
“Wavelet-Based Texture Retrieval Using Generalized Gaussian Density and Kullback-Leibler Distance”
Shape
“Shape Matching and Object Recognition Using Shape Contexts” (is fairly compact yet robust to a number of geometric transformations)

8. Feature Extraction Segmentation
“Normalized Cuts and Image Segmentation” (最重要的方向之一)
“Blobworld: Image Segmentation Using Expectation-maximization and Its Application to Image Querying” (我之前用過的方法)
“Segmentation of Brain MR Images Through a Hidden Markov Random Field Model and the Expectation-Maximization Algorithm” (處理medical imaging)

9. Feature Extraction 線條相似度 如何選擇feature
“Image retrieval using wavelet-based salient points”
如何選擇feature
Application-specific feature sets (最直觀的)
“SIMPLIcity:Semantics-Sensitive Integrated Matching for Picture Libraries” (semantics-sensitive feature selection)
“Feature Selection for SVMs” (用classifier)

10. Approaches to Retrieval
Region based image retrieval
“A Scalable Integrated Region-Based Image Retrieval System”
region-based querying (BlobWorld)
Vector quantization (VQ) on image blocks
“Keyblock: An Approach for Content-based Image Retrieval” (generate codebooks for representation and retrieval, taking inspiration from data compression and text-based strategies)

11. Approaches to Retrieval
Windowed search
“Object-Based Image Retrieval Using the Statistical Structure of Images” (more effective than methods based on inaccurate segmentation)
Anchoring-based image retrieval
“A Study of Image Retrieval by Anchoring” (Anchoring is based on the idea of finding a set of representative “anchor” images and deciding semantic proximity between an arbitrary image pair in terms of their similarity to these anchors)

12. Approaches to Retrieval
Probabilistic frameworks for image retrieval
“A Probabilistic Architecture for Content-based Image Retrieval”

13. Annotation and Concept Detection
Supervised classification
“Image Classification for Content-Based Indexing” (involving simple concepts such as city, landscape, sunset,and forest, have been achieved with high accuracy)
Translation approach
“Object recognition as machine translation: Learning a lexicon for a fixed image vocabulary” (我們在clef 2004就是follow這方法)

14. Annotation and Concept Detection
為何如此困難
We humans segment objects better than machines, having learned to associate over a long period of time, through multiple viewpoints, and literally through a “streaming video” at all times
The association of words and blobs become truly meaningful only when blobs isolate objects well

15. Relevance Feedback and Learning
“Relevance Feedback in Image Retrieval: A Comprehensive Review”
Problems
One problem with RF is that after every round of user interaction, usually the top results with respect to the query have to be recomputed
Another issue is the user’s patience in supporting multi-round feedbacks

16. REAL-WORLD REQUIREMENTS
Performance
Semantic learning
Volume of Data
Concurrent Usage
Heterogeneity
Multi-modal features
User-interface
Operating Speed
System Evaluation

17. CURRENT RESEARCH TRENDS
Journals
IEEE T. Pattern Analysis and Machine Intelligence (PAMI)
IEEE T. Image Processing (TIP)
IEEE T. Circuits and Systems for Video Technology (CSVT)
IEEE T. Multimedia (TOM)
J. Machine Learning Research (JMLR)
International J. Computer Vision (IJCV)

18. CURRENT RESEARCH TRENDS
Pattern Recognition Letters (PRL)
ACM Computing Surveys (SURV)
Conferences
IEEE Computer Vision and Pattern Recognition (CVPR)
International Conference on Computer Vision (ICCV)
European Conference on Computer Vision (ECCV)
IEEE International Conference on Image Processing (ICIP)

19. CURRENT RESEARCH TRENDS
ACM Multimedia (MM)
ACM SIG Information Retrieval (IR)
ACM Human Factors in Computing Systems (CHI)

22. CONCLUSIONS
We have presented a brief survey on work related to the young and exciting fields of content-based image retrieval and automated image annotation, spanning 120 publications in the current decade
We have laid out some guidelines for building practical, real-world systems