Relevance Feedback in Image Retrieval System: A Survey Tao Huang Lin Luo Chengcui Zhang

Outline  Introduction  Background  Relevance Feedback Techniques  Current Research Work  Multi-level Image Object Model  System Examples  Promising Directions  Global Optimization Methods  Semantic Information Incorporation Methods

Background - (1) There has been an explosion in the quantity and complexity of multimedia data in recent years due to:  the development of digital technique ;  the advance of computer technologies;  the widespread of network systems. The need for tools and systems to manage multimedia data is greater than ever, especially the management and retrieval of images. The need of Image Retrieval

Background - (2) Text-based information retrieval approaches are the most conventional approaches. Queries are performed on document surrogates such as keywords, titles and abstracts. Such traditional approaches have also been applied for image data. Text-based Information Retrieval Approaches

Images are annotated manually by keywords and be retrieved by their corresponding annotation. Major drawbacks of this approach:  The vast amount of labor required in manual image annotation  Inconsistency of image annotation among different indexers Keyword Annotation Image Retrieval Background (3)  The differences in interpretation of image content

Content-based retrieval approaches use numerical features computed by direct analysis of the image content such as color, shape, texture. This approach is favorable for:  Features can be computed automatically;  Information used during the retrieval process is always consistent. Content-based Image Retrieval Approaches Background (4)

CBIR stands for Content-base Image Retrieval. CBIR systems use visual features of images like texture, color, shape, line orientation, …, to represent the image content. Data objects in CBIR systems are thus represented by feature vectors and retrieval is performed on computing similarity in the feature space. The similarity between different images is typically defined using the distance between image points in a multi-dimensional feature space. CBIR System Background (5)

Background (6) General Retrieval Model for CBIR query model: image model: similarity model: Retrieval model

Background (7) Design Strategy of CBIR System  Find the “best” representation for the visual features;  Then,  user selects visual feature(s);  System tries to find the similar images

Relevance Feedback Technique - (1) Two distinct characteristics of CBIR system limit their usefulness:  gap between low-level features and high-level concepts  subjectivity of human perception of visual content Limitation of CBIR Approaches

Relevance Feedback - (2) Basic Idea of Relevance Feedback Relevance feedback is a technique of query refinement. Major feature: user is incorporated as part of the image retrieval loop. Advantage: remove weight-specifying burden from user establish more accurate link between low- level features and high-level concept better the retrieval performance of system

Relevance Feedback - (3) (1) System provides a group of images according to user’s query; (2) Users feed back negative/positive images information; (3) System learns from user’s feedback; (4) System then refines the original query; (5) repeats steps (1) ~ (4) till user is satisfied. Steps of Relevance Feedback

Relevance Feedback - (4) Formula for Relevance Feedback