Relevance Feedback in Image Retrieval Systems: A Survey Part II Lin Luo, Tao Huang, Chengcui Zhang School of Computer Science Florida International University

Introduction Multi-level Image Model For RF Retrieval System Examples  Model Formalization  RF based on this model  FourEyes(PhotoBook)  PicToSeek

Model Formalization D: raw image data, e.g. a JPEG image. F = { f i }: low-level visual features. R = { r ij }: representations for a given f i. where r ij =[ r ij1, r ij2 …,r ijK ] K is the length of the vector. An image object O is represented as O = O ( D, F, R)

Model Formalization (cont.) This object model supports multiple representations of weights based on the content of image objects. Weights exist at various levels. W i, W ij, and W ijk, are associated with features f i, representations r ij, and components r ijk, respectively.  Support weights

RF Based on This Model The goal of RF based on this model is to find the appropriate weights to model the user’s information request. RF method based on this model has been effectively developed in some systems, such as Mars97, MindReader and etc.

Retrieval System Examples FourEyes(Photobook) PicToSeek  About FourEyes  RF Related Concepts in FourEyes  RF Method in FourEyes  About PicToSeek  Image and Query Model in PicToSeek  RF Method in PicToSeek

FourEyes About FourEyes An interactive, power-assisted tool for segmenting and annotating image, embedded in the most recent version of Photobook. FourEyes offers a practical way to get interactive performance by using RF technique RF Related Concepts in FourEyes Grouping: A set of image regions that are associated in some way. FourEyes includes Within-image grouping and Across- image grouping.

RF Method in FourEyes FourEyes forms compound groupings for users. FourEyes adapt the grouping to the user’s needs with user feedback FourEyes uses adaptive weighting mechanism. Each grouping has its own weight. User adds the grouping which maximizes the product of this number and the prior weight of the grouping to the compound grouping.

Interface for PhotoBook

About PicToSeek An image search engine for cataloging and search images on the Web entirely on the basis of the pictorial content, proposed by Gevers et al in Univ. of Amsterdam, The Netherlands.

Image and Query Model in PicToSeek Each image can be presented by its image vectors as Query is presented by its corresponding image vectors Q in the same form

Image and Query Model in PicToSeek (cont.) Formula used in PicToSeek to assign weights Similarity function used in PicToSeek

RF Methods in PicToSeek Main Idea Users feed back negative/positive images information. System learns which image features are more important from users’ feedback and find the images according to the new features weighting.

RF Method in PicToSeek RF process Formula Use this formula to produce improved query specification. User need not to give a precise initial query formulation, the RF technique can move the query into the user-desired direction.

Interface for PicToSeek

Performance Greatly reduce the user’s effort of composing a query. Capture the user’s information need more precisely.

Conclusion All the approaches described above perform RF at the low- level feature vector level, but failed to take into account the actual semantics for the images themselves. There have been some efforts on incorporating semantics in RF for image retrieval, which will be covered in the next presentation. ======================= END =======================