Metric Inverted - An efficient inverted indexing method for metric spaces Benjamin Sznajder Jonathan Mamou Yosi Mass Michal Shmueli-Scheuer IBM Research - Haifa Presented by: Shai Erera

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Motivation Web 2.0 enables mass multimedia productions Still, search is limited to manually added metadata State of the art solutions for CBIR (Content Based Image Retrieval) do not scale – Reveal linear scalability in the collection size due to large number of distance computations Can we use textIR methods to scale up CBIR?

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Problem definition Low level image features can be generalized to Metric Spaces Metric Space: An ordered pair (S,d), where S is a domain and d a distance function d: S x S  R such that – d satisfies non-negativity, reflexibility, symmetry and triangle inequality The best-k results for a query in a metric space are the k objects with the smallest distance to the query – Convert distances to scores (small distance – high score) between [0,1]

Problem definition Top-K Problem: – Assume m metric spaces, a Query Q, an aggregate function f and a score function sd(): – Retrieve the best k objects D with highest f(sd 1 (Q,D), sd 2 (Q,D)…sd m (Q,D)) q k=5

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Metric Inverted Index Assume a collection of objects each having m features – Object D = {F 1 :v 1, F 2 :v 2,…, F m :v m } – m metric spaces Indexing steps – Lexicon creation (select candidates) – Invert objects (canonization to lexicon terms)

Metric inverted indexing – Lexicon creation Number of different features too large Need to select candidates – Naïve solution: Lexicon of fixed size l Select randomly l/m documents and extract their features These l features form our lexicon – Improvement Replace the random choice by clustering (K-Means etc.) Keep the lexicon in an M-Tree structure

Metric inverted indexing – invert objects Given object D = {F 1 :v 1, F 2 :v 2,…, F m :v m } Canonization – map features (F i :v i ) to lexicon entries – For each feature select the n nearest lexicon terms – D’ = {F 1 :v 11, F 1 :v 12, …F 1 :v 1n, F 2 :v 21, F 2 :v 22, …F 2 :v 2n, … F m :v m1, F m :v m2, …F m :v mn } Index D’ in the relevant posting-lists

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Retrieval stage – term selection Given Q = {F 1 :qv 1, F 2 :qv 2,…, F m :qv m } Canonization – For each feature select the n nearest lexicon terms – Q’ = {F 1 :qv 11, F 1 :qv 12, …F 1 :qv 1n, F 2 :qv 21, F 2 :qv 22, …F 2 :qv 2n, … F m :qv m1, F m :qv m2, …F m :qv mn }

Retrieval stage – Boolean Filtering These m*n posting-lists will be queried via a Boolean Query Two possible modes: – Strict-query-mode: – Fuzzy-query-mode:

Retrieval stage – Scoring Documents retrieved by the Boolean Query are fully scored Return the best k objects with the highest aggregate score f(sd_1(Q,D),sd_2(Q,D),…,sd_m(Q,D))

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

Experiments Focus on: – Efficiency – Effectiveness Collection of 160,000 images from Flickr 3 features are extracted from each image – EdgeHistogram, ScalableColor and ColorLayout 180 queries – Fuzzy-Query-Mode – Sampled from the collection of images Compared to M-tree data-structure

Experiments – Measures Used Effectiveness: MAP is a natural candidate for measuring – Problem: In Image Retrieval, no document is irrelevant – Solution: we defined as relevant the k highest scored documents in the collection (according to the M-Tree computation) – MAP computed on relevant and retrieved lists of size k

Experiments – Measures Used contd. Efficiency: we compute the number of computations per query – A computation unit (cu) is a distance computation call between two feature values

Effectiveness MAP vs. number of Nearest Terms size of the lexicon = 12000

Effectiveness MAP vs. lexicon size Number Nearest Terms =30

Effectiveness vs. Efficiency MAP vs. number of comparisons Number Nearest Terms =30

M-Tree vs. Metric Inverted Number of comparisons vs. top-k Number Nearest Terms =30

Outline Motivation Problem Definition Metric Inverted Index Retrieval Experiments Conclusions

We reduce the gap between Text IR and Multimedia Retrieval Our method achieves very good approximation (MAP = 98%) Our method improves drastically the efficiency (90%) over state-of-the-art methods