What Is the Most Efficient Way to Select Nearest Neighbor Candidates for Fast Approximate Nearest Neighbor Search? Masakazu Iwamura, Tomokazu Sato and Koichi Kise (Osaka Prefecture University, Japan) ICCV’2013 Sydney, Australia

Finding similar data  Basic but important problem in information processing  Possible applications include  Near-duplicate detection  Object recognition  Document image retrieval  Character recognition  Face recognition  Gait recognition  A typical solution: Nearest Neighbor (NN) Search 2

Finding similar data by NN Search  Desired properties  Fast and accurate  Applicable to large-scale data 3 The paper presents a way to realize faster approximate nearest neighbor search for certain accuracy Benefit from improvement of computing power

Contents  NN and Approximate NN Search  Performance comparison  Keys to improve performance 4

Contents  NN and Approximate NN Search  Performance comparison  Keys to improve performance 5

Nearest Neighbor (NN) Search  This is a problem that the true NN is always found  In a naïve way 6 NN Data Query For more data, more time is required For more data, more time is required

Nearest Neighbor (NN) Search  Finding nearest neighbor efficiently 7 Before query is given 1.Index data NN 1.Select search regions 2.Calculate distances of selected data After query is given The true NN must be contained in the selected search regions Ensuring this takes so long time Search regions

Approximate Nearest Neighbor Search  Finding nearest neighbor more efficiently 8 NN Search regions Much faster “Approximate” means that the true NN is not guaranteed to be retrieved

Performance of approximate nearest neighbor search 9 There is a trade-off relationship! Accuracy Computational time Shorter comp. time  Lower accuracy Stronger approximation HighLow Short Long

Contents  NN and Approximate NN Search  Performance comparison  Keys to improve performance 10

ANN search on 100M SIFT features BAD GOOD Selected results

ANN search on 100M SIFT features BAD GOOD IMI (Babenko 2012) IVFADC (Jegou 2011) Selected results

ANN search on 100M SIFT features BAD GOOD IMI (Babenko 2012) IVFADC (Jegou 2011) BDH (Proposed method) 2.0 times 4.5 times 9.4 times 2.9 times Selected results

ANN search on 100M SIFT features BAD GOOD IMI (Babenko 2012) IVFADC (Jegou 2011) BDH (Proposed method) 2.0 times 4.5 times 9.4 times 2.9 times The novelty of BDH was reduced by IMI before we succeeded in publishing it… (For more detail, check out the Wakate program on Aug. 1) Selected results

ANN search on 100M SIFT features BAD GOOD IMI (Babenko 2012) IVFADC (Jegou 2011) BDH (Proposed method) 2.0 times 4.5 times 9.4 times 2.9 times So-called binary coding is not suitable for fast retrieval but for saving memory usage Selected results

Contents  NN and Approximate NN Search  Performance comparison  Keys to improve performance 16

Keys to improve performance  Select search regions in subspaces  Find the closest ones in the original space efficiently 17

Keys to improve performance  Select search regions in subspaces  Find the closest ones in the original space efficiently 18

Select search regions in subspaces  In past methods (IVFADC, Jegou 2011 & VQ-index, Tuncel 2002) Search regions Query Indexed by k-means clustering

Select search regions in subspaces  In past methods (IVFADC, Jegou 2011 & VQ-index, Tuncel 2002) Search regions Query Indexed by k-means clustering Taking very much time to select the search regions Proven to be the least quantization error Pros. Cons. Indexed by vector quantization

Select search regions in subspaces  In the past state-of-the-art (IMI, Babenko 2012) Feature vectors Divide into two or more Calculate distances in subspaces Calculate distances in subspaces Select the regions in the original space Indexed by k-means clustering

Select search regions in subspaces  In the past state-of-the-art (IMI, Babenko 2012) Feature vectors Divide into two or more Calculate distances in subspaces Calculate distances in subspaces Select the regions in the original space Less accurate (More quantization error) Less accurate (More quantization error) Much less processing time Pros. Cons. > Indexed by product quantization

Keys to improve performance  Select search regions in subspaces  Find the closest ones in the original space efficiently 23

Find the closest search regions in original space  In the past state-of-the-art (IMI, Babenko 2012) Centroid in original space Search regions are selected in the ascending order of distances in the original space Subspace 2 Subspace 1 Distances in subspace 2 Distances in subspace 1 Centroid in subspace

Find the closest search regions in original space  In the past state-of-the-art (IMI, Babenko 2012) Centroid in original space Subspace 2 Subspace 1 Distances in subspace 2 Distances in subspace 1 Centroid in subspace This can be done more efficiently with the branch and bound method It does not consider the order of selecting buckets Search regions are selected in the ascending order of distances in the original space

Find the closest search regions in original space efficiently  In the proposed method Centroid in original space Subspace 2 Subspace 1 Centroid in subspace Assume that upper limit is set to 8 Distances in subspace 1 Distances in subspace 2

Find the closest search regions in original space efficiently  In the proposed method Centroid in original space Subspace 2 Subspace 1 Centroid in subspace Distances in subspace 2 Distances in subspace Assume that upper limit is set to 8 Max 8 0 0

Find the closest search regions in original space efficiently  In the proposed method Centroid in original space Subspace 2 Subspace 1 Centroid in subspace Distances in subspace 2 Distances in subspace Assume that upper limit is set to 8 Max

Find the closest search regions in original space efficiently  In the proposed method Centroid in original space Subspace 2 Subspace 1 Centroid in subspace Distances in subspace 2 Distances in subspace Assume that upper limit is set to 8 Max

Find the closest search regions in original space efficiently  In the proposed method Centroid in original space Subspace 2 Subspace 1 Centroid in subspace Distances in subspace 2 Distances in subspace Assume that upper limit is set to 8 Max

Find the closest search regions in original space efficiently  In the proposed method  The upper and lower bounds are increased in a step-by-step manner until enough number of data are selected 31

What Is the Most Efficient Way to Select Nearest Neighbor Candidates for Fast Approximate Nearest Neighbor Search? Masakazu Iwamura, Tomokazu Sato and Koichi Kise (Osaka Prefecture University, Japan) ICCV’2013 Sydney, Australia