1. Coherency Sensitive Hashing (CSH) Simon Korman and Shai Avidan Dept. of Electrical Engineering Tel Aviv University ICCV2011 | 13th International Conference on Computer Vision

2. Outline Introduction Locality Sensitive Hashing for Finding Nearest Neighbors Coherency Sensitive Hashing Experiments Conclusions

3. Introduction(1/2) Patch : k*k block 1920 1080 Find the closest patch linear search(search every patch one by one) query

4. Introduction(2/2) (Streaming) Massive Data Sets => High Dimensional Vectors E.g. 8*8 patch => v = [ v 1, v 2,..., v i, …, v N ], dimension N = 64 Linear search = find nearest neighbor For very large databases of high-dimensional items Time-consuming Needs to find Approximate Nearest Neighbors (ANN) for each patch in real time. Curse of dimensionality Existing ANN methods include trees and hashes. KD-trees Locality-Sensitive Hashing(LSH)

5. A collision occurs when two points hash to the same value Hash table bucket Look up table Projection = hash function Repeat L times =>the closest point will collide most times Repeat L times =>the closest point will collide most times Random line Locality Sensitive Hashing for Finding Nearest Neighbors(1/2)

6. Locality Sensitive Hashing for Finding Nearest Neighbors(2/2) Hash function: a is d-dimensional random vector r is predefine integer Constant width of each quantization bin b is random value from range [0, r] To balance quantization error v is the original vector 1. indexing 2. search

7. Coherency Sensitive Hashing (CSH) s=1, 4x4 kernel White:1, Black:-1

8. Coherency Sensitive Hashing (CSH)

9. 4.2.1 Candidate Creation Patch a, a 1,a 2 of image A; Patch b, b 1,b 2 of image B : Each entry can store k patches from each image=>total k+2*(k+1)+k=4k+2 candidates

10. 4.2.2 Candidate Ranking Given the candidate set (of size 4k + 2), to find the nearest one. Main overall time consumer Approximate the process, have a little impact, greatly reduce time. Use Walsh Hadamard (WH) projections Already computed in the indexing stage. Accumulating the projections of the differences of patches on the WH kernels.

11. Experiments We collected 133 pairs of images, taken from 1080p HD

12. Experiments We computed the exact nearest neighbor match to serve as a ground truth. A novel algorithm PatchMatch [4] to compare : Not as accurate as LSH or KD-trees. So fast. The key to this speedup is spatial coherent. [4] C. Barnes, E. Shechtman, A. Finkelstein, and D. B. Goldman. PatchMatch: A randomized correspondence algorithm for structural image editing. In SIGGRAPH, 28(3), 2009. Error : not the same match with ground truth

13. Image Reconstruction Reconstruct image A, use image B Such reconstructions are very common in many applications. image editing (e.g. retargetting, inpainting), image denoising and super-resolution… It simply replaces each pixel with the average of the corresponding pixels. Image A Image B

19. More results on web : http://www.eng.tau.ac.il/~simonk/CSH/index.html

20. Conclusions We proposed an algorithm for computing ANN fields termed Coherency Sensitivity Hashing. Follows LSH search scheme But combines image coherency cues It was shown to be faster and more accurate than PatchMatch.