1. CVPR 2006 New York City Spatial Random Partition for Common Visual Pattern Discovery Junsong Yuan and Ying Wu EECS Dept. Northwestern Univ. {j-yuan,yingwu}@northwestern.edu ICCV 2007

2. ICCV 2007 Rio de Janeiro, Brazil2 The Problem Can you find common posters in the two images?

3. ICCV 2007 Rio de Janeiro, Brazil3 Challenges No prior knowledge of the common patterns –What are they ? appearances –Where are they ? locations –How large are they ? scales –How many of them ? number of instances Computationally demanding –Exponentially large solution space –Large image dataset Robust similarity matching

4. ICCV 2007 Rio de Janeiro, Brazil4 Related Work Pattern Discovery by Matching Visual Words –J. Sivic and A. Zisserman, CVPR04 –J. Yuan, Y. Wu and M.Yang, CVPR07 –S. Nowozin, K. Tsuda, T. Uno, T. Kudo and G. Bakir, CVPR07 –T. Quack, V. Ferrari, B. Leibe and L. V. Gool, ICCV07 –… Pattern Discovery by Direct Matching –O. Boiman and M. Irani, ICCV05, NIPS06 –K. Grauman and T. Darrell, CVPR06, NIPS07 –K.-K. Tan and C.-W. Ngo, ICCV05 –N. Ahuja and S. Todorovic, CVPR06, ICCV07 –…

5. ICCV 2007 Rio de Janeiro, Brazil5 Spatial Random Partition

6. ICCV 2007 Rio de Janeiro, Brazil6 Visual Primitives Visual Primitives: Scale Invariant Feature Transformation (SIFT, D.Lowe, IJCV04 ) Locality Sensitive Hashing (LSH) for matching visual primitives –For each visual primitive, search for its matches from other images, based on Euclidean distance

7. ICCV 2007 Rio de Janeiro, Brazil7 Matching Subimages A many-to-many assignment problem. Fast approximation by set intersection: where is the # of visual primitives in the subimage

8. ICCV 2007 Rio de Janeiro, Brazil8 Another View: Max Flow Visual primitives subimage Problem: Matching two sets of m and n points (feature vectors) Fast Approximate Solution: set intersection (linear complexity)

9. ICCV 2007 Rio de Janeiro, Brazil9 An Example Final estimation of similarity score: = 3

10. ICCV 2007 Rio de Janeiro, Brazil10 Voting for Common Patterns

11. ICCV 2007 Rio de Janeiro, Brazil11 Asymptotic Property Theorem: Given two pixel i and j, where i locates in a common pattern while j locates in the background, let and the total votes i and j receives regarding to K random partitions. Both and are discrete random variables and we have Proof: using the weak law of large numbers, see the Appendix for details

12. ICCV 2007 Rio de Janeiro, Brazil12 Localization of Common Patterns

13. ICCV 2007 Rio de Janeiro, Brazil13 Various Number of Partitions

14. ICCV 2007 Rio de Janeiro, Brazil14

15. ICCV 2007 Rio de Janeiro, Brazil15 Image Irregularity Detection Differences from common pattern discovery –disocver unpopular subimages instead of popular ones –Adjust voting weight proportional to the subimage size: the larger the unpopular subimage, the more possible it contains an irregular pattern

16. ICCV 2007 Rio de Janeiro, Brazil16 Evaluation Collect 8 image datasets, each contains 4-8 images. An image dataset contains 1-3 common patterns each has 2-4 instances (* indicates the dataset containing multiple common patterns ) Comparisons of computational complexity, around 12 sec. for 2 images J. Sivic & A. Zisserman 04 O. Boiman & M. Irani 05

17. ICCV 2007 Rio de Janeiro, Brazil17 Conclusion A novel spatial random partition method for common pattern discovery and irregularity detection in images –No construction of visual vocabularies –Trade-off of performance and efficiency by the total number of random partitions –Efficient by using LSH and approximate matching between subimages –Theoretically justified by the asymptotic property of the algorithm