1 Image Matching using Local Symmetry Features Daniel Cabrini Hauagge Noah Snavely Cornell University

2 Outline Introduction Local Symmetry Scale Space Local Symmetry Features Experimental Results Conclusion

3 Introduction Analysis of symmetry : A long –standing problem in computer vision.

4 Introduction

5 Local Symmetry Method- takes an image as input, and computes local symmetry scores over the image and across scale space. Scoring local symmetries Gradient histogram-based score

6 Local Symmetry Bilateral symmetry 2n-fold rotational symmetry For both symmetry types

7 Local Symmetry

8 Scoring local symmetries Define our score Symmetry type If the image f exhibits perfect symmetry types at location p, then f(q) = f(M s,p (q)) for all q. Distance function d(q,r)= | f(q)-f(r) | Weight mask gives the importance of each set of corresponding point pairs around the center point p in determining the symmetry score at p.

9 Scoring local symmetries Define a function that we call the local symmetry distance SS = L*SD

10 Gradient histogram-based score

11 Scale space For the purpose of feature detection, we choose a different function for r is the distance to the point of interest A is a normalization constant r 0 is the ring radius ψ controls the width of the ring

12 Scale space

13 Local Symmetry Features By finding local maxima of the score, and feature description, by building a feature vector from local symmetry scores Feature detector Feature descriptor

14 Feature detector Using the SYM-IR function as a detector. Represent the support of each feature as a circle of radius s centered at (x,y) in the original image.

15 Feature detector

16 Feature detector

17 Feature descriptor SYMD encodes the distribution of the three SYM-I scores around a feature location at the detected scale

18 Experimental Results Evaluate the detector, comparing its repeatability to that of the common DoG and MSER detectors. Evaluating detections Evaluating descriptors

19 Evaluating detections For each image pair(I 1,I 2 ) in the dataset, and each detector, we detect sets of keypoints K 1 and K 2, and compare these detections using the known homography H 12 mapping points from I 1 to I 2. K1 is rescaled by a factor s so that it has a fixed area A ; we denote this scaled detection K 1 S also applied to K 2. Finally, the relative overlap of the support regions of H 12 K 1 S and K 2 gives an overlap score.

20 Evaluating detections

21 Evaluating detections

22 Evaluating descriptors A precision-call curve that summarizes the quality of the match scores.

23 Evaluating descriptors We generate two sets of perfectly matched synthetic detections by creating a set of keypoints K 1 on a grid in I 1 (in our experiments the spacing between points is 25 pixels and the scale of each keypoint is set to 6.25). We then map these keypoints to I 2 using H 12, creating a matched set of keys K 2. We discard keypoints whose support regions are not fully within the image.

24 Evaluating descriptors

25 Evaluating descriptors

26 Conclusion To evaluate our method, we created a new dataset of image pairs with dramatic appearance changes, and showed that our features are more repeatable, and yield complementary information, compared to standard features such as SIFT.

27 Thanks for your listening.