1. 776 Computer Vision Jan-Michael Frahm Fall 2015

2. SIFT-detector Problem: want to detect features at different scales (sizes) and with different orientations!

3. SIFT-detector Scale and image-plane-rotation invariant feature descriptor [Lowe 2004] - Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging parameters

4. SIFT-detector Scale = 2.5 Rotation = 45 0 Empirically found to perform very good [Mikolajczyk 2003]

5. Difference of Gaussian for Scale invariance Difference-of-Gaussian with constant ratio of scales is a close approximation to Lindeberg’s scale-normalized Laplacian [Lindeberg 1998] Gaussian Difference of Gaussian

6. Difference of Gaussian for Scale invariance Difference-of-Gaussian with constant ratio of scales is a close approximation to Lindeberg’s scale-normalized Laplacian [Lindeberg 1998]

7. Key point localization Detect maxima and minima of difference-of-Gaussian in scale space Fit a quadratic to surrounding values for sub-pixel and sub- scale interpolation (Brown & Lowe, 2002) Taylor expansion around point: Offset of extremum (use finite differences for derivatives):

8. Orientation normalization Histogram of local gradient directions computed at selected scale Assign principal orientation at peak of smoothed histogram Each key specifies stable 2D coordinates (x, y, scale, orientation)

9. Example of keypoint detection Threshold on value at DOG peak and on ratio of principle curvatures (Harris approach) (a) 233x189 image (b) 832 DOG extrema (c) 729 left after peak value threshold (d) 536 left after testing ratio of principle curvatures courtesy Lowe XX

10. SIFT vector formation SIFT vector formation Thresholded image gradients are sampled over 16x16 array of locations in scale space Create array of orientation histograms 8 orientations x 4x4 histogram array = 128 dimensions © Lowe example 2x2 histogram array

11. Sift feature detector

12. Goal 12 Image Patch [0, 0, 1, 0, 1, 1, 0, 1, …] Binary Descriptor slide: J. Heinly

13. BRIEF Binary Robust Independent Elementary Features Calonder et al. ECCV 2010 slide: J. Heinly

14. Feature Description 14 slide: J. Heinly

15. BRIEF: Method 15 Descriptor: 011 0 10… slide: J. Heinly

16. BRIEF: Sampling 16 Endpoints from 2D Gaussian slide: J. Heinly

17. BRIEF: Descriptor 17 slide: J. Heinly

18. BRIEF: Descriptor 128, 256, or 512 bits o 16, 32, or 64 bytes Hamming distance matching 18 slide: J. Heinly

19. BRIEF: Summary Pros o Highly efficient Cons o No scale invariance o No rotation invariance o Sensitive to noise 19 slide: J. Heinly

20. ORB An Efficient Alternative to SIFT or SURF Rublee et at. ICCV 2011 slide: J. Heinly

21. Fast Corner Detector Continuous arc of pixels all much o brighter than center pixel p (brighter than p+threshold) or o darker than center pixel p (darker than p-threshold) ≥ 12 pixel brighter or darker rapid rejection by testing pixel 1,9,5, and 13 non-maxima suppression adapted from Ed Rosten

22. ORB: Method 22 Feature Direction Descriptor: 011 0 10… slide: J. Heinly

23. ORB: Rotation Invariance Intensity Centroid Feature Direction adopted from : J. Heinly moment of patch: choose orientation corrected gradients for descriptor generation

24. ORB: Descriptor 24 Low Endpoint Correlation High Candidate ArrangementLearned Arrangement slide: J. Heinly

25. ORB: Summary Pros o Efficient o Rotation invariance Cons o No scale invariance o Sensitive to noise 25 slide: J. Heinly

26. BRISK Binary Robust Invariant Scalable Keypoints Leutenegger et al. ICCV 2011 slide: J. Heinly

27. BRISK: Method 27 Descriptor: 011 0 10… slide: J. Heinly

28. BRISK: Rotation Invariance 28 Long-distance comparisons Gradient direction slide: J. Heinly

29. BRISK: Scale Invariance 29 Find maximum response slide: J. Heinly

30. BRISK: Descriptor 30 Centers: BLUE Gaussian: RED 2D Gaussian around each feature Robust to noise slide: J. Heinly

31. BRISK: Descriptor 31 Centers: BLUE Gaussian: RED 512 Comparisons 64 bytes Avoid short-distance comparisons slide: J. Heinly Rotation normalization through patch gradient

32. BRISK: Summary Pros o Efficient o Rotation invariance o Scale invariance o Robust to noise 32 slide: J. Heinly

33. Summary 33 BRIEF BRISK ORB Efficient Rotation Efficient Rotation Scale Noise slide: J. Heinly

34. Results: BRIEF 34 Increased Difficulty No Rotation Dimensionali ty Reduction Recognition Rate % slide: J. Heinly

35. Results: BRIEF 35 Increased Difficulty Recognition Rate % slide: J. Heinly

36. Results: ORB 36 Percentage of Inliers Angle of Rotation ORB slide: J. Heinly

37. Results: BRISK 37 slide: J. Heinly

38. Results: BRISK 38 slide: J. Heinly

39. Results Many more tests… 39 Key Observation: Results are comparable to traditional feature descriptors. slide: J. Heinly

40. Efficiency 40 SURFSIFTBRIEFORBBRISK 1.019.00.0270.0700.087 37.214.211.5 Normalized Time Speedup slide: J. Heinly

41. Summary 41 BRIEF BRISK ORB Efficient Binary Descriptors slide: J. Heinly

42. BRIEFBRISKORB slide: J. Heinly