1. Distinctive Image Features from Scale-Invariant Keypoints
David G. Lowe
International Journal of Computer Vision(IJCV), 2004

2. Extracting distinctive invariant features
Points are individually ambiguous
More unique matches are possible with small regions of images

3. Desired properties for features
Invariant: invariant to scale, rotation, affine, illumination and noise for robust matching across a substantial range of affine distortion, viewpoint change and so on.
Distinctive: a single feature can be correctly matched with high probability

4. Moravec corner detector (1980)
We should easily recognize the point by looking through a small window
Shifting a window in any direction should give a large change in intensity

5. Moravec corner detector
flat

6. Moravec corner detector
flat

7. Moravec corner detector
flat
edge

8. Moravec corner detector
isolated point
flat
edge

9. Moravec corner detector
Change of intensity for the shift [u,v]:
Intensity
Window function
Shifted intensity
Four shifts: (u,v) = (1,0), (1,1), (0,1), (-1, 1)
Problem: responds too strong for edges because only minimum of E is taken into account

10. Harris corner detector [1992]
Consider all small shifts by Taylor’s expansion
W(x, y): Gaussian function
=>
M: 2x2 Hessian matrix,
1, 2 – eigenvalues of M

11. Harris corner detector1 2
Corner
1 and 2 are large, 1 ~ 2; E increases in all directions
edge 1 >> 2
edge 2 >> 1
flat
Measure of corner response:
Classification of image points using eigenvalues of M:

12. Harris Detector: Problem
non-invariant to image scale!
All points will be classified as edges
Corner !

13. Scale-invariant feature transform (SIFT)
Scale-invariant feature transform (or SIFT) is an algorithm to detect and describe local features in images.
Distinctive features
Invariant to image scale, rotation and affine distortion
Applied locally on key-points
Based upon the image gradients in a local neighborhood

14. Scale-space extrema detection Keypoint localization
SIFT stages:
Scale-space extrema detection
Keypoint localization
Orientation assignment
Keypoint descriptor
detector
descriptor
local descriptor

15. 1. Detection of scale-space extrema
Convolution with a variable-scale Gaussian
Difference-of-Gaussian (DoG) filter

16. Scale space  doubles for the next octave 2
K=2(1/s), s+3 images for each octave
k∙ 

17. DoG Efficient function to compute
A close approximation to the scale-normalized Laplacian of Gaussian

18. 2. Keypoint localization
X is selected if it is larger or smaller than all 26 neighbors

19. Decide scale sampling frequency

20. Pre-smoothing
 =1.6, plus a double expansion

21. 2. Accurate keypoint localization
Reject points with low contrast and poorly localized along an edge
If has offset larger than 0.5, sample point is changed.
If is less than 0.03 (low contrast), it is discarded.

22. Eliminating edge responses
Let
Keep the points with
r=10

24. 3. Orientation assignment
By assigning a consistent orientation, the keypoint descriptor can be orientation invariant.
For a keypoint, L is the image with the closest scale
36-bin orientation histogram over 360°
weighted by m
Peak is the dominant orientation
Local peak within 80% creates multiple orientations
About 15% has multiple orientations

25. 4. Local image descriptor
Image gradients are sampled over 16x16 array of locations in scale space
Create array of orientation histograms
8 orientations x 4x4 histogram array = 128 dimensions

26. Recognition examples