1. Lecture 06 06/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.

2. Today Local invariant features –Detection of interest points Harris corner detection –Scale Estimation

3. What would we like our local features to be invariant to? Geometric Photometric Noise Blur Compression Artifacts

4. Main questions Where will the interest points come from? –What are salient features that we’ll detect in multiple views? How to describe a local region? How to establish correspondences, i.e., compute matches?

5. Finding Corners Key property: in the region around a corner, image gradient has two or more dominant directions Corners are repeatable and distinctive C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“ Proceedings of the 4th Alvey Vision Conference: pages 147--151. "A Combined Corner and Edge Detector.“ Source: Lana Lazebnik

6. Local measures of uniqueness Suppose we only consider a small window of pixels What defines whether a feature is a good or bad candidate? Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

7. Feature detection “flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions Local measure of feature uniqueness How does the window change when you shift by a small amount? Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.

8. Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by Sum of the Squared Differences (SSD) this defines an SSD “error” E(u,v): Feature detection: the math W

9. Taylor Series expansion of I: If the motion (u,v) is small, then first order approx is good Plugging this into the formula on the previous slide… Small motion assumption

10. Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences this defines an “error” of E(u,v): Feature detection: the math W =

11. Quick eigenvalue/eigenvector review The eigenvectors of a matrix A are the vectors x that satisfy: The scalar is the eigenvalue corresponding to x The eigenvalues are found by solving: In our case, A = H is a 2x2 matrix, so we have The solution: Once you know, you find x by solving

12. Feature detection: the math This can be rewritten: Eigenvalues and eigenvectors of H Define shifts with the smallest and largest change (E value) x + = direction of largest increase in E. + = amount of increase in direction x + x - = direction of smallest increase in E. - = amount of increase in direction x + x - x +

13. Feature detection: the math How are +, x +, -, and x - relevant for feature detection? What’s our feature scoring function? Want E(u,v) to be large for small shifts in all directions the minimum of E(u,v) should be large, over all unit vectors [u v] this minimum is given by the smaller eigenvalue ( - ) of H

14. Feature detection summary Here’s what you do Compute the gradient at each point in the image Create the H matrix from the entries in the gradient Compute the eigenvalues. Find points with large response ( - > threshold) Choose those points where - is a local maximum as features

15. Feature detection summary Here’s what you do Compute the gradient at each point in the image Create the H matrix from the entries in the gradient Compute the eigenvalues. Find points with large response ( - > threshold) Choose those points where - is a local maximum as features

16. The Harris operator - is a variant of the “Harris operator” for feature detection The trace is the sum of the diagonals, i.e., trace(H) = h 11 + h 22 Very similar to - but less expensive (no square root) Called the “Harris Corner Detector” or “Harris Operator” Lots of other detectors, this is one of the most popular Here’s another variant: α : constant (0.04 to 0.06)

17. Interpreting the eigenvalues 1 2 “Corner” 1 and 2 are large, 1 ~ 2 ; E increases in all directions 1 and 2 are small; E is almost constant in all directions “Edge” 1 >> 2 “Edge” 2 >> 1 “Flat” region Classification of image points using eigenvalues of M:

18. Corner response function “Corner” R > 0 “Edge” R < 0 “Flat” region |R| small α : constant (0.04 to 0.06)

19. Harris Detector: Workflow Slide adapted form Darya Frolova, Denis Simakov, Weizmann Institute.

20. Harris Detector: Workflow Compute corner response R

21. Harris Detector: Workflow Find points with large corner response: R>threshold

22. Harris Detector: Workflow Take only the points of local maxima of R

23. Harris Detector: Workflow

24. Harris Detector: Properties Rotation invariance Ellipse rotates but its shape (i.e. eigenvalues) remains the same Corner response R is invariant to image rotation

25. Harris Detector: Properties Not invariant to image scale All points will be classified as edges Corner !

26. How can we detect scale invariant interest points?

27. Scale-invariant features: Blobs

28. Recall: Edge detection f Source: S. Seitz Edge Derivative of Gaussian Edge = maximum of derivative

29. Edge detection, Take 2 f Edge Second derivative of Gaussian (Laplacian) Edge = zero crossing of second derivative Source: S. Seitz

30. From edges to blobs Edge = ripple Blob = superposition of two ripples Spatial selection: the magnitude of the Laplacian response will achieve a maximum at the center of the blob, provided the scale of the Laplacian is “matched” to the scale of the blob maximum

31. Scale selection We want to find the characteristic scale of the blob by convolving it with Laplacians at several scales and looking for the maximum response However, Laplacian response decays as scale increases: Why does this happen? increasing σ original signal (radius=8)

32. Scale normalization The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases

33. Scale normalization The response of a derivative of Gaussian filter to a perfect step edge decreases as σ increases To keep response the same (scale-invariant), must multiply Gaussian derivative by σ Laplacian is the second Gaussian derivative, so it must be multiplied by σ 2

34. Effect of scale normalization Scale-normalized Laplacian response Unnormalized Laplacian response Original signal maximum

35. Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D

36. Blob detection in 2D Laplacian of Gaussian: Circularly symmetric operator for blob detection in 2D Scale-normalized:

37. Scale selection At what scale does the Laplacian achieve a maximum response for a binary circle of radius r? r imageLaplacian

38. Scale selection The 2D Laplacian is given by Therefore, for a binary circle of radius r, the Laplacian achieves a maximum at r image Laplacian response scale (σ) (up to scale)

39. Characteristic scale We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection." International Journal of Computer Vision 30 (2): pp 77--116."Feature detection with automatic scale selection."

40. Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial K. Grauman, B. Leibe What Is A Useful Signature Function? Laplacian-of-Gaussian = “blob” detector 40 K. Grauman, B. Leibe

41. Characteristic scale We define the characteristic scale as the scale that produces peak of Laplacian response characteristic scale T. Lindeberg (1998). "Feature detection with automatic scale selection." International Journal of Computer Vision 30 (2): pp 77--116."Feature detection with automatic scale selection." Source: Lana Lazebnik

42. Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial K. Grauman, B. Leibe Laplacian-of-Gaussian (LoG) Interest points: Local maxima in scale space of Laplacian-of- Gaussian K. Grauman, B. Leibe       List of (x, y, σ )

43. Scale-space blob detector: Example Source: Lana Lazebnik

44. Scale-space blob detector: Example Source: Lana Lazebnik

45. Scale-space blob detector: Example Source: Lana Lazebnik