1. Detecting Patterns So far Specific patterns (eyes) Generally useful patterns (edges) Also (new) “Interesting” distinctive patterns ( No specific pattern: anything that can be recognized easily from one image of an object to the next) Examples: corners (matching points between video frames) SIFT, HOG (recognizing objects)

2. Finding corners One motivation: panorama stitching We have two images – how do we combine them?

3. Finding corners One motivation: panorama stitching We have two images – how do we combine them? Step 1: Find corners Step 2: Match corners across images

4. Finding corners One motivation: panorama stitching We have two images – how do we combine them? Step 1: extract features Step 2: match features Step 3: align images

5. Why corners? Repeatability The same feature can be found in several images despite geometric and photometric transformations Locality Accurately specifies location in the image.

6. Applications Corner detection used for: Motion tracking Image alignment 3D reconstruction Object recognition Indexing and database retrieval Robot navigation

7. Another motivation Edge detectors fail at corners!

8. Another motivation Edge detectors fail at corners! Edge detector with too much smoothing misses the black/white transitions near corner

9. Basic Idea: detecting (not just) corners Look in small image windows (more efficient) ‘Corner’ distinctive  easily recognized Precise location: Shifting `corner’ in any direction changes window’s brightness pattern “edge”: no change along edge direction “corner”: significant change in all directions “flat” region: no change in all directions

10. Not just corners… Can use any distinctive brightness pattern that can be assigned a definite location. I(x, y) E(u, v) E(0,0) E(3,2)

11. Detecting `corners’ Main criterion: Over given window, every direction should have big brightness changes  Big gradients in all directions First try: Both and should be large in window  Look for large values of and (somewhere)

12. Problem Diagonal brightness change is not a corner

13. Detecting corners Second try. Change to rotated coordinate system (x’,y’) Must also have large and in new coordinates After rotation:

14. Large and

16. { {

17. So really we want large for any direction

18. Calculate the max and min of over using singular value decomposition (SVD). Idea: Suppose M is diagonal

19. So if then Min = Max = We have a `corner’ if and both large

20. If M diagonal problem solved. But we can always find rotated coordinates so M is diagonal!

21. Harris Detector: Steps

22. Compute corner response R

23. Harris Detector: Steps Find points with large corner response: R>threshold

24. Harris Detector: Steps Take only the points of local maxima of R

25. Harris Detector: Steps

26. Spot detector? Not corner, but also a distinctive localizable pattern More generally: blob detector (detects regions of roughly uniform brightness)  Can use to detect sizes of important image structures eg, SIFT. Use to detect faces of any size in the image.

27. Blob detector: Example