Detecting Patterns So far Specific patterns (eyes) Generally useful patterns (edges) Also (new) “Interesting” distinctive patterns ( No specific pattern: anything that can be recognized easily from one image of an object to the next) Examples: corners (matching points between video frames) SIFT, HOG (recognizing objects)

Finding corners One motivation: panorama stitching We have two images – how do we combine them?

Finding corners One motivation: panorama stitching We have two images – how do we combine them? Step 1: Find corners Step 2: Match corners across images

Finding corners One motivation: panorama stitching We have two images – how do we combine them? Step 1: extract features Step 2: match features Step 3: align images

Why corners? Repeatability The same feature can be found in several images despite geometric and photometric transformations Locality Accurately specifies location in the image.

Applications Corner detection used for: Motion tracking Image alignment 3D reconstruction Object recognition Indexing and database retrieval Robot navigation

Another motivation Edge detectors fail at corners!

Another motivation Edge detectors fail at corners! Edge detector with too much smoothing misses the black/white transitions near corner

Basic Idea: detecting (not just) corners Look in small image windows (more efficient) ‘Corner’ distinctive  easily recognized Precise location: Shifting `corner’ in any direction changes window’s brightness pattern “edge”: no change along edge direction “corner”: significant change in all directions “flat” region: no change in all directions

Not just corners… Can use any distinctive brightness pattern that can be assigned a definite location. I(x, y) E(u, v) E(0,0) E(3,2)

Detecting `corners’ Main criterion: Over given window, every direction should have big brightness changes  Big gradients in all directions First try: Both and should be large in window  Look for large values of and (somewhere)

Problem Diagonal brightness change is not a corner

Detecting corners Second try. Change to rotated coordinate system (x’,y’) Must also have large and in new coordinates After rotation:

Large and

{ {

So really we want large for any direction

Calculate the max and min of over using singular value decomposition (SVD). Idea: Suppose M is diagonal

So if then Min = Max = We have a `corner’ if and both large

If M diagonal problem solved. But we can always find rotated coordinates so M is diagonal!

Harris Detector: Steps

Compute corner response R

Harris Detector: Steps Find points with large corner response: R>threshold

Harris Detector: Steps Take only the points of local maxima of R

Harris Detector: Steps

Spot detector? Not corner, but also a distinctive localizable pattern More generally: blob detector (detects regions of roughly uniform brightness)  Can use to detect sizes of important image structures eg, SIFT. Use to detect faces of any size in the image.

Blob detector: Example