(1) Feature-point matching by D.J.Duff for CompVis 2007.11.15 Online: Feature Point Matching Detection,

Slides:



Advertisements
Similar presentations
Feature extraction: Corners
Advertisements

Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.
BRISK (Presented by Josh Gleason)
CSE 473/573 Computer Vision and Image Processing (CVIP)
Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
Distinctive Image Features from Scale- Invariant Keypoints Mohammad-Amin Ahantab Technische Universität München, Germany.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Lecture 6: Feature matching CS4670: Computer Vision Noah Snavely.
1 Interest Operators Find “interesting” pieces of the image –e.g. corners, salient regions –Focus attention of algorithms –Speed up computation Many possible.
Lecture 4: Feature matching
1 Interest Operator Lectures lecture topics –Interest points 1 (Linda) interest points, descriptors, Harris corners, correlation matching –Interest points.
Feature matching and tracking Class 5 Read Section 4.1 of course notes Read Shi and Tomasi’s paper on.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Feature extraction: Corners and blobs
(1) Feature-point matching by D.J.Duff for CompVis Online: Feature Point Matching Detection, Extraction.
Scale Invariant Feature Transform (SIFT)
Blob detection.
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detection.
CS4670: Computer Vision Kavita Bala Lecture 8: Scale invariance.
Scale-Invariant Feature Transform (SIFT) Jinxiang Chai.
Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Computer vision.
05 - Feature Detection Overview Feature Detection –Intensity Extrema –Blob Detection –Corner Detection Feature Descriptors Feature Matching Conclusion.
1 Interest Operators Harris Corner Detector: the first and most basic interest operator Kadir Entropy Detector and its use in object recognition SIFT interest.
Lecture 06 06/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Reporter: Fei-Fei Chen. Wide-baseline matching Object recognition Texture recognition Scene classification Robot wandering Motion tracking.
CSE 185 Introduction to Computer Vision Local Invariant Features.
CSCE 643 Computer Vision: Extractions of Image Features Jinxiang Chai.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Distinctive Image Features from Scale-Invariant Keypoints Ronnie Bajwa Sameer Pawar * * Adapted from slides found online by Michael Kowalski, Lehigh University.
Features Digital Visual Effects, Spring 2006 Yung-Yu Chuang 2006/3/15 with slides by Trevor Darrell Cordelia Schmid, David Lowe, Darya Frolova, Denis Simakov,
Distinctive Image Features from Scale-Invariant Keypoints David Lowe Presented by Tony X. Han March 11, 2008.
Edges.
Feature extraction: Corners and blobs. Why extract features? Motivation: panorama stitching We have two images – how do we combine them?
Features, Feature descriptors, Matching Jana Kosecka George Mason University.
A Tutorial on using SIFT Presented by Jimmy Huff (Slightly modified by Josiah Yoder for Winter )
Local features: detection and description
Features Jan-Michael Frahm.
CS654: Digital Image Analysis
Instructor: Mircea Nicolescu Lecture 10 CS 485 / 685 Computer Vision.
CSE 185 Introduction to Computer Vision Local Invariant Features.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Lecture 10: Harris Corner Detector CS4670/5670: Computer Vision Kavita Bala.
Keypoint extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Blob detection.
SIFT Scale-Invariant Feature Transform David Lowe
Interest Points EE/CSE 576 Linda Shapiro.
Distinctive Image Features from Scale-Invariant Keypoints
3D Vision Interest Points.
TP12 - Local features: detection and description
Lecture 4: Harris corner detection
Local features: detection and description May 11th, 2017
Image Primitives and Correspondence
CSE 455 – Guest Lectures 3 lectures Contact Interest points 1
From a presentation by Jimmy Huff Modified by Josiah Yoder
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Edge detection f(x,y) viewed as a smooth function
Lecture VI: Corner and Blob Detection
Lecture 5: Feature invariance
Lecture 5: Feature invariance
Detecting and analysing motion
Presentation transcript:

(1) Feature-point matching by D.J.Duff for CompVis Online: Feature Point Matching Detection, Extraction & Matching

(2) Feature-point matching by D.J.Duff for CompVis Online: Contents ● Feature point matching  What is it?  What is it for? ● Feature point detection  Moravec feature point detector  Harris corner detector  Scale space detection ● Feature point extraction & matching  Matching using templates ● Cross-correlation ● Problems  Other possibilities ● Summary

(3) Feature-point matching by D.J.Duff for CompVis Online: Feature point matching ● Feature Point:  A point on an image that is useful for image processing (by human or computer) because of the local properties in the image in the neighbourhood of the point. ● One thing a feature point might be useful for is matching. ● The matching process in general has 3 steps:  Feature point detection  Feature point extraction  Feature point matching Also motion estimation & salience maps (previous lectures)

(4) Feature-point matching by D.J.Duff for CompVis Online: Feature point matching ● Useful for  Object tracking  Motion detection  Object recognition  Multi-view reconstruction ● Stereo+ vision ● Structure from motion  Image stitching  Localisation  Simultaneous Localisation And Mapping

(5) Feature-point matching by D.J.Duff for CompVis Online: Feature point matching ● Useful for  Object tracking ● Useful for 3D localisation ● Resistant to occlusion (local features) ● Resistant to clutter (local features)  Motion detection ● Defeats aperture problem (distinctive features)

(6) Feature-point matching by D.J.Duff for CompVis Online: Feature point matching - phases 1.Feature Point Detection:  The process of finding such useful points in an image in the first place. One common example is the Harris “Corner Detector”. 2.Feature Point Extraction:  The process of finding a description of the local properties of an image (its features) around a feature point (this might just be a patch extracted from the image). 3.Feature Point Matching:  The process of using the local properties of images around feature points to identify the points across images that refer to the same points in the world

(7) Feature-point matching by D.J.Duff for CompVis Online: Matching between images Extraction

(8) Feature-point matching by D.J.Duff for CompVis Online: Matching between images Matching

(9) Feature-point matching by D.J.Duff for CompVis Online: Matching to a known textured object Extraction

(10) Feature-point matching by D.J.Duff for CompVis Online: Matching to a known textured object Matching

(11) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection ● Probably because it is not so well defined, feature point detection is an unsolved problem.  Change the matching method, and the criteria for a good feature point changes. ● May detection methods exist. Some examples are:  Moravec (1977)  Harris (1988)  Scale space extrema (1999) ● Often conceptualised as a search for a generalised “corner”, but other definitions work!

(12) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection ● General Desiderata  Points repeatably detectable under transforms  Computationally efficient  Easily localisable  High detection rate  Robust to noise ● Useful for  Finding candidates for point matching  Creating salience maps (see previous lecture on attention)

(13) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec ● Recall Moravec point detector  Used on the Stanford Cart in the 70s.  At each pixel an interest score is calculated and local maxima in this score are used (i.e. where the score at a pixel is higher than in all other pixels in the neighbourhood).  This score at a point is the change in intensity around the point in the direction that the intensity changes slowest – BUT you only check 4 key directions around the point.

(14) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec ● Recall Moravec point detector  Here is how to calculate the interest score:  Where P is the set of perturbations ● { (-1,1),(0,1),(1,0),(1,1)}  W is the local window. I image intensity. x, y, u, v pixel indices.  Example to follow.

(15) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Local Window W(x,y) Shifted Window W(x+1,y+1) Sum = (1-0)^2 + (0-1)^2 + (1-1)^2 + (0-0)^2 + (0-1)^2 + (1-1)^2 + (1-1)^2 = 3 4

(16) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Local Window W(x,y) Shifted Window W(x,y-1) Sum = (1-1)^2 + (1-0)^2 + (0-1)^2 + (1-0)^2 + (0-0)^2 + (0-1)^2 + (1-1)^2 + (1-1)^2 = 4 4 3

(17) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Shifted Windows 3 Minimum =

(18) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Minimum = 3

(19) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector

(20) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Thresholding

(21) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Performing non- maximal suppression

(22) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec Moravec point detector Corner points!

(23) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection ● Moravec point detector  Fast  Terrible with angles away from 45 deg  Has problems with noise  Not invariant to scale  Not invariant to rotation  In other words, obsolete. ● More robust detectors exist.

(24) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec ● Moravec point detector  Terrible with angles away from 45 deg Lots of false matches away from the diagonal

(25) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec ● Moravec point detector  Has problems with noise

(26) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection - Moravec ● Moravec point detector ● Implicit assumption is that points of interest are points where the image brightness changes fast in all directions. ● More recent approaches have attempted to create better detectors, incorporating the same basic assumption. ● But without the limitation of 4 cardinal directions.

(27) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● The Harris detector generalises the Moravec detector: 1.Attempting to calculate the strength of image change in a window in the direction of largest change and of smallest change (rather than in 4 principal directions, which allows for better rotational invariance) 2.Using a circular gaussian window rather than a square one (less noise) 3.Taking into account both the direction of minimum change in brightness and maximum change in brightness (to reduce the probability of picking up edges).

(28) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● The Harris detector generalises the Moravec detector: Strength of fastest change Strength of slowest change Strength in direction of slowest change is low Strength in direction of fastest change is high......so this must be an edge pixel!

(29) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? Estimate the X and Y gradients for all pixels in the region of your point of interest. (this can be done efficiently by convolution with a small mask e.g a sobel mask: )

(30) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? The sum of squares of these values will give us an idea of the strength of change in the x direction. The sum of squares of these values will give us an idea of the strength of change in the y direction

(31) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? Very similar to the Moravec operator… which wasn’t good enough for us! We need a way to calculate the strength of change in the directions of maximum & minimum change. The sum of squares of these values will give us an idea of the strength of change in the x direction. The sum of squares of these values will give us an idea of the strength of change in the y direction.

(32) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? With: Check the following special cases: - it’s the same as Moravec’s!

(33) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? ● Now we can analyse this matrix:

(34) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? ● It turns out that if we assume that then the maxima and minima of this function are the eigenvalues of matrix, (* how to see why?)

(35) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris Now let’s try to calculate M W from our data: ● How to calculate these strengths? Strength of fastest change Strength of slowest change

(36) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● How to calculate these strengths? We sum up the gradients for all nearby pixels, weighted by distance. Usually we use a circular gaussian window when summing to create our matrix M W.

(37) Feature-point matching by D.J.Duff for CompVis Online: ● How to calculate these directions? ● Actually, because it is only the relationship between the eigenvalues that is of interest, and not the values themselves, the eigenvalues of M are usually not computed directly. Instead the interest score is computed: ● Where k is some constant ~ Feature point detection – Harris

(38) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● The Harris detector generalises the Moravec detector: Moravec did not generalise to non-cardinal directions but the Harris operator does.  Generalisations of the Harris detector exist too, for larger stability with scaling (generally using pyramidal images).

(39) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – Harris ● The Harris detector has problems with scale changes Where is the interest point really? Up close From afar

(40) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Before getting into scale-space detection, recall your use of Difference of Gaussian (DoG) filters in salience maps - also often used is the very similar Laplace of Gaussian (LoG). ● It turned out that these were useful filters for finding the salience of each point in an image. ● Zero crossings in images filtered with DoG/LoG can also be used for finding edges. ● LoG/DoG are also useful for finding interest points.

(41) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Here is an image ● Here is the DoG of that image

(42) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Local maxima and minima in the DoG image correspond to interesting points in our original image… ● This is a new definition of interest point: DoG extrema. ● …i.e. points that have a larger or lower DoG response than all their neighbours.

(43) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● DoG extrema are about as good as Harris points:  they are resistant to rotation (since the filter is rotationally invariant)  they are relatively resistant to noise (because they are the response of a smooth filter) ● BUT:  Like Harris points they are not robust to changes in scale.

(44) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Problems with scaling motivate another concept – that of scale space. ● The concept of scale space motivates another tentative definition for feature points – DoG scale space extrema.

(45) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Let us examine this notion of scale space.

(46) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace Image Function Gaussian Function ● Let us examine this notion of scale space. Gaussian Scale Space * the star here is convolution

(47) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Let us examine this notion of scale space. Gaussian Scale Space * the star here is convolution

(48) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Let us examine this notion of scale space extrema. * the star here is convolution The scale space extrema of interest to us are maxima and minima in the DoG scale space function. Difference of Gaussian Scale Space Function Equivalent Form – Difference of Gaussian Scale Space Function (sometimes faster to compute)

(49) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Here is the scale space function for the DoG function... Difference of Gaussian Scale Space Function

(50) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Local extrema can be found by checking all pixels in the neighbourhood of each candidate pixel.

(51) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Local extrema can be found by checking all pixels in the neighbourhood of each candidate pixel. Lets check for interest points here Higher value Lower value Therefore our candidate point is not a local extrema

(52) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Local extrema can be found by checking all pixels in the neighbourhood of each candidate pixel. Lets check for interest points here But no nearby higher value Lower value Therefore our candidate point is a local extrema

(53) Feature-point matching by D.J.Duff for CompVis Online: Feature point detection – scalespace ● Local extrema can be found by checking all pixels in the neighbourhood of each candidate pixel. Local Extrema from Difference of Gaussian Scale Space Function

(54) Feature-point matching by D.J.Duff for CompVis Online: Feature point extraction ● Once a feature-point has been detected, but before we attempt to match it we may decide to process it (this is dubbed feature point extraction) to enable more efficient and more general matching. For example:  Extract a colour histogram from the image in the neighbourhood of the feature-point.  Extract a histogram of local gradients from the image in the neighbourhood of the feature-point.  Rotationally pre-align the image in the neighbourhood of the feature-point according to some criteria.  And so forth. “Local Patch”

(55) Feature-point matching by D.J.Duff for CompVis Online: Some further reading ● Feature points:  Trucco, E. & Verri, A. (1998). Introductory techniques for 3D computer vision. Section 4.3.  Demo:  More: ● Cross-correlation & template matching:  Russ, J.C. (2002). The image processing handbook, 4 th Ed. Chapter 10.  Demo:  More: ● SIFT:  Lowe, D.G. (2004). Distinctive image features from scale-invariant keypoints. International Journal of Computer Vision, 60(2). Pages Available OnlineAvailable Online  Implementations:  More: