Bart M. ter Haar Romeny.  Question: can top-points be used for object- retrieval tasks?

Slides:



Advertisements
Similar presentations
Feature extraction: Corners
Advertisements

Ter Haar Romeny, ICPR 2010 Introduction to Scale-Space and Deep Structure.
Distinctive Image Features from Scale-Invariant Keypoints David Lowe.
Scale & Affine Invariant Interest Point Detectors Mikolajczyk & Schmid presented by Dustin Lennon.
Presented by Xinyu Chang
Dynamic Occlusion Analysis in Optical Flow Fields
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Fitting: The Hough transform. Voting schemes Let each feature vote for all the models that are compatible with it Hopefully the noise features will not.
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
The Image Histogram.
Object Recognition using Invariant Local Features Applications l Mobile robots, driver assistance l Cell phone location or object recognition l Panoramas,
Distinctive Image Features from Scale- Invariant Keypoints Mohammad-Amin Ahantab Technische Universität München, Germany.
Uncertainty Representation. Gaussian Distribution variance Standard deviation.
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
La Parguera Hyperspectral Image size (250x239x118) using Hyperion sensor. INTEREST POINTS FOR HYPERSPECTRAL IMAGES Amit Mukherjee 1, Badrinath Roysam 1,
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
Fitting: The Hough transform
Lecture 6: Feature matching CS4670: Computer Vision Noah Snavely.
A Study of Approaches for Object Recognition
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
Chapter 4 Digital Multimedia, 2nd edition Vector Graphics.
Lecture 4: Feature matching
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Object Recognition Using Distinctive Image Feature From Scale-Invariant Key point D. Lowe, IJCV 2004 Presenting – Anat Kaspi.
Scale Invariant Feature Transform (SIFT)
Stockman MSU Fall Computing Motion from Images Chapter 9 of S&S plus otherwork.
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detection.
Lecture 6: Feature matching and alignment CS4670: Computer Vision Noah Snavely.
Scale-Invariant Feature Transform (SIFT) Jinxiang Chai.
Stereo vision A brief introduction Máté István MSc Informatics.
Manifold learning: Locally Linear Embedding Jieping Ye Department of Computer Science and Engineering Arizona State University
Computer vision.
05 - Feature Detection Overview Feature Detection –Intensity Extrema –Blob Detection –Corner Detection Feature Descriptors Feature Matching Conclusion.
Object Tracking/Recognition using Invariant Local Features Applications l Mobile robots, driver assistance l Cell phone location or object recognition.
Top-points in Image Matching Bram Platel Evgenya Balmashnova Luc Florack Bart ter Haar Romeny.
Local invariant features Cordelia Schmid INRIA, Grenoble.
Reporter: Fei-Fei Chen. Wide-baseline matching Object recognition Texture recognition Scene classification Robot wandering Motion tracking.
CVPR 2003 Tutorial Recognition and Matching Based on Local Invariant Features David Lowe Computer Science Department University of British Columbia.
CSCE 643 Computer Vision: Extractions of Image Features Jinxiang Chai.
CS654: Digital Image Analysis Lecture 25: Hough Transform Slide credits: Guillermo Sapiro, Mubarak Shah, Derek Hoiem.
Wenqi Zhu 3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Local invariant features Cordelia Schmid INRIA, Grenoble.
Extracting features from spatio-temporal volumes (STVs) for activity recognition Dheeraj Singaraju Reading group: 06/29/06.
Edge Detection and Geometric Primitive Extraction Jinxiang Chai.
Kylie Gorman WEEK 1-2 REVIEW. CONVERTING AN IMAGE FROM RGB TO HSV AND DISPLAY CHANNELS.
Content-Based Image Retrieval (CBIR) By: Victor Makarenkov Michael Marcovich Noam Shemesh.
Methods for 3D Shape Matching and Retrieval
3D Reconstruction Using Image Sequence
Machine Vision Edge Detection Techniques ENT 273 Lecture 6 Hema C.R.
What is Digital Image processing?. An image can be defined as a two-dimensional function, f(x,y) # x and y are spatial (plane) coordinates # The function.
Introduction to Scale Space and Deep Structure. Importance of Scale Painting by Dali Objects exist at certain ranges of scale. It is not known a priory.
Digital Image Processing CSC331
Sparse RecoveryAlgorithmResults  Original signal x = x k + u, where x k has k large coefficients and u is noise.  Acquire measurements Ax = y. If |x|=n,
Image Representation and Description – Representation Schemes
SIFT Scale-Invariant Feature Transform David Lowe
CS262: Computer Vision Lect 09: SIFT Descriptors
Miguel Tavares Coimbra
Fitting: The Hough transform
Feature description and matching
Features Readings All is Vanity, by C. Allan Gilbert,
CAP 5415 Computer Vision Fall 2012 Dr. Mubarak Shah Lecture-5
Aim of the project Take your image Submit it to the search engine
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
ECE734 Project-Scale Invariant Feature Transform Algorithm
Lecture 5: Feature invariance
Lecture 5: Feature invariance
Detecting and analysing motion
Presentation transcript:

Bart M. ter Haar Romeny

 Question: can top-points be used for object- retrieval tasks?

 The locations of particularly characteristic points are called the interest points or key points.  These interest points have to be as invariant as possible, but at the same time they have to carry a lot of distinctive information.

 Information in interest points is defined by their neighborhood. But how big should we choose this neighborhood? Let’s take the corners of the mouth as interest points. The red circles are the areas in which the information is gathered. If we make the picture bigger, the size of the neighborhood is too small. The neighborhood should scale with the image

 When the interest points are detected in scale space they do not only have spatial coordinates x and y, but also a scale .  This scale tells us how big the neighborhood should be.  

 Our interest points have to be detected in scale space.  They also have to… ◦ …contain a lot of information ◦ …be reproducible ◦ …be stable ◦ …be well understood

Critical Points, Paths and Top-Points Maxima Minimum Saddles  L=0 Critical Points

Maxima Minimum Saddles  L=0 Critical Points Det(H)=0 Top-Points Critical Points, Paths and Top-Points

OriginalGradient Magnitude LaplacianDet(H)

 Since for a critical path  L=0  Intersection of Level Surfaces L x =0 with L y =0  Will give the critical paths.

 Since for a top-point both  L=0 and Det[H]=Lxx Lyy-Lxy 2 =0  We can find them by intersecting the paths with the level surface Det[H]=0

 Top-points are invariant to certain transformations.  By invariant we mean that they move according to the transformation. Allowed Trans.

 It is possible to make a reconstruction of the original image from its top-points.  We can generate reconstructed images which give the same (plus more) top-points as the original image.  This reconstruction resembles the original image.

Original Image Top-Points and Features Reconstruction

Original By adjusting boundary and smoothness constraints we can improve the visual performance. For this 300x300 picture 1000 top-points with 6 features were used.

 For points close to top-points it is possible to calculate a vector pointing towards the position of the top-point. x y  Approximated Top- Points Displacement Vectors Real Locations

 For points close to top-points it is possible to calculate a vector pointing towards the position of the top-point.  This enables us to use fast top-point detection algorithms which do not have to be very accurate.

 The locations of top-points change when noise is added to the image.

 We can calculate the variance of the displacement of top- points under noise.  We need 4 th order derivatives in the top- points for that.

Thresholding on Stability Stable Paths Unstable Paths

Databas e Query Image

 A simple image retrieval task.  Using a small version of the Olivetti Faces Database.  Consisting of 200 images of 20 different people (10 p.p.)

Look at all scales simultaneously Scale x y

Critical Points, Paths and Top Points Maxima Minimum Saddles  u=0 Critical Points Det(H)=0 Top Points

CompareEMD

Earth Movers Distance (EMD) wiwi f ij c ij AB ujuj [*] Rubner, Tomasi, Guibas, 1998, IEEE Conf. on Computer Vision PilesHoles

a93%78%68%62%58%54%49%45%43% b93%82%90%73%70%68%63%59%56% c95%88%83%76%73%66%62%59%58% d100%97%96%95%92%88%86%84%81% a.Using Euclidean Distance b.Using Eberly Distance c.As b. including stability norm d.As c. including 2 nd order derivatives.

 To distinguish top-points from each other a set of distinctive features are needed in every top-point.  These local features describe the neighborhood of the top-point.

 We use the complete set of irreducible 3 rd order differential invariants.  These features are rotation and scaling invariant.

 The top-points and differential invariants are calculated for the query object and the scene.

 We now compare the differential invariant features. compare distance = 0.5distance = 0.2distance = 0.3

The vectors with the smallest distance are paired. smallest distance distance = 0.2

 A set of coordinates is formed from the differences in scale (Log(  o1 )- Log(  s2 )) and in angles (  o1 -  s2 ). (  1,  1 )

 Dq Important Clusters For these clusters we calculate the mean  and   If these coordinates are plotted in a scatter plot clusters can be identified. In this scatter plot we find two dense clusters

 The stability criterion removes much of the scatter

Rotate and scale according to the cluster means.

The translations we find correspond to the location of the objects in the scene.

In this example we have two clusters of correctly matched points. C1 C2

We can transform the outline of the query object and project it on the scene image.

 Top-points have proved to be invariant interest points which are useful for matching.  The differential invariants have shown to be very distinctive.  Experiments show good results.