Special Topic on Image Retrieval Local Feature Matching Verification.

Slides:



Advertisements
Similar presentations
Recognising Panoramas M. Brown and D. Lowe, University of British Columbia.
Advertisements

Object Recognition using Local Descriptors Javier Ruiz-del-Solar, and Patricio Loncomilla Center for Web Research Universidad de Chile.
Object Recognition from Local Scale-Invariant Features David G. Lowe Presented by Ashley L. Kapron.
Aggregating local image descriptors into compact codes
Presented by Xinyu Chang
Summary of Friday A homography transforms one 3d plane to another 3d plane, under perspective projections. Those planes can be camera imaging planes or.
Instance-level recognition II.
Feature Matching and Robust Fitting Computer Vision CS 143, Brown James Hays Acknowledgment: Many slides from Derek Hoiem and Grauman&Leibe 2008 AAAI Tutorial.
Query Specific Fusion for Image Retrieval
3D Shape Histograms for Similarity Search and Classification in Spatial Databases. Mihael Ankerst,Gabi Kastenmuller, Hans-Peter-Kriegel,Thomas Seidl Univ.
Object Recognition using Invariant Local Features Applications l Mobile robots, driver assistance l Cell phone location or object recognition l Panoramas,
Global spatial layout: spatial pyramid matching Spatial weighting the features Beyond bags of features: Adding spatial information.
Multi-model Estimation with J-linkage Jeongkyun Lee.
Image alignment Image from
776 Computer Vision Jared Heinly Spring 2014 (slides borrowed from Jan-Michael Frahm, Svetlana Lazebnik, and others)
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
CVPR 2008 James Philbin Ondˇrej Chum Michael Isard Josef Sivic
Packing bag-of-features ICCV 2009 Herv´e J´egou Matthijs Douze Cordelia Schmid INRIA.
Bundling Features for Large Scale Partial-Duplicate Web Image Search Zhong Wu ∗, Qifa Ke, Michael Isard, and Jian Sun CVPR 2009.
Bag of Features Approach: recent work, using geometric information.
Robust and large-scale alignment Image from
WISE: Large Scale Content-Based Web Image Search Michael Isard Joint with: Qifa Ke, Jian Sun, Zhong Wu Microsoft Research Silicon Valley 1.
Object retrieval with large vocabularies and fast spatial matching
Object Recognition with Invariant Features n Definition: Identify objects or scenes and determine their pose and model parameters n Applications l Industrial.
Recognising Panoramas
Fitting. We’ve learned how to detect edges, corners, blobs. Now what? We would like to form a higher-level, more compact representation of the features.
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman ICCV 2003 Presented by: Indriyati Atmosukarto.
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman University of Oxford ICCV 2003.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
1Jana Kosecka, CS 223b EM and RANSAC EM and RANSAC.
Fitting.
Feature Matching and RANSAC : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and Rick Szeliski.
Keypoint-based Recognition and Object Search
כמה מהתעשייה? מבנה הקורס השתנה Computer vision.
Image Stitching Ali Farhadi CSE 455
Yuping Lin and Gérard Medioni.  Introduction  Method  Register UAV streams to a global reference image ▪ Consecutive UAV image registration ▪ UAV to.
CSE 185 Introduction to Computer Vision
Image Based Positioning System Ankit Gupta Rahul Garg Ryan Kaminsky.
Keypoint-based Recognition Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 03/04/10.
Advanced Computer Vision Feature-based Alignment Lecturer: Lu Yi & Prof. Fuh CSIE NTU.
1 Interest Operators Harris Corner Detector: the first and most basic interest operator Kadir Entropy Detector and its use in object recognition SIFT interest.
COMMON EVALUATION FINAL PROJECT Vira Oleksyuk ECE 8110: Introduction to machine Learning and Pattern Recognition.
A Statistical Approach to Speed Up Ranking/Re-Ranking Hong-Ming Chen Advisor: Professor Shih-Fu Chang.
1 Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval Ondrej Chum, James Philbin, Josef Sivic, Michael Isard and.
Example: line fitting. n=2 Model fitting Measure distances.
Video Google: A Text Retrieval Approach to Object Matching in Videos Josef Sivic and Andrew Zisserman.
Model Fitting Computer Vision CS 143, Brown James Hays 10/03/11 Slides from Silvio Savarese, Svetlana Lazebnik, and Derek Hoiem.
Fitting image transformations Prof. Noah Snavely CS1114
CSE 185 Introduction to Computer Vision Feature Matching.
Bundling Features for Large Scale Partial-Duplicate Web Image Search Zhong Wu ∗, Qifa Ke, Michael Isard, and Jian Sun Microsoft Research.
776 Computer Vision Jan-Michael Frahm Spring 2012.
Robust Estimation Course web page: vision.cis.udel.edu/~cv April 23, 2003  Lecture 25.
Video Google: Text Retrieval Approach to Object Matching in Videos Authors: Josef Sivic and Andrew Zisserman University of Oxford ICCV 2003.
Fitting.
COS 429 PS3: Stitching a Panorama Due November 10 th.
EE 7730 Parametric Motion Estimation. Bahadir K. Gunturk2 Parametric (Global) Motion Affine Flow.
Computer Photography -Scene Fixed 陳立奇.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
776 Computer Vision Jan-Michael Frahm Spring 2012.
Capturing, Processing and Experiencing Indian Monuments
Video Google: Text Retrieval Approach to Object Matching in Videos
Mixtures of Gaussians and Advanced Feature Encoding
Image Stitching Slides from Rick Szeliski, Steve Seitz, Derek Hoiem, Ira Kemelmacher, Ali Farhadi.
Feature Matching and RANSAC
Noah Snavely.
SIFT.
CSE 185 Introduction to Computer Vision
Computational Photography
Image Stitching Linda Shapiro ECE/CSE 576.
Image Stitching Linda Shapiro ECE P 596.
Presentation transcript:

Special Topic on Image Retrieval Local Feature Matching Verification

Geometric Verification Motivation –Remove false matches by checking geometric consistency 2 Red line: geometric consistent match Blue line: geometric inconsistent match

Global Verification: RANSAC Take RANSAC as an example – Check geometric consistency from matched feature pairs. Random sampling

Local Geometric-Verification Locally nearest neighbors ( Video Goole, cvpr’03 ) – Matched regions should have a similar spatial layout. – For each match define its search area – Region in the search area that also matches casts a vote for the image – Reject matches with no support Drawback – Sensitive to clutter

Hamming Embedding (ECCV’08) Introduced as an extension of BOV [Jegou 08] – Combination of – A partitioning technique (k-means) – A binary code that refine the descriptor Representation of a descriptor x – Vector-quantized to q(x) as in standard BOV – short binary vector b(x) for an additional localization in the Voronoi cell Two descriptors x and y match iif

Hamming Embedding Binary signature generation – Off-line learning Random matrix generation Descriptor projection and assignment Median values of projected descriptors – On-line binarization Quantization assignment Descriptor projection Computing the signature:

Local Geometric-Verification Bundled feature (CVPR’09) – Group local features in local MSER region. – Increase discriminative power of visual words. – Allowed to have large overlap error. Bundle comparison: – M m (q; p): number of common visual words between two bundles – M g (q; p): inconsistency of geometric order in x- and y- direction. Drawbacks: Infeasible for rotated bundles.

– Visual words are bundled in MSER regions. – Spatial consistency for bundled features is utilized to weight visual words. Z. Wu, J. Sun, and Q. Ke, “Bundling Features for Large Scale Partial-Duplicate Web Image Search,” CVPR 09 # of shared visual words Spatial consistency – Great performance for partial-dup detection in over 1 M database – Drawbacks: Infeasible for rotated bundles. Local Geometric-Verification Bundled feature (CVPR’09)

Global Verification: RANSAC  RANSAC: remove outliers by inlier classification  Inliers: true matched features  Outliers: false matched features  Assumption of RANdom SAmple Consensus (RANSAC)  The original data consists of inliers and outliers.  A subset of inliers can estimate a model to optimally explain the inliers.  Estimate the affine transformation by RANSAC  Procedure: Iteratively select a random subset as hypothetical inliers 1.A model is fitted to the hypothetical inliers. 2.All other data are tested against the fitted model for inlier classification. 3.The model is re-estimated from all hypothetical inliers. 4.The model is evaluated by estimating the error of the inliers relative to the model.  Drawbacks: Computationally expensive, not scalable Fischler, et al., RANdom SAmple Consensus: a paradigm for model fitting with applications to image analysis and automated cartography, Comm. of the ACM, 24: , 1981

Spatial Coding for Geometric Verification (ACM MM’ 10) Motivation – Encode local features’ relative positions into compact binary maps – Check spatial consistency of local matches for geometric verification Spatial coding maps – Relative spatial positions between local features. – Very efficient and high precision Zhou & Tian, Spatial Coding for large scale partial-duplicate image search. ACM Multimedia 2010.

Spatial Map Generation Rotate 45 degree counterclockwise  In previous case, each quadrant has one part  Consider each quadrant is uniformly divided into two parts. 11 =

Spatial Map Generation  Generalized spatial map: GX and GY  Each quadrant is uniformly divided into r parts. 12 … … k=r-1 k=1 k=0 X-map Y-map

Generalized Spatial Coding  Spatial coding maps:  Each quadrant uniformly divided into r parts.  Decompose the division into r sub-division.  Rotate each sub-division to align the axis. New feature locations after rotation : Generalized spatial maps : 13

Spatial Verification  Verification with spatial maps GX and GY  Compare the spatial maps of matched features:  k=0, …, r-1; i, j=1, …, N; N: number of matched features  Find and delete the most inconsistent matched pair, recursively: 14 V x : inconsistent degree in X-map V y : inconsistent degree in Y-map Identify i* and remove

x y

Geometric Verification with Coding Maps 16 SUM x y

Image Plane Division (TOMCCAP’ 10) (a) (d) (c) (b) (e) (f)

Geometric Square Coding Coordinate adjustment Square coding map Generalized map :

Geometric Fan Coding Fan coding maps Coordinate adjustment Generalized coding maps 19

Geometric Verification Compare the fan coding maps of matched features: Inconsistency measurement from geometric fan coding: Inconsistency measurement from geometric square coding: Inconsistency matrix: