Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely.

Slides:



Advertisements
Similar presentations
Distinctive Image Features from Scale-Invariant Keypoints
Advertisements

Feature extraction: Corners
Feature Detection. Description Localization More Points Robust to occlusion Works with less texture More Repeatable Robust detection Precise localization.
Feature matching “What stuff in the left image matches with stuff on the right?” Necessary for automatic panorama stitching (Part of Project 4!) Slides.
Presented by Xinyu Chang
CSE 473/573 Computer Vision and Image Processing (CVIP)
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
Instructor: Mircea Nicolescu Lecture 15 CS 485 / 685 Computer Vision.
Image alignment Image from
Instructor: Mircea Nicolescu Lecture 13 CS 485 / 685 Computer Vision.
IBBT – Ugent – Telin – IPI Dimitri Van Cauwelaert A study of the 2D - SIFT algorithm Dimitri Van Cauwelaert.
Fast High-Dimensional Feature Matching for Object Recognition David Lowe Computer Science Department University of British Columbia.
Robust and large-scale alignment Image from
Patch Descriptors CSE P 576 Larry Zitnick
CS4670: Computer Vision Kavita Bala Lecture 10: Feature Matching and Transforms.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Lecture 6: Feature matching CS4670: Computer Vision Noah Snavely.
Affine Linear Transformations Prof. Graeme Bailey (notes modified from Noah Snavely, Spring 2009)
Lecture 4: Feature matching
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Feature extraction: Corners and blobs
Lecture 11: Structure from motion CS6670: Computer Vision Noah Snavely.
Distinctive image features from scale-invariant keypoints. David G. Lowe, Int. Journal of Computer Vision, 60, 2 (2004), pp Presented by: Shalomi.
Feature-based object recognition Prof. Noah Snavely CS1114
Detecting Patterns So far Specific patterns (eyes) Generally useful patterns (edges) Also (new) “Interesting” distinctive patterns ( No specific pattern:
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
Lecture 6: Feature matching and alignment CS4670: Computer Vision Noah Snavely.
Overview Introduction to local features
Interest Point Descriptors
Distinctive Image Features from Scale-Invariant Keypoints By David G. Lowe, University of British Columbia Presented by: Tim Havinga, Joël van Neerbos.
Computer Vision Lecture 4 FEATURE DETECTION. Feature Detection Feature Description Feature Matching Today’s Topics 2.
4.1 Feature: Point and Patches Xuejin Chen Reading: Szeliski Chap ~238.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Lecture 8: Feature matching CS6670: Computer Vision Noah Snavely.
Feature extraction: Corners and blobs. Why extract features? Motivation: panorama stitching We have two images – how do we combine them?
Fitting image transformations Prof. Noah Snavely CS1114
Project 3 questions? Interest Points and Instance Recognition Computer Vision CS 143, Brown James Hays 10/21/11 Many slides from Kristen Grauman and.
Local features: detection and description
CS654: Digital Image Analysis
776 Computer Vision Jan-Michael Frahm Spring 2012.
Image matching by Diva SianDiva Sian by swashfordswashford TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA.
Announcements Project 1 Out today Help session at the end of class.
Recognizing specific objects Matching with SIFT Original suggestion Lowe, 1999,2004.
Lecture 14: Feature matching and Transforms CS4670/5670: Computer Vision Kavita Bala.
Blob detection.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
Lecture 5: Feature detection and matching CS4670 / 5670: Computer Vision Noah Snavely.
Recognizing objects Prof. Noah Snavely CS1114
776 Computer Vision Jan-Michael Frahm Spring 2012.
Lecture 13: Feature Descriptors and Matching
Lecture 07 13/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Image Features: Points and Regions
TP12 - Local features: detection and description
Lecture 4: Harris corner detection
Local features: detection and description May 11th, 2017
Feature description and matching
Features Readings All is Vanity, by C. Allan Gilbert,
Announcements Project 1 Project 2 (panoramas)
Aim of the project Take your image Submit it to the search engine
The SIFT (Scale Invariant Feature Transform) Detector and Descriptor
Lecture 5: Feature detection and matching
Announcement Project 1 1.
SIFT keypoint detection
Lecture VI: Corner and Blob Detection
Feature descriptors and matching
Corner Detection COMP 4900C Winter 2008.
Lecture 6: Feature matching
Presentation transcript:

Lecture 4: Feature matching CS4670 / 5670: Computer Vision Noah Snavely

Announcements Project 1 artifact due tonight at 11:59pm (CMS) Quiz on Friday

Basic idea: Take 16x16 square window around detected feature Compute edge orientation (angle of the gradient - 90  ) for each pixel Throw out weak edges (threshold gradient magnitude) Create histogram of surviving edge orientations Scale Invariant Feature Transform Adapted from slide by David Lowe 0 22 angle histogram

SIFT descriptor Full version Divide the 16x16 window into a 4x4 grid of cells (2x2 case shown below) Compute an orientation histogram for each cell 16 cells * 8 orientations = 128 dimensional descriptor Adapted from slide by David Lowe

Properties of SIFT Extraordinarily robust matching technique – Can handle changes in viewpoint Up to about 60 degree out of plane rotation – Can handle significant changes in illumination Sometimes even day vs. night (below) – Fast and efficient—can run in real time – Lots of code available

SIFT Example sift 868 SIFT features

Feature matching Given a feature in I 1, how to find the best match in I 2 ? 1.Define distance function that compares two descriptors 2.Test all the features in I 2, find the one with min distance

Feature distance How to define the difference between two features f 1, f 2 ? – Simple approach: L 2 distance, ||f 1 - f 2 || – can give good scores to ambiguous (incorrect) matches I1I1 I2I2 f1f1 f2f2

f1f1 f2f2 f2'f2' Feature distance How to define the difference between two features f 1, f 2 ? Better approach: ratio distance = ||f 1 - f 2 || / || f 1 - f 2 ’ || f 2 is best SSD match to f 1 in I 2 f 2 ’ is 2 nd best SSD match to f 1 in I 2 gives large values for ambiguous matches I1I1 I2I2

Feature matching example 51 matches

Feature matching example 58 matches

Evaluating the results How can we measure the performance of a feature matcher? feature distance

True/false positives The distance threshold affects performance – True positives = # of detected matches that are correct Suppose we want to maximize these—how to choose threshold? – False positives = # of detected matches that are incorrect Suppose we want to minimize these—how to choose threshold? false match true match feature distance How can we measure the performance of a feature matcher?

0.7 Evaluating the results false positive rate true positive rate # true positives # correctly matched features (positives) 0.1 How can we measure the performance of a feature matcher? “recall” # false positives # incorrectly matched features (negatives) 1 - “precision”

0.7 Evaluating the results false positive rate true positive rate 0.1 ROC curve (“Receiver Operator Characteristic”) How can we measure the performance of a feature matcher? # true positives # correctly matched features (positives) “recall” # false positives # incorrectly matched features (negatives) 1 - “precision”

Lots of applications Features are used for: – Image alignment (e.g., mosaics) – 3D reconstruction – Motion tracking – Object recognition (e.g., Google Goggles) – Indexing and database retrieval – Robot navigation – … other

Object recognition (David Lowe)

3D Reconstruction Internet Photos (“Colosseum”) Reconstructed 3D cameras and points

Sony Aibo SIFT usage: Recognize charging station Communicate with visual cards Teach object recognition

Questions?

Image alignment Why don’t these image line up exactly?