Features CSE 455, Winter 2010 February 1, 2010. Neel Joshi, CSE 455, Winter 2010 2 Review From Last Time.

Slides:



Advertisements
Similar presentations
Feature extraction: Corners
Advertisements

Feature matching “What stuff in the left image matches with stuff on the right?” Necessary for automatic panorama stitching (Part of Project 4!) Slides.
Last 4 lectures Camera Structure HDR Image Filtering Image Transform.
CSE 473/573 Computer Vision and Image Processing (CVIP)
Interest points CSE P 576 Ali Farhadi Many slides from Steve Seitz, Larry Zitnick.
TP14 - Local features: detection and description Computer Vision, FCUP, 2014 Miguel Coimbra Slides by Prof. Kristen Grauman.
Algorithms and Applications in Computer Vision
Interest points CSE P 576 Larry Zitnick Many slides courtesy of Steve Seitz.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Affine Linear Transformations Prof. Graeme Bailey (notes modified from Noah Snavely, Spring 2009)
Lecture 4: Feature matching
Announcements Project 2 questions Midterm out on Thursday
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2005 with a lot of slides stolen from Steve Seitz and.
Feature extraction: Corners and blobs
Panorama signups Panorama project issues? Nalwa handout Announcements.
Lecture 16: Single-view modeling, Part 2 CS6670: Computer Vision Noah Snavely.
Lecture 3a: Feature detection and matching CS6670: Computer Vision Noah Snavely.
Panoramas and Calibration : Rendering and Image Processing Alexei Efros …with a lot of slides stolen from Steve Seitz and Rick Szeliski.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
CSE 576, Spring 2008Projective Geometry1 Lecture slides by Steve Seitz (mostly) Lecture presented by Rick Szeliski.
CS4670: Computer Vision Kavita Bala Lecture 7: Harris Corner Detection.
CS4670: Computer Vision Kavita Bala Lecture 8: Scale invariance.
Lecture 6: Feature matching and alignment CS4670: Computer Vision Noah Snavely.
Today: Calibration What are the camera parameters?
Summary of Previous Lecture A homography transforms one 3d plane to another 3d plane, under perspective projections. Those planes can be camera imaging.
Projective Geometry and Single View Modeling CSE 455, Winter 2010 January 29, 2010 Ames Room.
Single-view modeling, Part 2 CS4670/5670: Computer Vision Noah Snavely.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Last Lecture (optical center) origin principal point P (X,Y,Z) p (x,y) x y.
Lecture 06 06/12/2011 Shai Avidan הבהרה: החומר המחייב הוא החומר הנלמד בכיתה ולא זה המופיע / לא מופיע במצגת.
Image Segmentation And Representation Cosimo Distante.
Feature Detection and Matching
CSE 185 Introduction to Computer Vision Local Invariant Features.
Project 2 –contact your partner to coordinate ASAP –more work than the last project (> 1 week) –you should have signed up for panorama kitssigned up –sign.
Feature extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
776 Computer Vision Jan-Michael Frahm, Enrique Dunn Spring 2013.
Lecture 7: Features Part 2 CS4670/5670: Computer Vision Noah Snavely.
Feature extraction: Corners and blobs. Why extract features? Motivation: panorama stitching We have two images – how do we combine them?
Lecture 5: Image Interpolation and Features
Local features: detection and description
Features Jan-Michael Frahm.
Project 1 Due NOW Project 2 out today –Help session at end of class Announcements.
Image matching by Diva SianDiva Sian by swashfordswashford TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA.
Announcements Project 1 Out today Help session at the end of class.
Projects Project 1a due this Friday Project 1b will go out on Friday to be done in pairs start looking for a partner now.
Project 2 went out on Tuesday You should have a partner You should have signed up for a panorama kit Announcements.
CSE 185 Introduction to Computer Vision Local Invariant Features.
MASKS © 2004 Invitation to 3D vision Lecture 3 Image Primitives andCorrespondence.
Lecture 10: Harris Corner Detector CS4670/5670: Computer Vision Kavita Bala.
Keypoint extraction: Corners 9300 Harris Corners Pkwy, Charlotte, NC.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
Lecture 5: Feature detection and matching CS4670 / 5670: Computer Vision Noah Snavely.
Recognizing objects Prof. Noah Snavely CS1114
Interest Points EE/CSE 576 Linda Shapiro.
Image Features: Points and Regions
TP12 - Local features: detection and description
Lecture 4: Harris corner detection
Local features: detection and description May 11th, 2017
Feature description and matching
Features Readings All is Vanity, by C. Allan Gilbert,
Announcements Project 1 Project 2 (panoramas)
Lecture 5: Feature detection and matching
Announcement Project 1 1.
Projective geometry Readings
Edge detection f(x,y) viewed as a smooth function
Lecture VI: Corner and Blob Detection
Announcements Project 2 extension: Friday, Feb 8
Corner Detection COMP 4900C Winter 2008.
Presentation transcript:

Features CSE 455, Winter 2010 February 1, 2010

Neel Joshi, CSE 455, Winter Review From Last Time

Neel Joshi, CSE 455, Winter 2010 Measuring height Camera height

Neel Joshi, CSE 455, Winter The cross ratio  A Projective Invariant  Something that does not change under projective transformations (including perspective projection) P1P1 P2P2 P3P3 P4P4 The cross-ratio of 4 collinear points Can permute the point ordering 4! = 24 different orders (but only 6 distinct values) This is the fundamental invariant of projective geometry

Neel Joshi, CSE 455, Winter Monocular Depth Cues  Stationary Cues:  Perspective  Relative size  Familiar size  Aerial perspective  Occlusion  Peripheral vision  Texture gradient

Neel Joshi, CSE 455, Winter Camera calibration  Goal: estimate the camera parameters  Version 1: solve for projection matrix Version 2: solve for camera parameters separately –intrinsics (focal length, principle point, pixel size) –extrinsics (rotation angles, translation) –radial distortion

Neel Joshi, CSE 455, Winter Calibration using a reference object  Place a known object in the scene  identify correspondence between image and scene  compute mapping from scene to image Issues must know geometry very accurately must know 3D->2D correspondence

Neel Joshi, CSE 455, Winter 2010 Alternative: multi-plane calibration Images courtesy Jean-Yves Bouguet, Intel Corp. Advantage Only requires a plane Don’t have to know positions/orientations Good code available online! –Intel’s OpenCV library: –Matlab version by Jean-Yves Bouget: –Zhengyou Zhang’s web site:

Neel Joshi, CSE 455, Winter Today  Features  What are Features?  Edges  Corners  Generally “interesting” parts of an image All is Vanity, by C. Allan Gilbert, Readings M. Brown et al. Multi-Image Matching using Multi-Scale Oriented Patches, CVPR 2005Multi-Image Matching using Multi-Scale Oriented Patches

Neel Joshi, CSE 455, Winter Edge detection  Convert a 2D image into a set of curves  Extracts salient features of the scene  More compact than pixels

Neel Joshi, CSE 455, Winter Image gradient  The gradient of an image:  The gradient direction is given by: how does this relate to the direction of the edge? The edge strength is given by the gradient magnitude The gradient points in the direction of most rapid increase in intensity

Neel Joshi, CSE 455, Winter The Canny edge detector  original image (Lena)

Neel Joshi, CSE 455, Winter The Canny edge detector norm of the gradient

Neel Joshi, CSE 455, Winter The Canny edge detector thresholding

Neel Joshi, CSE 455, Winter The Canny edge detector thinning (non-maximum suppression)

Neel Joshi, CSE 455, Winter Image matching by Diva SianDiva Sian by swashfordswashford TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAAAAA

Neel Joshi, CSE 455, Winter Harder case by Diva SianDiva Sianby scgbtscgbt

Neel Joshi, CSE 455, Winter Even harder case “How the Afghan Girl was Identified by Her Iris Patterns” Read the storystory

Neel Joshi, CSE 455, Winter 2010 Harder still? NASA Mars Rover images

Neel Joshi, CSE 455, Winter 2010 NASA Mars Rover images with SIFT feature matches Figure by Noah Snavely Answer below (look for tiny colored squares…)

Neel Joshi, CSE 455, Winter 2010 Can we do image matching with Edges?

Neel Joshi, CSE 455, Winter Image Matching

Neel Joshi, CSE 455, Winter Image Matching

Neel Joshi, CSE 455, Winter 2010 Invariant local features  Find features that are invariant to transformations  geometric invariance: translation, rotation, scale  photometric invariance: brightness, exposure, … Feature Descriptors

Neel Joshi, CSE 455, Winter Advantages of local features  Locality  features are local, so robust to occlusion and clutter  Distinctiveness:  can differentiate a large database of objects  Quantity  hundreds or thousands in a single image  Efficiency  real-time performance achievable  Generality  exploit different types of features in different situations

Neel Joshi, CSE 455, Winter More motivation…  Feature points are used for:  Image alignment (e.g., mosaics)  3D reconstruction  Motion tracking  Object recognition  Indexing and database retrieval  Robot navigation  … other

Neel Joshi, CSE 455, Winter What makes a good feature? Snoop demo

Neel Joshi, CSE 455, Winter Want uniqueness  Look for image regions that are unusual  Lead to unambiguous matches in other images  How to define “unusual”?

Neel Joshi, CSE 455, Winter Local measures of uniqueness  Suppose we only consider a small window of pixels  What defines whether a feature is a good or bad candidate? Slide adapted form Darya Frolova, Denis Simakov, Weizmann Institute.

Neel Joshi, CSE 455, Winter Feature detection “flat” region: no change in all directions “edge”: no change along the edge direction “corner”: significant change in all directions  Local measure of feature uniqueness  How does the window change when you shift it?  Shifting the window in any direction causes a big change Slide adapted form Darya Frolova, Denis Simakov, Weizmann Institute.

Neel Joshi, CSE 455, Winter Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences (SSD) this defines an SSD “error” of E(u,v): Feature detection: the math W

Neel Joshi, CSE 455, Winter Taylor Series expansion of I: If the motion (u,v) is small, then first order approx is good Plugging this into the formula on the previous slide… Small motion assumption

Neel Joshi, CSE 455, Winter Consider shifting the window W by (u,v) how do the pixels in W change? compare each pixel before and after by summing up the squared differences this defines an “error” of E(u,v): Feature detection: the math W

Neel Joshi, CSE 455, Winter Feature detection: the math This can be rewritten: For the example above You can move the center of the green window to anywhere on the blue unit circle Which directions will result in the largest and smallest E values? We can find these directions by looking at the eigenvectors of H

Neel Joshi, CSE 455, Winter Quick eigenvalue/eigenvector review  The eigenvectors of a matrix A are the vectors x that satisfy:  The scalar is the eigenvalue corresponding to x  The eigenvalues are found by solving:  In our case, A = H is a 2x2 matrix, so we have  The solution:  Once you know, you find x by solving

Neel Joshi, CSE 455, Winter Feature detection: the math This can be rewritten: Eigenvalues and eigenvectors of H Define shifts with the smallest and largest change (E value) x + = direction of largest increase in E. + = amount of increase in direction x + x - = direction of smallest increase in E. - = amount of increase in direction x + x - x +

Neel Joshi, CSE 455, Winter Feature detection: the math How are +, x +, -, and x + relevant for feature detection? What’s our feature scoring function?

Neel Joshi, CSE 455, Winter Feature detection: the math How are +, x +, -, and x + relevant for feature detection? What’s our feature scoring function? Want E(u,v) to be large for small shifts in all directions the minimum of E(u,v) should be large, over all unit vectors [u v] this minimum is given by the smaller eigenvalue ( - ) of H

Neel Joshi, CSE 455, Winter Feature detection summary Here’s what you do Compute the gradient at each point in the image Create the H matrix from the entries in the gradient Compute the eigenvalues. Find points with large response ( - > threshold) Choose those points where - is a local maximum as features

Neel Joshi, CSE 455, Winter Feature detection summary Here’s what you do Compute the gradient at each point in the image Create the H matrix from the entries in the gradient Compute the eigenvalues. Find points with large response ( - > threshold) Choose those points where - is a local maximum as features

Neel Joshi, CSE 455, Winter The Harris operator - is a variant of the “Harris operator” for feature detection The trace is the sum of the diagonals, i.e., trace(H) = h 11 + h 22 Very similar to - but less expensive (no square root) Called the “Harris Corner Detector” or “Harris Operator” Lots of other detectors, this is one of the most popular

Neel Joshi, CSE 455, Winter The Harris operator Harris operator

Neel Joshi, CSE 455, Winter Harris detector example

Neel Joshi, CSE 455, Winter f value (red high, blue low)

Neel Joshi, CSE 455, Winter Threshold (f > value)

Neel Joshi, CSE 455, Winter Find local maxima of f

Neel Joshi, CSE 455, Winter Harris features (in red)

Neel Joshi, CSE 455, Winter Infinite Images

Neel Joshi, CSE 455, Winter Image Analogies