Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint.

Slides:



Advertisements
Similar presentations
Shape Matching and Object Recognition using Low Distortion Correspondence Alexander C. Berg, Tamara L. Berg, Jitendra Malik U.C. Berkeley.
Advertisements

Shape Context and Chamfer Matching in Cluttered Scenes
Multiclass SVM and Applications in Object Classification
Handwritten digit recognition
Presented by Xinyu Chang
Classification using intersection kernel SVMs is efficient Joint work with Subhransu Maji and Alex Berg Jitendra Malik UC Berkeley.
MIT CSAIL Vision interfaces Approximate Correspondences in High Dimensions Kristen Grauman* Trevor Darrell MIT CSAIL (*) UT Austin…
Detecting Categories in News Video Using Image Features Slav Petrov, Arlo Faria, Pascal Michaillat, Alex Berg, Andreas Stolcke, Dan Klein, Jitendra Malik.
Image and video descriptors
Instructor: Mircea Nicolescu Lecture 17
Silhouette Lookup for Automatic Pose Tracking N ICK H OWE.
Image alignment Image from
Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA
Fast intersection kernel SVMs for Realtime Object Detection
Model: Parts and Structure. History of Idea Fischler & Elschlager 1973 Yuille ‘91 Brunelli & Poggio ‘93 Lades, v.d. Malsburg et al. ‘93 Cootes, Lanitis,
Computer Vision Group University of California Berkeley Estimating Human Body Configurations using Shape Context Matching Greg Mori and Jitendra Malik.
The Visual Recognition Machine Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.
One-Shot Multi-Set Non-rigid Feature-Spatial Matching
Robust and large-scale alignment Image from
Iterative closest point algorithms
Computer Vision Group University of California Berkeley Recognizing objects and actions in images and video Jitendra Malik U.C. Berkeley.
A Study of Approaches for Object Recognition
CS335 Principles of Multimedia Systems Content Based Media Retrieval Hao Jiang Computer Science Department Boston College Dec. 4, 2007.
Recognizing Objects and Actions in Images Jitendra Malik U.C. Berkeley.
CVR05 University of California Berkeley 1 Familiar Configuration Enables Figure/Ground Assignment in Natural Scenes Xiaofeng Ren, Charless Fowlkes, Jitendra.
Computer Vision Group University of California Berkeley Visual Grouping and Object Recognition Jitendra Malik * U.C. Berkeley * with S. Belongie, C. Fowlkes,
Matching and Recognition in 3D. Moving from 2D to 3D Some things harderSome things harder – Rigid transform has 6 degrees of freedom vs. 3 – No natural.
Image Matching via Saliency Region Correspondences Alexander Toshev Jianbo Shi Kostas Daniilidis IEEE Conference on Computer Vision and Pattern Recognition.
Nearest Neighbour Condensing and Editing David Claus February 27, 2004 Computer Vision Reading Group Oxford.
Computer Vision Group University of California Berkeley Matching Shapes Serge Belongie *, Jitendra Malik and Jan Puzicha U.C. Berkeley * Present address:
1 Invariant Local Feature for Object Recognition Presented by Wyman 2/05/2006.
Computer Vision Group University of California Berkeley Recognizing Objects in Adversarial Clutter: Breaking a Visual CAPTCHA Greg Mori and Jitendra Malik.
Automatic Image Alignment (feature-based) : Computational Photography Alexei Efros, CMU, Fall 2006 with a lot of slides stolen from Steve Seitz and.
A New Correspondence Algorithm Jitendra Malik Computer Science Division University of California, Berkeley Joint work with Serge Belongie, Jan Puzicha,
An Integrated Pose and Correspondence Approach to Image Matching Anand Rangarajan Image Processing and Analysis Group Departments of Electrical Engineering.
PhD Thesis. Biometrics Science studying measurements and statistics of biological data Most relevant application: id. recognition 2.
Face Recognition Using Neural Networks Presented By: Hadis Mohseni Leila Taghavi Atefeh Mirsafian.
Overview Introduction to local features
3D Global Registration. The Problem Given: n scans around an objectGiven: n scans around an object Goal: align them allGoal: align them all First attempt:
Efficient Algorithms for Matching Pedro Felzenszwalb Trevor Darrell Yann LeCun Alex Berg.
CSE 185 Introduction to Computer Vision
Unsupervised Learning of Categories from Sets of Partially Matching Image Features Kristen Grauman and Trevor Darrel CVPR 2006 Presented By Sovan Biswas.
Final Exam Review CS485/685 Computer Vision Prof. Bebis.
2D Shape Matching (and Object Recognition)
Characterizing activity in video shots based on salient points Nicolas Moënne-Loccoz Viper group Computer vision & multimedia laboratory University of.
Handwritten digit recognition Jitendra Malik. Handwritten digit recognition (MNIST,USPS) LeCun’s Convolutional Neural Networks variations (0.8%, 0.6%
Andrew Bender Alexander Cobian
Shape Matching Tuesday, Nov 18 Kristen Grauman UT-Austin.
Visual Grouping and Recognition Jitendra Malik University of California at Berkeley Jitendra Malik University of California at Berkeley.
Classifying Images with Visual/Textual Cues By Steven Kappes and Yan Cao.
Image Registration as an Optimization Problem. Overlaying two or more images of the same scene Image Registration.
Recognizing Action at a Distance Alexei A. Efros, Alexander C. Berg, Greg Mori, Jitendra Malik Computer Science Division, UC Berkeley Presented by Pundik.
David Claus and Christoph F. Eick: Nearest Neighbor Editing and Condensing Techniques Nearest Neighbor Editing and Condensing Techniques 1.Nearest Neighbor.
Computer Vision Group University of California Berkeley On Visual Recognition Jitendra Malik UC Berkeley.
Handwritten digit recognition
Visual Categorization With Bags of Keypoints Original Authors: G. Csurka, C.R. Dance, L. Fan, J. Willamowski, C. Bray ECCV Workshop on Statistical Learning.
Methods for 3D Shape Matching and Retrieval
Distinctive Image Features from Scale-Invariant Keypoints
776 Computer Vision Jan-Michael Frahm Spring 2012.
Affine Registration in R m 5. The matching function allows to define tentative correspondences and a RANSAC-like algorithm can be used to estimate the.
Finding Clusters within a Class to Improve Classification Accuracy Literature Survey Yong Jae Lee 3/6/08.
Jo˜ao Carreira, Abhishek Kar, Shubham Tulsiani and Jitendra Malik University of California, Berkeley CVPR2015 Virtual View Networks for Object Reconstruction.
Invariant Local Features Image content is transformed into local feature coordinates that are invariant to translation, rotation, scale, and other imaging.
SIFT Scale-Invariant Feature Transform David Lowe
Paper Presentation: Shape and Matching
Digit Recognition using SVMS
Shape matching and object recognition using shape contexts
Aim of the project Take your image Submit it to the search engine
Brief Review of Recognition + Context
Shape Matching and Object Recognition
Presentation transcript:

Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint work with S. Belongie, J. Puzicha, G. Mori)

Computer Vision Group University of California Berkeley Outline Shape matching and isolated object recognition Scaling up to general object recognition

Computer Vision Group University of California Berkeley Biological Shape D’Arcy Thompson: On Growth and Form, 1917 –studied transformations between shapes of organisms

Computer Vision Group University of California Berkeley Deformable Templates: Related Work Fischler & Elschlager (1973) Grenander et al. (1991) Yuille (1991) von der Malsburg (1993)

Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

Computer Vision Group University of California Berkeley Comparing Pointsets

Computer Vision Group University of California Berkeley Shape Context Count the number of points inside each bin, e.g.: Count = 4 Count = FCompact representation of distribution of points relative to each point

Computer Vision Group University of California Berkeley Shape Context

Computer Vision Group University of California Berkeley Shape Contexts Invariant under translation and scale Can be made invariant to rotation by using local tangent orientation frame Tolerant to small affine distortion –Log-polar bins make spatial blur proportional to r Cf. Spin Images (Johnson & Hebert) - range image registration

Computer Vision Group University of California Berkeley Comparing Shape Contexts Compute matching costs using Chi Squared distance: Recover correspondences by solving linear assignment problem with costs C ij [Jonker & Volgenant 1987]

Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

Computer Vision Group University of California Berkeley 2D counterpart to cubic spline: Minimizes bending energy: Solve by inverting linear system Can be regularized when data is inexact Thin Plate Spline Model Duchon (1977), Meinguet (1979), Wahba (1991)

Computer Vision Group University of California Berkeley Matching Example modeltarget

Computer Vision Group University of California Berkeley Outlier Test Example

Computer Vision Group University of California Berkeley Synthetic Test Results Fish - deformation + noiseFish - deformation + outliers ICPShape ContextChui & Rangarajan

Computer Vision Group University of California Berkeley Matching Framework Find correspondences between points on shape Estimate transformation Measure similarity modeltarget...

Computer Vision Group University of California Berkeley Terms in Similarity Score Shape Context difference Local Image appearance difference –orientation –gray-level correlation in Gaussian window –… (many more possible) Bending energy

Computer Vision Group University of California Berkeley Object Recognition Experiments Kimia silhouette dataset Handwritten digits COIL 3D objects (Nayar-Murase) Human body configurations Trademarks

Computer Vision Group University of California Berkeley Shape Similarity: Kimia dataset

Computer Vision Group University of California Berkeley Quantitative Comparison rank Number correct

Computer Vision Group University of California Berkeley Handwritten Digit Recognition MNIST : –linear: 12.0% –40 PCA+ quad: 3.3% –1000 RBF +linear: 3.6% –K-NN: 5% –K-NN (deskewed) : 2.4% –K-NN (tangent dist.) : 1.1% –SVM: 1.1% –LeNet 5: 0.95% MNIST (distortions): –LeNet 5: 0.8% –SVM: 0.8% –Boosted LeNet 4: 0.7% MNIST : –K-NN, Shape Context matching: 0.63%

Computer Vision Group University of California Berkeley

Computer Vision Group University of California Berkeley Results: Digit Recognition 1-NN classifier using: Shape context * bending * image appearance

Computer Vision Group University of California Berkeley COIL Object Database

Computer Vision Group University of California Berkeley Error vs. Number of Views

Computer Vision Group University of California Berkeley Prototypes Selected for 2 Categories Details in Belongie, Malik & Puzicha (NIPS2000)

Computer Vision Group University of California Berkeley Editing: K-medoids Input: similarity matrix Select: K prototypes Minimize: mean distance to nearest prototype Algorithm: –iterative –split cluster with most errors Result: Adaptive distribution of resources (cfr. aspect graphs)

Computer Vision Group University of California Berkeley Error vs. Number of Views

Computer Vision Group University of California Berkeley Human body configurations

Computer Vision Group University of California Berkeley Automatically Locating Keypoints User marks keypoints on exemplars Find correspondence with test shape Transfer keypoint position from exemplar to the test shape.

Computer Vision Group University of California Berkeley Results

Computer Vision Group University of California Berkeley Trademark Similarity

Computer Vision Group University of California Berkeley Outline Shape matching and isolated object recognition Scaling up to general object recognition –Many objects (Mori, Belongie & Malik, CVPR 01) –Gray scale matching (Berg & Malik, CVPR 01) –Objects in scenes (scanning or segmentation)

Computer Vision Group University of California Berkeley Mori, Belongie, Malik (CVPR 01) Fast Pruning –Given a query shape, quickly return a shortlist of candidate matches –Database of known objects will be large: ~30000 Detailed Matching –Perform computationally expensive comparisons on only the few shapes in the shortlist

Computer Vision Group University of California Berkeley Representative Shape Contexts Match using only a few shape contexts –Don’t need to compare every one

Computer Vision Group University of California Berkeley Snodgrass Results

Computer Vision Group University of California Berkeley Results

Computer Vision Group University of California Berkeley Conclusion Introduced new matching algorithm matching based on shape contexts and TPS Robust to outliers & noise Forms basis of object recognition technique that performs well in a variety of domains using exactly the same algorithm