A Tensor-Based Algorithm for High-Order Graph Matching

Slides:

Advertisements

Similar presentations

Shape Matching and Object Recognition using Low Distortion Correspondence Alexander C. Berg, Tamara L. Berg, Jitendra Malik U.C. Berkeley.

Advertisements

Location Recognition Given: A query image A database of images with known locations Two types of approaches: Direct matching: directly match image features.

BiG-Align: Fast Bipartite Graph Alignment

Matrix Factorization with Unknown Noise

Three things everyone should know to improve object retrieval

Foreground Focus: Finding Meaningful Features in Unlabeled Images Yong Jae Lee and Kristen Grauman University of Texas at Austin.

Spectral graph reduction for image and streaming video segmentation Fabio Galasso 1 Margret Keuper 2 Thomas Brox 2 Bernt Schiele 1 1 Max Planck Institute.

Olivier Duchenne ， Armand Joulin ， Jean Ponce Willow Lab ， ICCV2011.

SuperMatching: Feature Matching using Supersymmetric Geometric Constraints Submission ID: 0208.

Unsupervised Detection of Regions of Interest Using Iterative Link Analysis Gunhee Kim 1 Antonio Torralba 2 1: SCS, CMU 2: CSAIL, MIT Neural Information.

Learning to Combine Bottom-Up and Top-Down Segmentation Anat Levin and Yair Weiss School of CS&Eng, The Hebrew University of Jerusalem, Israel.

Efficient And Accurate Ranking of Multidimensional Drug Profiling Data by Graph-Based Algorithm Dorit S. Hochbaum Chun-nan Hsu Yan T. Yang.

MIT CSAIL Vision interfaces Approximate Correspondences in High Dimensions Kristen Grauman* Trevor Darrell MIT CSAIL (*) UT Austin…

Fast Iterative Alignment of Pose Graphs with Poor Initial Estimates Edwin Olson John Leonard, Seth Teller

Shape From Light Field meets Robust PCA

Probabilistic Graph and Hypergraph Matching

Qualifying Exam: Contour Grouping Vida Movahedi Supervisor: James Elder Supervisory Committee: Minas Spetsakis, Jeff Edmonds York University Summer 2009.

CS4670 / 5670: Computer Vision Bag-of-words models Noah Snavely Object

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Biased Normalized Cuts 1 Subhransu Maji and Jithndra Malik University of California, Berkeley IEEE Conference on Computer Vision and Pattern Recognition.

Bag-of-features models Many slides adapted from Fei-Fei Li, Rob Fergus, and Antonio Torralba.

Computer Vision Group University of California Berkeley Shape Matching and Object Recognition using Shape Contexts Jitendra Malik U.C. Berkeley (joint.

Silhouettes in Multiview Stereo Ian Simon. Multiview Stereo Problem Input: – a collection of images of a rigid object (or scene) – camera parameters for.

One-Shot Multi-Set Non-rigid Feature-Spatial Matching

Distributed Message Passing for Large Scale Graphical Models Alexander Schwing Tamir Hazan Marc Pollefeys Raquel Urtasun CVPR2011.

Good morning, everyone, thank you for coming to my presentation.

Graph Based Semi- Supervised Learning Fei Wang Department of Statistical Science Cornell University.

Robust Higher Order Potentials For Enforcing Label Consistency

Lecture 28: Bag-of-words models

1 Unsupervised Modeling and Recognition of Object Categories with Combination of Visual Contents and Geometric Similarity Links Gunhee Kim Christos Faloutsos.

1 Unsupervised Modeling of Object Categories Using Link Analysis Techniques Gunhee Kim Christos Faloutsos Martial Hebert Gunhee Kim Christos Faloutsos.

CVR05 University of California Berkeley 1 Familiar Configuration Enables Figure/Ground Assignment in Natural Scenes Xiaofeng Ren, Charless Fowlkes, Jitendra.

CS294‐43: Visual Object and Activity Recognition Prof. Trevor Darrell Spring 2009 March 17 th, 2009.

Bag-of-features models

Triangulation and Multi-View Geometry Class 9 Read notes Section 3.3, , 5.1 (if interested, read Triggs’s paper on MVG using tensor notation, see.

Linear Solution to Scale and Rotation Invariant Object Matching Professor: 王聖智教授 Student : 周節.

A Tensor-Based Algorithm for High-Order Graph Matching Olivier Duchenne, Francis Bach, Inso Kweon, Jean Ponce École Normale Supérieure, INRIA, KAIST Team.

Large Scale Recognition and Retrieval. What does the world look like? High level image statistics Object Recognition for large-scale search Focus on scaling.

Matthew Brown University of British Columbia (prev.) Microsoft Research [ Collaborators: † Simon Winder, *Gang Hua, † Rick Szeliski † =MS Research, *=MS.

Review: Intro to recognition Recognition tasks Machine learning approach: training, testing, generalization Example classifiers Nearest neighbor Linear.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.

Unsupervised Learning of Categories from Sets of Partially Matching Image Features Kristen Grauman and Trevor Darrel CVPR 2006 Presented By Sovan Biswas.

Adaptive CSMA under the SINR Model: Fast convergence using the Bethe Approximation Krishna Jagannathan IIT Madras (Joint work with) Peruru Subrahmanya.

Bag-of-features models. Origin 1: Texture recognition Texture is characterized by the repetition of basic elements or textons For stochastic textures,

Handwritten digit recognition Jitendra Malik. Handwritten digit recognition (MNIST,USPS) LeCun’s Convolutional Neural Networks variations (0.8%, 0.6%

Graph-based Deformable Matching of 3D Line Segments with Application in Protein Fitting 12 1 HANG DOU 1, MATTHEW L BAKER 2, TAO JU Washington University.

Practical Message-passing Framework for Large-scale Combinatorial Optimization Inho Cho, Soya Park, Sejun Park, Dongsu Han, and Jinwoo Shin KAIST 2015.

CS654: Digital Image Analysis

Tightening LP Relaxations for MAP using Message-Passing David Sontag Joint work with Talya Meltzer, Amir Globerson, Tommi Jaakkola, and Yair Weiss.

IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit.

Lecture IX: Object Recognition (2)

Segmentation by clustering: normalized cut

Convolutional Neural Fabrics by Shreyas Saxena, Jakob Verbeek

Answering ‘Where am I?’ by Nonlinear Least Squares

Learning Mid-Level Features For Recognition

Object Matching Using a Locally Affine Invariant and Linear Programming Techniques - H. Li, X. Huang, L. He Ilchae Jung.

Paper Presentation: Shape and Matching

Digit Recognition using SVMS

By Suren Manvelyan, Crocodile (nile crocodile?) By Suren Manvelyan,

Graph matching algorithms

Shape matching and object recognition using shape contexts

Learning to Combine Bottom-Up and Top-Down Segmentation

Spectral Clustering Eric Xing Lecture 8, August 13, 2010

Segmentation (continued)

CSE 185 Introduction to Computer Vision

A Graph-Matching Kernel for Object Categorization

“Traditional” image segmentation

Presentation transcript:

A Tensor-Based Algorithm for High-Order Graph Matching Olivier Duchenne, Francis Bach, Inso Kweon, Jean Ponce École Normale Supérieure, INRIA, KAIST Team Willow

Extension of [Leordeanu & Hebert] to the case of hypergraph. Contributions Extension of [Leordeanu & Hebert] to the case of hypergraph. 1) Tensor Formulation and Power Method 2) Sparse output for the relaxed matching problem.

Generic Graph Matching Problem Previous Works Generic Graph Matching Problem

Generic Graph Matching Previous Works Generic Graph Matching NP-Hard combinatorial problem. Wide and active literature existing on approximation algorithms. Greedy Relaxed formulation Convex-concave optimization Many others… [1] M. Zaslavskiy, F. Bach and J.-P. Vert. PAMI, 2009. [2] D. C. Schmidt and L. E. Druffel. JACM, 1976. [3] J. R. Ullmann. JACM, 1976. [4] L. P. Cordella, P. Foggia, C. Sansone, and M. Vento. ICIAP, 1991. [5] Shinji Umeyama. PAMI, 1988. [6] S.Gold and A.Rangarajan. PAMI, 1996. [7] Hong Fang Wang and Edwin R. Hancock. PR, 2005. [8] Terry Caelli and Serhiy Kosinov. PAMI 2004. [9] H.A. Almohamad and S.O.Duffuaa. PAMI, 1993. [10] C.Schellewald and C.Schnor. LNCS, 2005.

Graph Matching for Computer Vision (1) Previous Works [1] M. Fishler and Elschlager. Computer, 1973. [2] Serge Belongie, Jitendra Malik and Jan Puzicha. NIPS 2000.

Graph Matching for Computer Vision (2) Previous Works [1] A. C. Berg, T. L. Berg, and J. Malik. CVPR 2005 [2] M. Leordeanu and M. Hebert. ICCV 2005 [3] T. Cour and J. Shi. NIPS 2006.

? = Why we need correspondance? Previous Works Why we need correspondance? Bag-Of-Word model works well when relationship between features is not important. ? = Graph Matching is an attempt to compare images when that relationship cannot be ignored. [1] Sivic & Zisserman. 2003 [2] Lazebnik et al. 2003 [3] Csurka et al. 2004

( ) ( ) Objective function Previous Works . m1 . m2 First order score Second order score [1] A. C. Berg, T. L. Berg, and J. Malik. CVPR 2005. [2] M. Leordeanu and M. Hebert. ICCV 2005. [3] T. Cour and J. Shi. NIPS 2006.

How graph matching enforces geometric consistency? Hypergraph Matching Problem How graph matching enforces geometric consistency? [1] A. C. Berg, T. L. Berg, and J. Malik. CVPR 2005.

How graph matching enforces geometric consistency? Hypergraph Matching Problem How graph matching enforces geometric consistency? [1] A. C. Berg, T. L. Berg, and J. Malik. CVPR 2005. [2] M. Leordeanu and M. Hebert. ICCV 2005.

Hypergraph Matching Problem How to improve it?

Hypergraph Matching Problem How to improve it?

Hypergraph Matching Hypergraph Matching Problem A hyper-edge can link more than 2 nodes at the same time. [1] Ron Zass and Amnon Shashua. CVPR, 2008

Hypergraph Matching Problem Formulation

Relaxation Main Eigenvector Problem Hypergraph Matching Problem Constraints: Relaxed constraints: Main Eigenvector Problem Each node is matched to at most one node. [1] M. Leordeanu and M. Hebert. ICCV 2005.

Hypergraph Matching Problem

Hypergraph Matching Problem Power Method The power method can efficiently find the main eigenvector of a matrix. It can efficiently use the sparsity of the Matrix. It is very simple to implement.

Tensor Power Iteration Hypergraph Matching Problem Tensor Power Iteration Converge to a local optimum (of the relaxed problem) Always keep the full degree of the hypergraph (never marginalize it in a frist or second order) [1] L. De Lathauwer, B. De Moor, and J. Vandewalle. SIAM J. Matrix Anal. Appl., 2000 [2] P. A. Regalia and E. Kofidis. ICASSP, 2000

Tensor Power Iteration Hypergraph Matching Problem Tensor Power Iteration It is also possible to integrate cues of different orders.

Sparse Output Sparse Output

Implementation Compute all descriptor triplets of image 2. Same for a subsample of triplets of image 1. Use Approximate Nearest Neighbor to find the closest triplets. Compute the sparse tensor. Do Tensor Power Iteration. Projection to binary solution.

Artificial cloud of point. Experiments Artificial cloud of point. We generate a cloud of random points. Add some noise to the position of points. Rotate it.

Accuracy depending on outliers number Experiments Accuracy depending on outliers number Our work Leordeanu & Hebert Zass & Shashua

Accuracy depending on scaling Experiments Accuracy depending on scaling Our work Leordeanu & Hebert Zass & Shashua

CMU hotel dataset Experiments

Error on Hotel data set Experiments Our work Zass & Shashua Leordeanu & Hebert

Examples of Caltech 256 silouhettes matching Experiments Examples of Caltech 256 silouhettes matching

Conclusion Method for Hyper-Graph matching. Tensor formulation Power Iteration Sparse output Future Work

Thank You !