Fast and accurate energy minimization for static or time-varying Markov Random Fields (MRFs) Nikos Komodakis (Ecole Centrale Paris) Nikos Paragios (Ecole.

Slides:



Advertisements
Similar presentations
Iterative Rounding and Iterative Relaxation
Advertisements

The Primal-Dual Method: Steiner Forest TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A A AA A A A AA A A.
Primal-dual Algorithm for Convex Markov Random Fields Vladimir Kolmogorov University College London GDR (Optimisation Discrète, Graph Cuts et Analyse d'Images)
Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London Tutorial at GDR (Optimisation Discrète, Graph Cuts.
1 LP, extended maxflow, TRW OR: How to understand Vladimirs most recent work Ramin Zabih Cornell University.
1 LP Duality Lecture 13: Feb Min-Max Theorems In bipartite graph, Maximum matching = Minimum Vertex Cover In every graph, Maximum Flow = Minimum.
Active Learning for Streaming Networked Data Zhilin Yang, Jie Tang, Yutao Zhang Computer Science Department, Tsinghua University.
Tutorial at ICCV (Barcelona, Spain, November 2011)
Introduction to Algorithms
Introduction to Markov Random Fields and Graph Cuts Simon Prince
Primal-Dual Algorithms for Connected Facility Location Chaitanya SwamyAmit Kumar Cornell University.
ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.
Combinatorial Algorithms for Market Equilibria Vijay V. Vazirani.
Discrete optimization methods in computer vision Nikos Komodakis Ecole Centrale Paris ICCV 2007 tutorial Rio de Janeiro Brazil, October 2007.
Algorithmic Game Theory and Internet Computing Vijay V. Vazirani Georgia Tech Combinatorial Algorithms for Convex Programs (Capturing Market Equilibria.
Learning with Inference for Discrete Graphical Models Nikos Komodakis Pawan Kumar Nikos Paragios Ramin Zabih (presenter)
1 Fast Primal-Dual Strategies for MRF Optimization (Fast PD) Robot Perception Lab Taha Hamedani Aug 2014.
Approximation Algoirthms: Semidefinite Programming Lecture 19: Mar 22.
Instructor Neelima Gupta Table of Contents Lp –rounding Dual Fitting LP-Duality.
Linear Programming and Approximation
Totally Unimodular Matrices Lecture 11: Feb 23 Simplex Algorithm Elliposid Algorithm.
1 Introduction to Linear and Integer Programming Lecture 9: Feb 14.
Primal Dual Method Lecture 20: March 28 primaldual restricted primal restricted dual y z found x, succeed! Construct a better dual.
Network Design Adam Meyerson Carnegie-Mellon University.
Approximation Algorithms
Stereo & Iterative Graph-Cuts Alex Rav-Acha Vision Course Hebrew University.
Job Scheduling Lecture 19: March 19. Job Scheduling: Unrelated Multiple Machines There are n jobs, each job has: a processing time p(i,j) (the time to.
Stereo Computation using Iterative Graph-Cuts
Primal-Dual Algorithms for Connected Facility Location Chaitanya SwamyAmit Kumar Cornell University.
Computer vision: models, learning and inference
Approximation Algorithms: Bristol Summer School 2008 Seffi Naor Computer Science Dept. Technion Haifa, Israel TexPoint fonts used in EMF. Read the TexPoint.
V. V. Vazirani. Approximation Algorithms Chapters 3 & 22
CS774. Markov Random Field : Theory and Application Lecture 08 Kyomin Jung KAIST Sep
Michael Bleyer LVA Stereo Vision
Procedural Modeling of Architectures towards 3D Reconstruction Nikos Paragios Ecole Centrale Paris / INRIA Saclay Ile-de-France Joint Work: P. Koutsourakis,
CS774. Markov Random Field : Theory and Application Lecture 13 Kyomin Jung KAIST Oct
Planar Cycle Covering Graphs for inference in MRFS The Typhon Algorithm A New Variational Approach to Ground State Computation in Binary Planar Markov.
1 Introduction to Approximation Algorithms. 2 NP-completeness Do your best then.
Approximation Algorithms Department of Mathematics and Computer Science Drexel University.
Linear Programming Data Structures and Algorithms A.G. Malamos References: Algorithms, 2006, S. Dasgupta, C. H. Papadimitriou, and U. V. Vazirani Introduction.
Primal-Dual Algorithms for Connected Facility Location Chaitanya SwamyAmit Kumar Cornell University.
Approximation Algorithms
Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London.
1 Markov Random Fields with Efficient Approximations Yuri Boykov, Olga Veksler, Ramin Zabih Computer Science Department CORNELL UNIVERSITY.
CSE 589 Part VI. Reading Skiena, Sections 5.5 and 6.8 CLR, chapter 37.
Lecture 19: Solving the Correspondence Problem with Graph Cuts CAP 5415 Fall 2006.
Approximate Inference: Decomposition Methods with Applications to Computer Vision Kyomin Jung ( KAIST ) Joint work with Pushmeet Kohli (Microsoft Research)
Presenter : Kuang-Jui Hsu Date : 2011/3/24(Thur.).
2) Combinatorial Algorithms for Traditional Market Models Vijay V. Vazirani.
MRF optimization based on Linear Programming relaxations Nikos Komodakis University of Crete IPAM, February 2008.
Linear Program Set Cover. Given a universe U of n elements, a collection of subsets of U, S = {S 1,…, S k }, and a cost function c: S → Q +. Find a minimum.
Lecture.6. Table of Contents Lp –rounding Dual Fitting LP-Duality.
1 Scale and Rotation Invariant Matching Using Linearly Augmented Tree Hao Jiang Boston College Tai-peng Tian, Stan Sclaroff Boston University.
A global approach Finding correspondence between a pair of epipolar lines for all pixels simultaneously Local method: no guarantee we will have one to.
Tightening LP Relaxations for MAP using Message-Passing David Sontag Joint work with Talya Meltzer, Amir Globerson, Tommi Jaakkola, and Yair Weiss.
MAP Estimation in Binary MRFs using Bipartite Multi-Cuts Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay TexPoint.
Approximation Algorithms Duality My T. UF.
Linear Programming Piyush Kumar Welcome to CIS5930.
Ion I. Mandoiu, Vijay V. Vazirani Georgia Tech Joseph L. Ganley Simplex Solutions A New Heuristic for Rectilinear Steiner Trees.
Approximation Algorithms based on linear programming.
Chapter 8 PD-Method and Local Ratio (5) Equivalence This ppt is editored from a ppt of Reuven Bar-Yehuda. Reuven Bar-Yehuda.
Lap Chi Lau we will only use slides 4 to 19
Greedy Technique.
Topics in Algorithms Lap Chi Lau.
The minimum cost flow problem
Markov Random Fields with Efficient Approximations
Analysis of Algorithms
Linear Programming and Approximation
Discrete Inference and Learning
Presentation transcript:

Fast and accurate energy minimization for static or time-varying Markov Random Fields (MRFs) Nikos Komodakis (Ecole Centrale Paris) Nikos Paragios (Ecole Centrale Paris) Georgios Tziritas (University of Crete)

The MRF optimization problem vertices G = set of objects edges E = object relationships set L = discrete set of labels V p (x p ) = cost of assigning label x p to vertex p (also called single node potential) V pq (x p,x q ) = cost of assigning labels (x p,x q ) to neighboring vertices (p,q) (also called pairwise potential) Find labels that minimize the MRF energy (i.e., the sum of all potentials):

MRF optimization in vision MRFs ubiquitous in vision and beyond Have been used in a wide range of problems: segmentationstereo matching optical flowimage restoration image completion object detection & localization... MRF optimization is thus a task of fundamental importance Yet, highly non-trivial, since almost all interesting MRFs are actually NP-hard to optimize Many proposed algorithms (e.g., [Boykov,Veksler,Zabih], [Kolmogorov], [Kohli,Torr], [Wainwright]…)

MRF hardness MRF pairwise potential MRF hardness linear exact global optimum arbitrary local optimum metric global optimum approximation Move right in the horizontal axis, But we want to be able to do that efficiently, i.e. fast and remain low in the vertical axis (i.e., still be able to provide approximately optimal solutions)

Our contributions to MRF optimization Can handle a very wide class of MRFs General framework for optimizing MRFs based on duality theory of Linear Programming (the Primal-Dual schema) Can guarantee approximately optimal solutions (worst-case theoretical guarantees) Can provide tight certificates of optimality per-instance (per-instance guarantees) Fast-PD Provides significant speed-up for static MRFs Provides significant speed-up for dynamic MRFs

Presentation outline The primal-dual schema Applying the schema to MRF optimization Algorithmic properties  Worst-case optimality guarantees  Per-instance optimality guarantees  Computational efficiency for static MRFs  Computational efficiency for dynamic MRFs

The primal-dual schema Highly successful technique for exact algorithms. Yielded exact algorithms for cornerstone combinatorial problems: matchingnetwork flow minimum spanning tree minimum branching shortest path... Soon realized that it’s also an extremely powerful tool for deriving approximation algorithms: set coversteiner tree steiner networkfeedback vertex set scheduling...

The primal-dual schema  Say we seek an optimal solution x* to the following integer program (this is our primal problem): (NP-hard problem)  To find an approximate solution, we first relax the integrality constraints to get a primal & a dual linear program: primal LP: dual LP:

The primal-dual schema Goal: find integral-primal solution x, feasible dual solution y such that their primal-dual costs are “close enough”, e.g., primal cost of solution x primal cost of solution x dual cost of solution y dual cost of solution y cost of optimal integral solution x* cost of optimal integral solution x* Then x is an f * -approximation to optimal solution x*

The primal-dual schema sequence of dual costs sequence of primal costs … unknown optimum … The primal-dual schema works iteratively Global effects, through local improvements! Instead of working directly with costs (usually not easy), use RELAXED complementary slackness conditions (easier) Different relaxations of complementary slackness Different approximation algorithms!!!

(only one label assigned per vertex) enforce consistency between variables x p,a, x q,b and variable x pq,ab The primal-dual schema for MRFs Binary variables x p,a =1 label a is assigned to node p x pq,ab =1 labels a, b are assigned to nodes p, q x p,a =1 label a is assigned to node p x pq,ab =1 labels a, b are assigned to nodes p, q

The primal-dual schema for MRFs During the PD schema for MRFs, it turns out that: each update of primal and dual variables solving max-flow in appropriately constructed graph Max-flow graph defined from current primal-dual pair (x k,y k )  (x k,y k ) defines connectivity of max-flow graph  (x k,y k ) defines capacities of max-flow graph Max-flow graph is thus continuously updated Resulting flows tell us how to update both:  the dual variables, as well as  the primal variables for each iteration of primal-dual schema

The primal-dual schema for MRFs Very general framework. Different PD-algorithms by RELAXING complementary slackness conditions differently. Theorem: All derived PD-algorithms shown to satisfy certain relaxed complementary slackness conditions Worst-case optimality properties are thus guaranteed E.g., simply by using a particular relaxation of complementary slackness conditions (and assuming V pq (·,·) is a metric) THEN resulting algorithm shown equivalent to a-expansion! PD-algorithms for non-metric potentials V pq (·,·) as well

Per-instance optimality guarantees Primal-dual algorithms can always tell you (for free) how well they performed for a particular instance unknown optimum …… per-instance approx. factor per-instance lower bound (per-instance certificate) per-instance upper bound

Computational efficiency (static MRFs) MRF algorithm only in the primal domain (e.g., a-expansion) primal k primal k-1 primal 1 … primal costs dual 1 fixed dual cost gap k STILL BIG Many augmenting paths per max-flow Theorem: primal-dual gap = upper-bound on #augmenting paths (i.e., primal-dual gap indicative of time per max-flow) dual k dual 1 dual k-1 … dual costs gap k primal k primal k-1 primal 1 … primal costs SMALL Few augmenting paths per max-flow MRF algorithm in the primal-dual domain (Fast-PD)

Computational efficiency (static MRFs) dramatic decrease always very high Incremental construction of max-flow graphs (recall that max-flow graph changes per iteration) This is possible only because we keep both primal and dual information Our framework provides a principled way of doing this incremental graph construction for general MRFs noisy imagedenoised image

Computational efficiency (static MRFs) penguin Tsukuba SRI-tree almost constant dramatic decrease

Computational efficiency (dynamic MRFs) Fast-PD can speed up dynamic MRFs [Kohli,Torr] as well (demonstrates the power and generality of our framework) gap primal x dual y SMALL primal x gap dual y SMALL few path augmentations primal x SMALL gap dual 1 fixed dual cost primal x gap LARGE many path augmentations Our framework provides principled (and simple) way to update dual variables when switching between different MRFs Fast-PD algorithm primal-based algorithm

Computational efficiency (dynamic MRFs) Essentially, Fast-PD works along 2 different “axes”  reduces augmentations across different iterations of the same MRF  reduces augmentations across different MRFs Handles general (multi-label) dynamic MRFs Time per frame for SRI-tree stereo sequence

primal-dual framework Handles wide class of MRFs Approximately optimal solutions Theoretical guarantees AND tight certificates per instance Significant speed-up for static MRFs Significant speed-up for dynamic MRFs - New theorems - New insights into existing techniques - New view on MRFs Take-home message: try to take advantage of duality, whenever you can Thank you!

Papers and presentations available at: