An Efficient Message-Passing Algorithm for the M-Best MAP Problem Dhruv Batra (Currently) Research Assistant Professor TTI-Chicago (Spring 2013) Assistant.

Slides:

Advertisements

Similar presentations

Mean-Field Theory and Its Applications In Computer Vision1 1.

Advertisements

Bayesian Belief Propagation

1 LP, extended maxflow, TRW OR: How to understand Vladimirs most recent work Ramin Zabih Cornell University.

O BJ C UT M. Pawan Kumar Philip Torr Andrew Zisserman UNIVERSITY OF OXFORD.

Diverse M-Best Solutions in Markov Random Fields

Focused Inference with Local Primal-Dual Gaps

ECE Longest Path dual 1 ECE 665 Spring 2005 ECE 665 Spring 2005 Computer Algorithms with Applications to VLSI CAD Linear Programming Duality – Longest.

Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.

Tutorial at ICCV (Barcelona, Spain, November 2011)

Exact Inference in Bayes Nets

ICCV 2007 tutorial Part III Message-passing algorithms for energy minimization Vladimir Kolmogorov University College London.

Discrete Optimization in Computer Vision Nikos Komodakis Ecole des Ponts ParisTech, LIGM Traitement de l’information et vision artiﬁcielle.

Convergent Message-Passing Algorithms for Inference over General Graphs with Convex Free Energies Tamir Hazan, Amnon Shashua School of Computer Science.

Overview of Inference Algorithms for Bayesian Networks Wei Sun, PhD Assistant Research Professor SEOR Dept. & C4I Center George Mason University, 2009.

1 Fast Primal-Dual Strategies for MRF Optimization (Fast PD) Robot Perception Lab Taha Hamedani Aug 2014.

Belief Propagation on Markov Random Fields Aggeliki Tsoli.

F IXING M AX -P RODUCT : A U NIFIED L OOK AT M ESSAGE P ASSING A LGORITHMS Nicholas Ruozzi and Sekhar Tatikonda Yale University.

Learning to Detect A Salient Object Reporter: 鄭綱 (3/2)

Distributed Message Passing for Large Scale Graphical Models Alexander Schwing Tamir Hazan Marc Pollefeys Raquel Urtasun CVPR2011.

P 3 & Beyond Solving Energies with Higher Order Cliques Pushmeet Kohli Pawan Kumar Philip H. S. Torr Oxford Brookes University CVPR 2007.

Message Passing Algorithms for Optimization

Efficiently Solving Convex Relaxations M. Pawan Kumar University of Oxford for MAP Estimation Philip Torr Oxford Brookes University.

Understanding Belief Propagation and its Applications Dan Yuan June 2004.

Integer Programming Difference from linear programming –Variables x i must take on integral values, not real values Lots of interesting problems can be.

What Energy Functions Can be Minimized Using Graph Cuts? Shai Bagon Advanced Topics in Computer Vision June 2010.

An Iterative Optimization Approach for Unified Image Segmentation and Matting Hello everyone, my name is Jue Wang, I’m glad to be here to present our paper.

Relaxations and Moves for MAP Estimation in MRFs M. Pawan Kumar STANFORDSTANFORD Vladimir KolmogorovPhilip TorrDaphne Koller.

Measuring Uncertainty in Graph Cut Solutions Pushmeet Kohli Philip H.S. Torr Department of Computing Oxford Brookes University.

A Trainable Graph Combination Scheme for Belief Propagation Kai Ju Liu New York University.

MAP Estimation Algorithms in M. Pawan Kumar, University of Oxford Pushmeet Kohli, Microsoft Research Computer Vision - Part I.

Probabilistic Inference Lecture 4 – Part 2 M. Pawan Kumar Slides available online

CS774. Markov Random Field : Theory and Application Lecture 08 Kyomin Jung KAIST Sep

Procedural Modeling of Architectures towards 3D Reconstruction Nikos Paragios Ecole Centrale Paris / INRIA Saclay Ile-de-France Joint Work: P. Koutsourakis,

MRFs and Segmentation with Graph Cuts Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/24/10.

Planar Cycle Covering Graphs for inference in MRFS The Typhon Algorithm A New Variational Approach to Ground State Computation in Binary Planar Markov.

CVPR 2013 Diversity Tutorial Closing Remarks: What can we do with multiple diverse solutions? Dhruv Batra Virginia Tech.

1 Structured Region Graphs: Morphing EP into GBP Max Welling Tom Minka Yee Whye Teh.

Multiplicative Bounds for Metric Labeling M. Pawan Kumar École Centrale Paris Joint work with Phil Torr, Daphne Koller.

Probabilistic Graphical Models

Markov Random Fields Probabilistic Models for Images

Fast Parallel and Adaptive Updates for Dual-Decomposition Solvers Ozgur Sumer, U. Chicago Umut Acar, MPI-SWS Alexander Ihler, UC Irvine Ramgopal Mettu,

Approximate Dynamic Programming Methods for Resource Constrained Sensor Management John W. Fisher III, Jason L. Williams and Alan S. Willsky MIT CSAIL.

Algorithms for MAP estimation in Markov Random Fields Vladimir Kolmogorov University College London.

Associative Hierarchical CRFs for Object Class Image Segmentation Ľubor Ladický 1 1 Oxford Brookes University 2 Microsoft Research Cambridge Based on the.

Discrete Optimization Lecture 3 – Part 1 M. Pawan Kumar Slides available online

Fast and accurate energy minimization for static or time-varying Markov Random Fields (MRFs) Nikos Komodakis (Ecole Centrale Paris) Nikos Paragios (Ecole.

Probabilistic Inference Lecture 5 M. Pawan Kumar Slides available online

Dynamic Tree Block Coordinate Ascent Daniel Tarlow 1, Dhruv Batra 2 Pushmeet Kohli 3, Vladimir Kolmogorov 4 1: University of Toronto3: Microsoft Research.

Two Approximate Algorithms for Belief Updating Mini-Clustering - MC Robert Mateescu, Rina Dechter, Kalev Kask. "Tree Approximation for Belief Updating",

Update any set S of nodes simultaneously with step-size We show fixed point update is monotone for · 1/|S| Covering Trees and Lower-bounds on Quadratic.

Approximate Inference: Decomposition Methods with Applications to Computer Vision Kyomin Jung ( KAIST ) Joint work with Pushmeet Kohli (Microsoft Research)

CVPR 2013 Diversity Tutorial Beyond MAP: Making Multiple Predictions: Diversity, DPPs and more. Dhruv Batra Virginia Tech Alex Kulesza Univ. of Michigan.

Discrete Optimization Lecture 2 – Part 2 M. Pawan Kumar Slides available online

Using Combinatorial Optimization within Max-Product Belief Propagation

Implicit Hitting Set Problems Richard M. Karp Erick Moreno Centeno DIMACS 20 th Anniversary.

Exact Inference in Bayes Nets. Notation U: set of nodes in a graph X i : random variable associated with node i π i : parents of node i Joint probability:

Beyond Trees: MRF Inference via Outer-Planar Decomposition [(To appear) CVPR ‘10] Dhruv Batra Carnegie Mellon University Joint work: Andrew Gallagher (Eastman.

Inference for Learning Belief Propagation. So far... Exact methods for submodular energies Approximations for non-submodular energies Move-making ( N_Variables.

Join-graph based cost-shifting Alexander Ihler, Natalia Flerova, Rina Dechter and Lars Otten University of California Irvine Introduction Mini-Bucket Elimination.

Eric Xing © Eric CMU, Machine Learning Algorithms and Theory of Approximate Inference Eric Xing Lecture 15, August 15, 2010 Reading:

Tightening LP Relaxations for MAP using Message-Passing David Sontag Joint work with Talya Meltzer, Amir Globerson, Tommi Jaakkola, and Yair Weiss.

MAP Estimation in Binary MRFs using Bipartite Multi-Cuts Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay TexPoint.

IEEE 2015 Conference on Computer Vision and Pattern Recognition Active Learning for Structured Probabilistic Models with Histogram Approximation Qing SunAnkit.

Perfect recall: Every decision node observes all earlier decision nodes and their parents (along a “temporal” order) Sum-max-sum rule (dynamical programming):

The M-Best Mode Problem

Alexander Shekhovtsov and Václav Hlaváč

CIS 700 Advanced Machine Learning for NLP Inference Applications

StatSense In-Network Probabilistic Inference over Sensor Networks

Parallel and Distributed Graph Cuts by Dual Decomposition

Discrete Inference and Learning

Presentation transcript:

An Efficient Message-Passing Algorithm for the M-Best MAP Problem Dhruv Batra (Currently) Research Assistant Professor TTI-Chicago (Spring 2013) Assistant Professor Virginia Tech

Local Ambiguity Graphical Models (C) Dhruv Batra2 x1x1 x2x2 … xnxn MAP Inference Most Likely Assignment MAP Problem Cat Hat

Global Ambiguity “ While hunting in Africa, I shot an elephant in my pajamas. How an elephant got into my pajamas, I’ll never know! ” –Groucho Marx (1930) (C) Dhruv Batra3

M-Best MAP Useful for: –Generating multiple hypotheses when model is inaccurate –Passing on hypotheses to next stage in cascade –Show multiple solutions to users Generalization of MAP, thus NP-Hard (C) Dhruv Batra4

History MAPM-Best MAP (C) Dhruv Batra5

History MAPM-Best MAP Message-Passing Algs - Dynamic Programming - Belief Propagation style [Pearl ’82], [Lauritzen & Spiegelhalter ‘88], [Shafer & Shenoy ‘86] [Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11] (C) Dhruv Batra6

History MAPM-Best MAP Message-Passing Algs - Dynamic Programming - Belief Propagation style [Pearl ’82], [Lauritzen & Spiegelhalter ‘88], [Shafer & Shenoy ‘86] [Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11] Linear Programming Formulation [Schlesinger ‘76], [Wainwright et al. ‘05], [Komodakis ’07] [Fromer & Globerson ’09] (C) Dhruv Batra7

History MAPM-Best MAP Message-Passing Algs - Dynamic Programming - Belief Propagation style [Pearl ’82], [Lauritzen & Spiegelhalter ‘88], [Shafer & Shenoy ‘86] [Seroussi & Golmard ‘94], [Flerova & Dechter ‘10], [Yanover & Weiss ‘03], [Flerova & Dechter ’11] Linear Programming Formulation [Schlesinger ‘76], [Wainwright et al. ‘05], [Komodakis ’07] [Fromer & Globerson ’09] Message-Passing for solving LP [Schlesinger ‘76], [Wainwright et al. ‘05], [Kolmogorov ‘06], [Komodakis ’07], [Werner ’07] (C) Dhruv Batra8 This Work [Batra UAI ’12] ?

Contributions First message-passing alg for solving M-Best MAP LP of [Fromer & Globerson NIPS09] Guaranteed to get exact solution to LP Orders of magnitude faster than a generic LP solver (C) Dhruv Batra9 LP-solver Our Approach #Nodes Time (sec) Better

Outline (C) Dhruv Batra10 Tree-MRF M=2 Tree-MRF M>2 Loopy MRF M=2 Loopy MRF M>2 M Cycles - Partition Enumeration Scheme [Fromer & Globerson NIPS09] - Others Details in Paper M=2  M>2 Schemes

Background Over-Complete Representation 11(C) Dhruv Batra x1x1 x2x2 … xnxn XiXi kxk … … … kx1 … … … ……

Background Over-Complete Representation 12(C) Dhruv Batra x1x1 x2x2 … xnxn XiXi …… k 2 x

Background MAP Integer Program (C) Dhruv Batra13

Background MAP Linear Program Properties –If LP-opt is integral, MAP is found –LP always integral for trees –Efficient message-passing schemes for solving LP (C) Dhruv Batra14

Outline (C) Dhruv Batra15 Tree-MRF M=2 Tree-MRF M>2 Loopy MRF M=2 Loopy MRF M>2 M Cycles

M-Best MAP LP: Tree (C) Dhruv Batra16 Spanning-Tree Inequality [Fromer & Globerson NIPS09]

M-Best MAP LP: Tree (C) Dhruv Batra17 ~ 10 6 x 10 6 Generic LP-solver: CPLEX [Fromer & Globerson NIPS09]

M-Best MAP LP: Tree Lagrangian Relaxation (C) Dhruv Batra18 Dualize 2-Pass Belief Propagation Similarity-Augmented Energy

M-Best MAP LP: Tree Lagrangian Relaxation Dual Problem (C) Dhruv Batra19 2 nd Best MAP energy Concave (Non-smooth) Lower-Bound on 2 nd Best MAP energy upergradient Ascent

M-Best MAP LP: Tree Lagrangian Relaxation Dual Problem (C) Dhruv Batra20 upergradient Ascent Primal Block Dual Block primal point dual point

M-Best MAP LP: Tree Lagrangian Relaxation Dual Problem Guarantees –Suitable choice of stepsize solves Lagrangian [Shor ‘85] –LP => Strong Duality (C) Dhruv Batra21 upergradient Ascent

Outline (C) Dhruv Batra22 Tree-MRF M=2 Tree-MRF M>2 Loopy MRF M=2 Loopy MRF M>2 M Cycles

M-Best MAP LP: Loopy-MRFs (C) Dhruv Batra23 …,,

M-Best MAP LP: Loopy-MRFs (C) Dhruv Batra24 Dualize …,, Problems 1. Exponentially many Lagrangian Terms 2. Collection of factors not a tree

Exponentially Many Terms (C) Dhruv Batra25 Primal Block primal point Dual Block dual point Constraint Management primal point dual point Tree Subset upergradient Ascent Dynamic …

Exponentially Many Terms (C) Dhruv Batra26 Primal Block Dual Block Constraint Management primal point dual point Tree Subset …,, upergradient Ascent Dynamic Max-Weight Spanning Tree Same as [Fromer & Globerson]

Loopy Graph (C) Dhruv Batra27 Problems 1. Exponentially many Lagrangian Terms 2. Collection of factors not a tree … Dual Decomposition

M-Best MAP LP: Loopy-MRFs Guarantees –Dynamic Supergradient Ascent w/ Max-Violation Oracle solves Lagrangian Relaxation [Emiel & Sagastizabal ‘08] –LP => Strong Duality (C) Dhruv Batra28

Experiments Synthetic Data –Trees –Grid Graphs –Energies sampled from Gaussians Methods –STEELARS: Spanning TREE LAgrangian Relaxation Scheme [Proposed] –STRIPES [Fromer & Globerson NIPS09] –BMMF [Yanover & Weiss NIPS03] –NILSSON [Nilsson Stat. & Comp. 98] (C) Dhruv Batra29

Results: Tree-MRFs (C) Dhruv Batra30 Better

Results: Loopy-MRFs (C) Dhruv Batra31 Better

Extension: Diverse M-Best (C) Dhruv Batra32 Diverse M-Best Solutions in MRFs Batra, Yadollahpour, Guzman, Shakhnarovich ECCV 2012 Task-Specific Diversity

Extension: Diverse M-Best Interactive Segmentation (C) Dhruv Batra33 Image + Scribbles2 nd Best Mode 2 nd Best MAPMAP 1-2 Nodes Flipped Nodes Flipped

Extension: Diverse M-Best (C) Dhruv Batra34 InputMAPBest Mode

Conclusions First message-passing alg for solving M-Best MAP LP Guaranteed to get exact solution to LP Orders of magnitude faster than a generic LP solver Extension: –Diverse M-Best Solutions in MRFs Batra, Yadollahpour, Guzman, Shakhnarovich ECCV 2012 (C) Dhruv Batra35

Thank you! (C) Dhruv Batra36

Results: Tree-MRFs (C) Dhruv Batra37

Quality of Solutions: Loopy-MRFs (C) Dhruv Batra38

Results: Loopy-MRFs (C) Dhruv Batra39

Applications What can we do with multiple solutions? –More choices for “human/expert in the loop” (C) Dhruv Batra40

Applications What can we do with multiple solutions? –More choices for “human/expert in the loop” –Input to next system in cascade (C) Dhruv Batra41 Step 1Step 2Step 3 Top M hypotheses Top M hypotheses

Applications What can we do with multiple solutions? –More choices for “human in the loop” –Rank solutions (C) Dhruv Batra42 [Carreira and Sminchisescu, CVPR10] State-of-art segmentation on PASCAL Challenge 2011 ~10,000