MAP Estimation in Binary MRFs using Bipartite Multi-Cuts Sashank J. Reddi Sunita Sarawagi Sundar Vishwanathan Indian Institute of Technology, Bombay TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AA A AAA A A A

MAP Estimation Energy Function MAP Estimation: Find the labeling which minimizes the energy function – NP Hard in general xixi xjxj θ ij (x i,x j ) θ i (x i )

Popular Approximation Based on this LP relaxation Can be solved efficiently using Message Passing algorithms (TRW-S) and Graph cut based algorithms (QPBO)

Tightening Pairwise LP Relaxation MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm (Nikos Komodakis et.al 2008) MPLP uses higher order relaxation

Tightening Pairwise LP Relaxation MPLP (David Sontag et.al 2008), Cycle Repairing Algorithm (Nikos Komodakis et.al 2008) MPLP uses higher order relaxation

MAP Estimation via Graph Cuts Construct a specialized graph for the particular energy function Minimum cut on this graph minimizes the energy function Exact MAP in polynomial time when the energy function is sub-modular –

Assumption on Parameters Symmetric, that is and Zero-normalized, that is and Any Energy function can be transformed in to an equivalent energy function of this form

Graph Construction

i0i0 i1i1

i0i0 i1i1 θ i (1) θ i (0)

Graph Construction i0i0 j0j0 θ i (1) θ i (0) θ j (1) θ j (0) θ ij /2 θ ij = θ ij (0,1)+ θ ij (1,0)- θ ij (0,0)- θ ij (1,1) > 0 i1i1 j1j1

Graph Construction i0i0 j1j1 θ i (1) θ i (0) -θ ij /2 θ ij = θ ij (0,1)+ θ ij (1,0)- θ ij (0,0)- θ ij (1,1) < 0 i1i1 j0j0 θ j (1) θ j (0)

Bipartite Multi-Cut problem Given an undirected graph with non-negative edge weights and k ST pairs – Objective: Find the minimum cut with divides the graph into 2 regions and separates the ST pairs LP and SDP Relaxations (Sreyash Kenkre et.al 2006) – LP Relaxation gives approximation – SDP Relaxation gives approximation Bipartite Multi-Cut vs Multi-Cut – Additional constraint on the number of regions the graph is cut

BMC LP are the ST pairs in the Bipartite Multi-Cut problem Terminals =

BMC LP are the ST pairs in the Bipartite Multi-Cut problem Terminals = isis jtjt dede D 0 0 (i s ) D 0 0 (j t ) k0k0 k1k1 D k 0 (i s ) D k 0 (j t )

BMC LP are the ST pairs in the Bipartite Multi-Cut problem Terminals = i0i0 i1i1 j0j0 j1j1

BMC LP are the ST pairs in the Bipartite Multi-Cut problem Terminals = isis jtjt dede D 0 0 (i s ) D 0 0 (j t )

BMC LP Infeasible to solve it using LP solvers – Large number of constraints Combinatorial Algorithm – Solving LP for general graphs may not be easy – Flexibility in construction of the graph – Combinatorial algorithm for a special kind of construction

Symmetric Graph Construction i0i0 j0j0 θ j (1)/2 θ j (0)/2θ i (0)/2 θ ij /4 θ ij = θ ij (0,1)+ θ ij (1,0)- θ ij (0,0)- θ ij (1,1) i1i1 j1j1 θ j (0)/2 θ j (1)/2 θ i (0)/2 θ i (1)/2 θ ij /4 θ i (1)/2

Symmetric Graph Construction More ST pairs are added Trade off is combinatorial algorithm

Approach of the Algorithm BMC LP Multicommodity flow LP MultiCut LP

Multi-Cut LP denotes all paths between pair of vertices in ST denotes the set of paths in which contain edge e Can be solved approximately i.e ε-approximation for any error parameter ε Multi – Cut LPMulti -Commodity Flow LP

Symmetric BMC LP (BMC-Sym LP) Very similar to Multi-Cut LP except for the symmetric constraints

Equivalence of LPs Theorem – When the constructed graph is symmetric, BMC LP, BMC-Sym LP and Multi-Cut LP are equivalent Proof Outline Any feasible solution of each of the LP can be transformed into a feasible solution of other LPs without changing the objective value

Combinatorial Algorithm Primal Step Find shortest path P between Update Update Complementary path Dual Step Let Update the flow in the path by Update flow in complementary path Converges in Can be improved to (Fleischer 1999)

Relationship with Cycle Inequalities BMC LP is closely related to cycle inequalities – BMC LP with slight modification is equivalent to cycle inequalities Relates our work to many recent works solving cycle inequalities

Empirical Results Data Sets – Synthetic Problems Clique Graphical models – Benchmark Data Set Algorithms – TRW-S – BMC ε = 0.02 – MPLP 1000 iterations or up to convergence

Empirical Results

Convergence Comparison of BMC and MPLP

Empirical Results

Conclusion & Future Work Conclusion – MAP estimation can be reduced to Bipartite Multi-Cut problem – Algorithms for multi commodity flow can be used for MAP estimation – BMC LP is closely related to cycle inequalities Future Work – Extensions to multi-label graphical models – Combinatorial algorithm for solving SDP relaxation