Download presentation
Presentation is loading. Please wait.
Published byMary Price Modified over 10 years ago
1
Solving Markov Random Fields using Second Order Cone Programming Relaxations M. Pawan Kumar Philip Torr Andrew Zisserman
2
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Labelling m = {1, 0, 0, 1} Random Variables V = {V 1,..,V 4 } Label Set L = {0,1}
3
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2
4
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1
5
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2
6
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2 + 1
7
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3
8
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1
9
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3
10
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Cost(m) = 2 + 1 + 2 + 1 + 3 + 1 + 3 = 13 Minimum Cost Labelling = MAP estimate Pr(m) exp(-Cost(m))
11
Aim Accurate MAP estimation of pairwise Markov random fields 2 5 4 2 6 3 3 7 0 1 1 0 0 2 3 1 1 41 0 V1V1 V2V2 V3V3 V4V4 Label 0 Label 1 Objectives Applicable for all neighbourhood relationships Applicable for all forms of pairwise costs Guaranteed to converge
12
Motivation Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 Unary costs are uniform V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D
13
Motivation Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 | d(m i,m j ) - d(V i,V j ) | < 1 2 YESNO Potts Model Pairwise Costs
14
Motivation V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Subgraph Matching - Torr - 2003, Schellewald et al - 2005
15
Motivation V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Subgraph Matching - Torr - 2003, Schellewald et al - 2005
16
Motivation Matching Pictorial Structures - Felzenszwalb et al - 2001 Part likelihoodSpatial Prior Outline Texture Image P1P1 P3P3 P2P2 (x,y,, ) MRF
17
Motivation Image P1P1 P3P3 P2P2 (x,y,, ) MRF Unary potentials are negative log likelihoods Valid pairwise configuration Potts Model Matching Pictorial Structures - Felzenszwalb et al - 2001 1 2 YESNO
18
Motivation P1P1 P3P3 P2P2 (x,y,, ) Pr(Cow)Image Unary potentials are negative log likelihoods Matching Pictorial Structures - Felzenszwalb et al - 2001 Valid pairwise configuration Potts Model 1 2 YESNO
19
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
20
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Unary Cost Unary Cost Vector u = [ 5 Cost of V 1 = 0 2 Cost of V 1 = 1 ; 2 4 ] Labelling m = {1, 0}
21
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Unary Cost Unary Cost Vector u = [ 5 2 ; 2 4 ] T Labelling m = {1, 0} Label vector x = [ -1 V 1 0 1 V 1 = 1 ; 1 -1 ] T Recall that the aim is to find the optimal x
22
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Unary Cost Unary Cost Vector u = [ 5 2 ; 2 4 ] T Labelling m = {1, 0} Label vector x = [ -11; 1 -1 ] T Sum of Unary Costs = 1 2 i u i (1 + x i )
23
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Pairwise Cost Labelling m = {1, 0} 0 Cost of V 1 = 0 and V 1 = 0 0 00 0 Cost of V 1 = 0 and V 2 = 0 3 Cost of V 1 = 0 and V 2 = 1 10 00 00 10 30 Pairwise Cost Matrix P
24
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Pairwise Cost Labelling m = {1, 0} Pairwise Cost Matrix P 00 00 0 3 10 00 00 10 30 Sum of Pairwise Costs 1 4 ij P ij (1 + x i )(1+x j )
25
Integer Programming Formulation 2 5 4 2 0 1 3 0 V1V1 V2V2 Label 0 Label 1 Pairwise Cost Labelling m = {1, 0} Pairwise Cost Matrix P 00 00 0 3 10 00 00 10 30 Sum of Pairwise Costs 1 4 ij P ij (1 + x i +x j + x i x j ) 1 4 ij P ij (1 + x i + x j + X ij )= X = x x T X ij = x i x j
26
Integer Programming Formulation Constraints Each variable should be assigned a unique label x i = 2 - |L| i V a Marginalization constraint X ij = (2 - |L|) x i j V b
27
Integer Programming Formulation Chekuri et al., SODA 2001 x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Convex Non-Convex
28
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
29
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Chekuri et al., SODA 2001 Retain Convex Part Relax Non-convex Constraint
30
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Chekuri et al., SODA 2001 Retain Convex Part Relax Non-convex Constraint
31
Linear Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] Chekuri et al., SODA 2001 Retain Convex Part
32
Feasible Region (IP) x {-1,1}, X = x 2 Linear Programming Formulation
33
Feasible Region (IP) Feasible Region (Relaxation 1) x {-1,1}, X = x 2 x [-1,1], X = x 2 Linear Programming Formulation
34
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1] Linear Programming Formulation
35
Bounded algorithms proposed by Chekuri et al, SODA 2001 -expansion - Komodakis and Tziritas, ICCV 2005 TRW - Wainwright et al., NIPS 2002 TRW-S - Kolmogorov, AISTATS 2005 Efficient because it uses Linear Programming Not accurate
36
Semidefinite Programming Formulation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i {-1,1} X = x x T Lovasz and Schrijver, SIAM Optimization, 1990 Retain Convex Part Relax Non-convex Constraint
37
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Semidefinite Programming Formulation Retain Convex Part Relax Non-convex Constraint Lovasz and Schrijver, SIAM Optimization, 1990
38
Semidefinite Programming Formulation x1x1 x2x2 xnxn 1... 1x1x1 x2x2... xnxn 1xTxT x X = Rank = 1 X ii = 1 Positive Semidefinite Convex Non-Convex
39
Semidefinite Programming Formulation x1x1 x2x2 xnxn 1... 1x1x1 x2x2... xnxn 1xTxT x X = X ii = 1 Positive Semidefinite Convex
40
Schurs Complement AB BTBT C = I0 B T A -1 I A0 0 C - B T A -1 B IA -1 B 0 I 0 A 0 C -B T A -1 B 0
41
Semidefinite Programming Formulation X - xx T 0 1xTxT x X = 10 x I 10 0 X - xx T IxTxT 0 1 Schurs Complement
42
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X = x x T Semidefinite Programming Formulation Relax Non-convex Constraint Retain Convex Part Lovasz and Schrijver, SIAM Optimization, 1990
43
x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] Semidefinite Programming Formulation X ii = 1 X - xx T 0 Retain Convex Part Lovasz and Schrijver, SIAM Optimization, 1990
44
Feasible Region (IP) x {-1,1}, X = x 2 Semidefinite Programming Formulation
45
Feasible Region (IP) Feasible Region (Relaxation 1) x {-1,1}, X = x 2 x [-1,1], X = x 2 Semidefinite Programming Formulation
46
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1], X x 2 Semidefinite Programming Formulation
47
Formulated by Lovasz and Schrijver, 1990 Finds a full X matrix Max-cut - Goemans and Williamson, JACM 1995 Max-k-cut - de Klerk et al, 2000 Accurate Not efficient because of Semidefinite Programming
48
Previous Work - Overview LPSDP Examples TRW-S, -expansion Max-k-Cut AccuracyLowHigh EfficiencyHighLow Is there a Middle Path ???
49
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
50
Second Order Cone Programming Second Order Cone || v || t OR || v || 2 st x 2 + y 2 z 2
51
Minimize f T x Subject to || A i x+ b i || <= c i T x + d i i = 1, …, L Linear Objective Function Affine mapping of Second Order Cone (SOC) Constraints are SOC of n i dimensions Feasible regions are intersections of conic regions Second Order Cone Programming
52
|| v || t tIv vTvT t 0 LP SOCP SDP = 10 vTvT I tI0 0 t 2 - v T v Iv 0 1 Schurs Complement
53
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
54
Matrix Dot Product AB = ij A ij B ij A 11 A 12 A 21 A 22 B 11 B 12 B 21 B 22 = A 11 B 11 + A 12 B 12 + A 21 B 21 + A 22 B 22
55
SDP Relaxation x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X ii = 1 X - xx T 0 Derive SOCP relaxation from the SDP relaxation Further Relaxation
56
1-D Example X - xx T 0 X - x 2 0 For two semidefinite matrices, the dot product is non-negative A A 0 x 2 X SOC of the form || v || 2 st
57
Feasible Region (IP) Feasible Region (Relaxation 1) Feasible Region (Relaxation 2) x {-1,1}, X = x 2 x [-1,1], X = x 2 x [-1,1], X x 2 SOCP Relaxation Same as the SDP formulation
58
2-D Example X 11 X 12 X 21 X 22 1X 12 1 = X = x1x1x1x1 x1x2x1x2 x2x1x2x1 x2x2x2x2 xx T = x12x12 x1x2x1x2 x1x2x1x2 = x22x22
59
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2. 0 10 00 1 - x 2 2 x 1 2 1 -1 x 1 1 C 1. 0 C 1 0
60
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 2. 0. 0 00 01 X 12 -x 1 x 2 1 - x 2 2 x 2 2 1 LP Relaxation -1 x 2 1 C 2 0
61
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 3. 0. 0 11 11 X 12 -x 1 x 2 1 - x 2 2 (x 1 + x 2 ) 2 2 + 2X 12 SOC of the form || v || 2 st C 3 0
62
2-D Example (X - xx T ) 1 - x 1 2 X 12 -x 1 x 2 C 4. 0. 0 1 1 X 12 -x 1 x 2 1 - x 2 2 (x 1 - x 2 ) 2 2 - 2X 12 SOC of the form || v || 2 st C 4 0
63
SOCP Relaxation Consider a matrix C 1 = UU T 0 (X - xx T ) ||U T x || 2 X. C 1 C 1. 0 Continue for C 2, C 3, …, C n SOC of the form || v || 2 st Kim and Kojima, 2000
64
SOCP Relaxation How many constraints for SOCP = SDP ? Infinite. For all C 0 We specify constraints similar to the 2-D example
65
SOCP Relaxation Muramatsu and Suzuki, 2001 10 00 00 01 11 11 1 1 Constraints hold for the above semidefinite matrices
66
SOCP Relaxation Muramatsu and Suzuki, 2001 10 00 00 01 11 11 1 1 a + b + c+ d a 0 b 0 c 0 d 0 Constraints hold for the linear combination
67
SOCP Relaxation Muramatsu and Suzuki, 2001 a+c+dc-d b+c+d a 0 b 0 c 0 d 0 Includes all semidefinite matrices where Diagonal elements Off-diagonal elements
68
SOCP Relaxation - A x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] X ii = 1 X - xx T 0
69
SOCP Relaxation - A x* = argmin 1 2 u i (1 + x i ) + 1 4 P ij (1 + x i + x j + X ij ) x i = 2 - |L| i V a X ij = (2 - |L|) x i j V b x i [-1,1] (x i + x j ) 2 2 + 2X ij (x i - x j ) 2 2 - 2X ij Specified only when P ij 0
70
Triangular Inequality At least two of x i, x j and x k have the same sign At least one of X ij, X jk, X ik is equal to one X ij + X jk + X ik -1 X ij - X jk - X ik -1 -X ij - X jk + X ik -1 -X ij + X jk - X ik -1 SOCP-B = SOCP-A + Triangular Inequalities
71
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
72
Robust Truncated Model Pairwise cost of incompatible labels is truncated Potts ModelTruncated Linear Model Truncated Quadratic Model Robust to noise Widely used in Computer Vision - Segmentation, Stereo
73
Robust Truncated Model Pairwise Cost Matrix can be made sparse P = [0.5 0.5 0.3 0.3 0.5] Q = [0 0 -0.2 -0.2 0] Reparameterization Sparse Q matrix Fewer constraints
74
Compatibility Constraint Q(m a, m b ) < 0 for variables V a and V b Relaxation Q ij (1 + x i + x j + X ij ) < 0 SOCP-C = SOCP-B + Compatibility Constraints
75
SOCP Relaxation More accurate than LP More efficient than SDP Time complexity - O( |V| 3 |L| 3 ) Same as LP Approximate algorithms exist for LP relaxation We use |V| 10 and |L| 200
76
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
77
Subgraph Matching Subgraph Matching - Torr - 2003, Schellewald et al - 2005 G1G1 G2G2 Unary costs are uniform V2V2 V3V3 V1V1 MRF A B C D A B C D A B C D Pairwise costs form a Potts model
78
Subgraph Matching 1000 pairs of graphs G 1 and G 2 #vertices in G 2 - between 20 and 30 #vertices in G 1 - 0.25 * #vertices in G 2 5% noise to the position of vertices NP-hard problem
79
Subgraph Matching Method Time (sec)Accuracy (%) LP0.856.64 SDP-A35.093.11 SOCP-A3.092.01 SOCP-B4.594.79 SOCP-C4.896.18
80
Outline Integer Programming Formulation Previous Work Our Approach –Second Order Cone Programming (SOCP) –SOCP Relaxation –Robust Truncated Model Applications –Subgraph Matching –Pictorial Structures
81
Pictorial Structures Image P1P1 P3P3 P2P2 (x,y,, ) MRF Matching Pictorial Structures - Felzenszwalb et al - 2001 Outline Texture
82
Pictorial Structures Image P1P1 P3P3 P2P2 (x,y,, ) MRF Unary costs are negative log likelihoods Pairwise costs form a Potts model | V | = 10| L | = 200
83
Pictorial Structures LBP GBP SOCP
84
Pictorial Structures LBP GBP SOCP
85
Pictorial Structures LBP GBP SOCP
86
Pictorial Structures LBP GBP SOCP LBP and GBP do not converge
87
Pictorial Structures ROC Curves for 450 +ve and 2400 -ve images
88
Pictorial Structures ROC Curves for 450 +ve and 2400 -ve images
89
Conclusions We presented an SOCP relaxation to solve MRF More efficient than SDP More accurate than LP, LBP, GBP #variables can be reduced for Robust Truncated Model Provides excellent results for subgraph matching and pictorial structures
90
Future Work Quality of solution –Additive bounds exist –Multiplicative bounds for special cases ?? Message passing algorithm ?? –Similar to TRW-S or -expansion –To handle image sized MRF
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.