1. An Analysis of Convex Relaxations for MAP Estimation
M. Pawan Kumar Vladimir Kolmogorov Philip H.S. Torr
Aim: To analyze Maximum a Posteriori (MAP) estimation methods based on convex relaxations
Comparing Relaxations
Two New SOCP Relaxations
Domination
For all (u,P)
For at least one (u,P)
All LP-S constraints
SOCP-C a b a b
MAP Estimation - Integer Programming Formulation
Cycle ‘G’ ≥
> 4
[ -1, 1 ; 1, -1] 2
Label Vector x c d c 3 1 5 2
strictly
dominates
[ 5, 2 ; 2, 4]
= 0
Unary Vector u A
dominates B A B Va Vb
= 1
Pairwise Matrix P - 3 1 a b b c c a
Equivalent relaxations: A dominates B, B dominates A.
Random Field Example
Unary Cost = 0
LP-S = 0
SOCP-C = 0.75
#variables n = 2
#labels h = 2
A  B = ∑ Aij Bij
Comparing Existing SOCP and QP Relaxations
Dominated by linear cycle inequalities?
Xab;ij
min P  X
xa;i
xb;j
SOCP-Q
All cycle inequalities
arg min xT (4u + 2P1) + P  X,
subject to ∑i xa;i = 2 - h
x  {-1,1}nh
X = x xT
Pab;ij ≥ 0
Xab;ij = infimum a b a b
MAP x*
Pab;ij < 0
Xab;ij = supremum
Clique ‘G’
(xa;i+ xb;j)2 ≤ Xab;ij c d c
(xa;i- xb;j)2 ≤ Xab;ij
SOCP-MS is QP-RL
SOCP-MS
Muramatsu and Suzuki, 2003
Ravikumar and Lafferty, 2006
= -1/3
LP-S vs. SOCP Relaxations over Trees and Cycles a b b c c d
G = (V,E)
= 1/3
Linear Programming (LP) Relaxation
Schlesinger, 1976 Va Vb a b a b 2 1 1 d a a c b d
LP-S 1 2 1 1
xa;1 = -1/3
xa;0 = -1/3
Non-convex
Constraint
X = x xT
x  {-1,1}nh 1 1 2 1
Dominates linear cycle inequalities Vd Vc d c d c 1 1 2
Open Questions
Convex
Relaxation V
(Constrained Variables)
1+xa;i+xb;j+Xab;ij ≥ 0 E
(Constrained Pairs)
x  [-1,1]nh
Random Field
C Matrix
Cycle inequalities vs. SOCP/QP?
j Xab;ij = (2-h) xa;i
Tree (SOCP-T)
Even Cycle (SOCP-E)
Odd Cycle (SOCP-O)
Best ‘C’ for special cases?
Second Order Cone Programming (SOCP) Relaxations
Efficient solutions for SOCP?
Non-convex
Constraint
X = x xT
Convex
Relaxation
Kim and Kojima, 2000
Future Work
C = U UT
50 random fields
4 neighbourhood
8 neighbourhood
||UTx||2≤ C  X
LP-S dominates SOCP-T
Pab;ij ≥ 0
Pab;ij ≥ 0 for one (a,b) … 
Special Case: Edge
Pab;ij ≤ 0
Pab;ij ≤ 0 for one/all (a,b)
SOCP-MS/ QP-RL
LP-S dominates SOCP-E
LP-S dominates SOCP-O