Download presentation
Presentation is loading. Please wait.
Published byEzra Charles Modified over 9 years ago
1
Discrete Optimization Lecture 2 – Part 2 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://cvn.ecp.fr/personnel/pawan/
2
Integer Programming Formulation LP Relaxation Outline
3
Integer Programming Formulation VaVa VbVb Label l 0 Label l 1 2 5 4 2 0 1 1 0 2 Unary Potentials a;0 = 5 a;1 = 2 b;0 = 2 b;1 = 4 Labeling f(a) = 1 f(b) = 0 y a;0 = 0y a;1 = 1 y b;0 = 1y b;1 = 0 Any f(.) has equivalent boolean variables y a;i
4
Integer Programming Formulation VaVa VbVb 2 5 4 2 0 1 1 0 2 Unary Potentials a;0 = 5 a;1 = 2 b;0 = 2 b;1 = 4 Labeling f(a) = 1 f(b) = 0 y a;0 = 0y a;1 = 1 y b;0 = 1y b;1 = 0 Find the optimal variables y a;i Label l 0 Label l 1
5
Integer Programming Formulation VaVa VbVb 2 5 4 2 0 1 1 0 2 Unary Potentials a;0 = 5 a;1 = 2 b;0 = 2 b;1 = 4 Sum of Unary Potentials ∑ a ∑ i a;i y a;i y a;i {0,1}, for all V a, l i ∑ i y a;i = 1, for all V a Label l 0 Label l 1
6
Integer Programming Formulation VaVa VbVb 2 5 4 2 0 1 1 0 2 Pairwise Potentials ab;00 = 0 ab;10 = 1 ab;01 = 1 ab;11 = 0 Sum of Pairwise Potentials ∑ (a,b) ∑ ik ab;ik y a;i y b;k y a;i {0,1} ∑ i y a;i = 1 Label l 0 Label l 1
7
Integer Programming Formulation VaVa VbVb 2 5 4 2 0 1 1 0 2 Pairwise Potentials ab;00 = 0 ab;10 = 1 ab;01 = 1 ab;11 = 0 Sum of Pairwise Potentials ∑ (a,b) ∑ ik ab;ik y ab;ik y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k Label l 0 Label l 1
8
Integer Programming Formulation min ∑ a ∑ i a;i y a;i + ∑ (a,b) ∑ ik ab;ik y ab;ik y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k
9
Integer Programming Formulation min T y y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k = [ … a;i …. ; … ab;ik ….] y = [ … y a;i …. ; … y ab;ik ….]
10
One variable, two labels y a;0 y a;1 y a;0 {0,1} y a;1 {0,1} y a;0 + y a;1 = 1 y = [ y a;0 y a;1 ] = [ a;0 a;1 ]
11
Two variables, two labels = [ a;0 a;1 b;0 b;1 ab;00 ab;01 ab;10 ab;11 ] y = [ y a;0 y a;1 y b;0 y b;1 y ab;00 y ab;01 y ab;10 y ab;11 ] y a;0 {0,1} y a;1 {0,1} y a;0 + y a;1 = 1 y b;0 {0,1} y b;1 {0,1} y b;0 + y b;1 = 1 y ab;00 = y a;0 y b;0 y ab;01 = y a;0 y b;1 y ab;10 = y a;1 y b;0 y ab;11 = y a;1 y b;1
12
In General Marginal Polytope
13
In General R (|V||L| + |E||L| 2 ) y {0,1} (|V||L| + |E||L| 2 ) Number of constraints |V||L| + |V| + |E||L| 2 y a;i {0,1} ∑ i y a;i = 1y ab;ik = y a;i y b;k
14
Integer Programming Formulation min T y y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k = [ … a;i …. ; … ab;ik ….] y = [ … y a;i …. ; … y ab;ik ….]
15
Integer Programming Formulation min T y y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k Solve to obtain MAP labelling y*
16
Integer Programming Formulation min T y y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k But we can’t solve it in general
17
Integer Programming Formulation LP Relaxation Outline
18
Linear Programming Relaxation min T y y a;i {0,1} ∑ i y a;i = 1 y ab;ik = y a;i y b;k Two reasons why we can’t solve this
19
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 y ab;ik = y a;i y b;k One reason why we can’t solve this
20
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 ∑ k y ab;ik = ∑ k y a;i y b;k One reason why we can’t solve this
21
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 One reason why we can’t solve this = 1 ∑ k y ab;ik = y a;i ∑ k y b;k
22
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 ∑ k y ab;ik = y a;i One reason why we can’t solve this
23
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 ∑ k y ab;ik = y a;i No reason why we can’t solve this * * memory requirements, time complexity
24
One variable, two labels y a;0 y a;1 y a;0 {0,1} y a;1 {0,1} y a;0 + y a;1 = 1 y = [ y a;0 y a;1 ] = [ a;0 a;1 ]
25
One variable, two labels y a;0 y a;1 y a;0 [0,1] y a;1 [0,1] y a;0 + y a;1 = 1 y = [ y a;0 y a;1 ] = [ a;0 a;1 ]
26
Two variables, two labels = [ a;0 a;1 b;0 b;1 ab;00 ab;01 ab;10 ab;11 ] y = [ y a;0 y a;1 y b;0 y b;1 y ab;00 y ab;01 y ab;10 y ab;11 ] y a;0 {0,1} y a;1 {0,1} y a;0 + y a;1 = 1 y b;0 {0,1} y b;1 {0,1} y b;0 + y b;1 = 1 y ab;00 = y a;0 y b;0 y ab;01 = y a;0 y b;1 y ab;10 = y a;1 y b;0 y ab;11 = y a;1 y b;1
27
Two variables, two labels = [ a;0 a;1 b;0 b;1 ab;00 ab;01 ab;10 ab;11 ] y = [ y a;0 y a;1 y b;0 y b;1 y ab;00 y ab;01 y ab;10 y ab;11 ] y a;0 [0,1] y a;1 [0,1] y a;0 + y a;1 = 1 y b;0 [0,1] y b;1 [0,1] y b;0 + y b;1 = 1 y ab;00 = y a;0 y b;0 y ab;01 = y a;0 y b;1 y ab;10 = y a;1 y b;0 y ab;11 = y a;1 y b;1
28
Two variables, two labels = [ a;0 a;1 b;0 b;1 ab;00 ab;01 ab;10 ab;11 ] y = [ y a;0 y a;1 y b;0 y b;1 y ab;00 y ab;01 y ab;10 y ab;11 ] y a;0 [0,1] y a;1 [0,1] y a;0 + y a;1 = 1 y b;0 [0,1] y b;1 [0,1] y b;0 + y b;1 = 1 y ab;00 + y ab;01 = y a;0 y ab;10 = y a;1 y b;0 y ab;11 = y a;1 y b;1
29
Two variables, two labels = [ a;0 a;1 b;0 b;1 ab;00 ab;01 ab;10 ab;11 ] y = [ y a;0 y a;1 y b;0 y b;1 y ab;00 y ab;01 y ab;10 y ab;11 ] y a;0 [0,1] y a;1 [0,1] y a;0 + y a;1 = 1 y b;0 [0,1] y b;1 [0,1] y b;0 + y b;1 = 1 y ab;00 + y ab;01 = y a;0 y ab;10 + y ab;11 = y a;1
30
In General Marginal Polytope Local Polytope
31
In General R (|V||L| + |E||L| 2 ) y [0,1] (|V||L| + |E||L| 2 ) Number of constraints |V||L| + |V| + |E||L|
32
Linear Programming Relaxation min T y y a;i [0,1] ∑ i y a;i = 1 ∑ k y ab;ik = y a;i No reason why we can’t solve this
33
Linear Programming Relaxation Extensively studied Optimization Schlesinger, 1976 Koster, van Hoesel and Kolen, 1998 Theory Chekuri et al, 2001Archer et al, 2004 Machine Learning Wainwright et al., 2001
34
Linear Programming Relaxation Many interesting properties Global optimal MAP for trees Wainwright et al., 2001 But we are interested in NP-hard cases Preserves solution for reparameterization Global optimal MAP for submodular energy Chekuri et al., 2001
35
Linear Programming Relaxation Large class of problems Metric Labeling Semi-metric Labeling Many interesting properties - Integrality Gap Manokaran et al., 2008 Most likely, provides best possible integrality gap
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.