Submodularity Reading Group Submodular Function Minimization via Linear Programming M. Pawan Kumar

Separation and Optimization Submodular Function Minimization Outline

Solving Linear Programs s.t. A x ≤ b max x c T x Optimization Feasibility asks if there exists an x such that c T x ≥ K A x ≤ b Optimization via binary search on K Feasible solution For a given K

Feasibility via Ellipsoid Method Feasible region of LP

Feasibility via Ellipsoid Method Ellipsoid containing feasible region of LP

Feasibility via Ellipsoid Method Centroid of ellipsoid

Feasibility via Ellipsoid Method Separating hyperplane for centroid

Feasibility via Ellipsoid Method Smallest ellipsoid containing “truncated” ellipsoid

Feasibility via Ellipsoid Method Centroid of ellipsoid

Feasibility via Ellipsoid Method Separating hyperplane for centroid

Feasibility via Ellipsoid Method Smallest ellipsoid containing “truncated” ellipsoid

Feasibility via Ellipsoid Method Centroid of ellipsoid

Feasibility via Ellipsoid Method Terminate when feasible solution is found

Separating hyperplane in polynomial time –Check each of the ‘m’ LP constraints in O(n) time New ellipsoid in polynomial time –Shor (1971), Nemirovsky and Yudin (1972) Polynomial iterations (Khachiyan 1979, 1980) –Volume of ellipsoid reduces exponentially Only requires a separation oracle –Constraint matrix A can be very large Ellipsoid Method

Separation implies easy optimization What about the reverse? Polymatroids admit greedy optimization Do they allow easy separation? Why are we even interested in this? Optimization vs. Separation

Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

Polar Polyhedron Polyhedron P = {x: Ax ≤ b} Polar Polyhedron P* = {y: for all x ∈ P, x T y ≤ 1} Assume 0 is in the interior of P (P*)* = PProof? b > 0 No “loss of generality” as P can be translated

P is a subset of (P*)* If x ∈ P, then for all y ∈ P* we have x T y ≤ 1 (P*)* = {z: for all y ∈ P*, z T y ≤ 1} Therefore, x ∈ (P*)*

(P*)* is a subset of P Let there be an x ∉ P There must exist a separating hyperplane c T x > d c T z ≤ d, for all z ∈ P Since 0 ∈ interior of P, d > 0 Without loss of generality, d = 1

(P*)* is a subset of P Let there be an x ∉ P There must exist a separating hyperplane c T x > 1 c T z ≤ 1, for all z ∈ P c ∈ P* x ∉ (P*)* Why? Hence proved

Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

Optimization Problem over P Polyhedron P = {x: Ax ≤ b} max c T x x ∈ P

Separation Problem over P* Polar Polyhedron P* = {y: for all x ∈ P, x T y ≤ 1} Given y, return ‘YES’ if y ∈ P* Otherwise, return separating hyperplane

Using Optimization for Separation Set c = y max c T x x ∈ P C* = If C* ≤ 1, then return ‘YES’ If C* > 1, then return x* Optimal solution x*

Separation and Optimization –Polar Polyhedron –Separation via Optimization –Poly-time Equivalence Submodular Function Minimization Outline

Poly-Time Equivalence Optimization on P Separation on P* Polarity Optimization on P* Ellipsoid method Separation on (P*)* = P Polarity Ellipsoid method

Separation and Optimization Submodular Function Minimization Outline

Submodular Function Minimization min T ⊆ S f(T) We will assume f(null set) = 0 If not, we can add a constant

Submodular Function Minimization min T ⊆ S f(T) Brute force search is exponential in |S| We will prove that SFM is easy First, we need two properties

Property 1 f(U) = max {x(U) | x ∈ EP f } Set w = v U Proof? f is a submodular function over S

Property 2 f is a submodular function over S Define f’(U) = min T ⊆ U f(T) f’ is also submodularProof?

Proof Sketch We have to prove the following f’(T) + f’(U) ≥ f’(T ∪ U) + f’(T ∩ U) for all T, U ⊆ S f’(T) = f(X) for some X ⊆ T f’(U) = f(Y) for some Y ⊆ S

Proof Sketch f’(T) + f’(U)= f(X) + f(Y) ≥ f(X ∪ Y) + f(X ∩ Y) ≥ f’(T ∪ U) + f’(T ∩ U)

Property 2 Continued f is a submodular function over S Define f’(U) = min T ⊆ U f(T) f’ is also submodular EP f’ = { x ∈ EP f, x ≤ 0} Proof?

Proof Sketch If x ∈ P, then x(U) ≤ f(U) for all U ⊆ S x(T\U) + x(U) ≤ f(U), for all U ⊆ T ⊆ S Why? Because x ∈ EP f Why? Because x ≤ 0 We can show that { x ∈ EP f, x ≤ 0} = P ⊆ EP f’

Proof Sketch We can show that { x ∈ EP f, x ≤ 0} = P ⊇ EP f’ If x ∈ EP f’ then x ∈ EP f For any s ∈ S, x s ≤ 0 Why? Because x(U) ≤ f(U) for all U ⊆ S Why? Because x s ≤ f(null set) = 0

Submodular Function Minimization min T ⊆ S f(T) = f’(S) f’(U) = min T ⊆ U f(T) Optimization over EP f is easy = max{x(S) | x ∈ EP f’ } = max{x(S) | x ∈ EP f, x ≤ 0} Separation over EP f is easy Separation over EP f’ is easy Optimization over EP f’ is easy Hence Proved