Submodular set functions Set function z on V is called submodular if For all A,B µ V: z(A)+z(B) ¸ z(A[B)+z(AÅB) Equivalent diminishing returns characterization: + ¸ + A A [ B B AÅB B + S Large improvement Submodularity: A + S Small improvement For AµB, sB, z(A [ {s}) – z(A) ¸ z(B [ {s}) – z(B)

Submodularity and convexity For V = {1,…,n}, and A µ V, let 1A = (w1A,…,wnA) with wiA = 1 if i 2 A, 0 otherwise Key result [Lovasz ’83]: Every submodular set function z induces a function g on Rn+, such that (i) z(A) = g(1A) for all A µ V (ii) g(w) is convex (iii) minA z(A) = minw g(w) s.t. w 2 [0,1]n

The submodular polyhedron Pz Pz = {x 2 Rn: x(A) · z(A) for all A µ V} Example: V = {a,b} A z(A) ; {a} -1 {b} 2 {a,b} x(A) = i2 A xi -1 x{a} x{b} 1 2 -2 x({b}) · z({b}) Pz x({a,b}) · z({a,b}) x({a}) · z({a})

Evaluating g(w) g(w) = maxx2Pz wTx -1 x{a} x{b} 1 2 xw Evaluating g(w) w g(w) = maxx2Pz wTx -2 Pz = {x 2 Rn: x(A) · z(A) for all A µ V} Theorem [Edmonds ’71, Lovasz ‘83]: For any given w, can get optimal solution xw to the LP using the following greedy algorithm: Order V={e1,…,en} so that w(e1)¸ …¸ w(en) Let xw(ei) = z({e1,…,ei}) – z({e1,…,ei-1}) Then wT xw = g(w) = maxx2 Pz wT x

Ellipsoid algorithm [Grötschel, Lovasz, Schrijver ’81] Feasible region minc,w c s.t. c ¸ wT x for all x2Pz w2 [0,1]n Separation oracle: Find most violated constraint: maxx wT x – c s.t. x 2 Pz Can solve separation using the greedy algorithm!!  Ellipsoid algorithm minimizes SFs in poly-time!

Minimizing submodular functions Ellipsoid algorithm not very practical Want combinatorial algorithm for minimization! Theorem [Iwata (2001)] There is a fully combinatorial, strongly polynomial algorithm for minimizing SFs, that runs in time O(n8 log2 n) Polynomial-time = Practical ???

A more practical alternative? [Fujishige ’91, Fujishige et al ‘06] x({a,b})=z({a,b}) -1 x{a} x{b} 1 2 -2 A z(A) ; {a} -1 {b} 2 {a,b} x* Base polytope: Bz = Pz Å {x(V) = z(V)} [-1,1] M Minimum norm algorithm: Find x* = argmin ||x||2 s.t. x 2 Bz x*=[-1,1] Return A* = {i: x*(i) < 0} A*={a} Theorem [Fujishige ’91]: A* is an optimal solution! Note: Can solve 1. using Wolfe’s algorithm Runtime finite but unknown!! 

Empirical comparison [Fujishige et al ’06] Cut functions from DIMACS Challenge Running time (seconds) Lower is better (log-scale!) Minimum norm algorithm 64 128 256 512 1024 Problem size (log-scale!) Minimum norm algorithm orders of magnitude faster! Matlab implementation can solve n = 10k in < 6 minutes!