Submodular Set Function Maximization A Mini-Survey Chandra Chekuri Univ. of Illinois, Urbana-Champaign

Submodular Set Functions A function f : 2 N  R is submodular if f(A) + f(B) ≥ f(A  B) + f(A  B) for all A,B  N Equivalently, f(A+j) – f(A) ≥ f(A+i+j) – f(A+i) for all A  N and i, j  N\A

Submodular Functions Non-negative submodular set functions f(A) ≥ 0 for all A Monotone submodular set functions f( ϕ ) = 0 and f(A) ≤ f(B) for all A  B Symmetric submodular set functions f(A) = f(N\A) for all A

Well-known Examples Cut functions in undirected graphs and hypergraphs (symmetric non-negative) Cut functions in directed graphs (non-negative) Rank functions of matroids (monotone) Coverage in set systems (monotone) many others...

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S  N f(S) S satisfies some constraints

Maximizing Submodular Set Functions Given f on a ground set N via a value oracle max S  N f(S) S satisfies some constraints Motivation: Many non-trivial applications (easy to miss!) Generalize known results for modular functions

Unconstrained Problem max S  N f(S) Uninteresting for monotone f NP-Hard for non-negative f (Max-Cut is a special case) Very hard to approximate for arbitrary f (reduction from Set Packing)

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] First O(1) approximation for non-negative f ! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41 [Vondrak])

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] First O(1) approximation for non-negative f ! Easy O(1) algorithms, ½ for symmetric case Non-trivial 2/5 = 0.4 approximation (slight improvement to 0.41 [Vondrak]) Better than ½ requires exponential # of value queries Open Problem: Close gap between 0.41 and ½

Unconstrained Problem [Feige-Mirrokni-Vondrak’07] Random set algorithm: pick each i in N with prob ½, let R be random set E[f(R)] ≥ OPT/4 E [f(R)] ≥ OPT/2 for symmetric f Simple Local Search: Initialize S to best singleton S = local optimum for adding or deleting if improvement Output better of S and N\S 1/3 approx for non-negative f and ½ for symmetric f

Local Search Analysis Lemma: If S is a local optimum then for any I  S or S  I, f(I) ≤ f(S). Proof: Say S  I and f(I) > f(S) then by submodularity there exists i in I\S s.t f(S+i) > f(S). Corollary: Let S* be an optimum solution and S be a local opt. f(S  S*) ≤ f(S) and f(S  S*) ≤ f(S)

Local Search Analysis f(S  S*) ≤ f(S) and f(S  S*) ≤ f(S) f(S  S*) + f(N\S) ≥ f(S*\S) + f(N) ≥ f(S*\S) 2f(S) + f(N\S) ≥ f(S*\S) + f(S*  S) ≥ f(S*) implies max (f(S), f(N\S)) ≥ f(S*)/3 S N\S S*

Maximizing Submodular Set Functions with Constraints max S  N f(S) S satisfies some constraints Question: what constraints? For maximization probs, packing constraints natural S  I I is a downward-closed: A  I, B  A implies B  I

Matroid and Knapsack Constraints Combinatorial packing constraints I is the intersection of some p matroids on N Lemma: every downward-close family I on N is the intersection of p matroids on N (for some p) Knapsack or matrix packing constraints A is a m x n non-negative matrix, b is m x 1 vector I = { x  {0,1} n | A x ≤ b } Combination of matroid and knapsack constraints

Matroid Constraints Uniform matroid: I = { S : |S| ≤ k } Partition matroid: I = { S : |S  N i | ≤ k i, 1 ≤ i ≤ h } where N 1,..., N h partition N, and k i are integers Laminar matroid: I = { S : |S  U| ≤ k(U), U in F } for a laminar family of sets F Graphic matroid Matroid polytope is integral and hence one can hope to capture constraints via relaxation in polytope

Monotone Functions

Cardinality Constraint max f(S) such that |S| ≤ k Max k-Cover problem is special case Greedy gives (1-1/e) approximation [Nemhauser- Wolsey-Fisher’78] Unless P=NP no better approximation [Feige’98] Many applications, routinely used

Greedy Algorithm 1.S =  2.While |S| < k do i  argmax j f S (j) S  S+i 3.Output S

Greedy Analysis S j : first j elements picked by Greedy f(S) = δ 1 + δ δ k δ j ≥ (OPT – f(S j-1 ))/k (monotonicity and submod) f(S) ≥ (1-1/k) k OPT

A Different Analysis S = {i 1, i 2,..., i k } picked by Greedy in order S* = {i* 1, i* 2,..., i* k } an optimum solution Form perfect matching between S and S* s.t elements in S  S* are matched to themselves S S*

A Different Analysis Form perfect matching between S and S* Renumber S* s.t i j is matched to i* j For each j, because Greedy chose i* j instead of i* j δ j = f(S j-1 + i j ) – f(S j-1 ) ≥ f(S j-1 + i* j ) – f(S j-1 ) ≥ f S (i* j ) Summing up f(S) ≥ Σ j f S (i* j ) ≥ f S (S*) = f(S  S*) – f(S) hence f(S) ≥ f(S  S*)/2 ≥ f(S*)/2 if f is monotone

Why weaker analysis? Let M=(N, I) be any matroid on N Solve max f(S) such that S  I [Nemhauser-Wolsey-Fisher’78] Theorem: Greedy is ½ approximation, and analysis is tight even for partition matroids Many applications, unfortunately unknown, till recently, to approximation algorithms community!

Base Exchange Theorem B and B’ are distinct bases in a matroid M=(N, I) Strong Base Exchange Theorem: There are elements i  B\B’ and i’  B’\B such that B-i+i’ and B’-i’+i are both bases. Corollary: There is a perfect matching between B\B’ and B’\B such that for each matched pair (i,i’), B-i+i’ is a base.

Proof for Greedy B is the solution that Greedy outputs B* is an optimum solution B and B* are bases of M by monotonicity of f Same argument as before works by using perfect matching between B and B*

Multiple Matroids max f(S) such that S is in intersection of p matroids [Fisher-Nemhauser-Wolsey’78] Theorem: Greedy is 1/(p+1) approximation Generalize matching argument to match one element of Greedy to p elements of OPT Also works for p-systems

Non-negative functions? max f(S) such that S is in intersection of p matroids [Lee-Mirrokni-Nagarajan-Sviridenko’08] Theorem: For fixed p, local search based algorithm that achieves Ω (1/p) approximation. [Gupta-Roth-Schoenbeck-Talwar’09] For all p and with simple proof combining Greedy and unconstrained algorithm. Slightly worse constants.

Knapsack Constraints Similar ideas can be used with standard guessing large items etc

Can we do better? ½ for one matroid, hardness is (1-1/e) 1/(p+1) for p matroids, hardness is  (p/log p)

Can we do better? [Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09] Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint.

Can we do better? [Calinescu-C-Pal-Vondrak’07]+[Vondrak’08]=[CCPV’09] Theorem: There is a randomized (1-1/e) approximation for maximizing a monotone f subject to a matroid constraint. [Lee-Sviridenko-Vondrak’09] Theorem: For fixed p≥2, there is a local-search based 1/(p+ ε ) approximation for intersection of p matroids New useful insight for (two) matroid intersection

Multilinear Extension of f Question: Is there a useful continuous relaxation of f such that it can be optimized? And can we round it effectively?

Multilinear Extension of f [CCPV’07] inspired by [Ageev-Sviridenko] For f: 2 N  R + define F:[0,1] N  R + as x = (x 1, x 2,..., x n )  [0,1] |N| F(x) = Expect[ f(x) ] =  S  N f(S) p x (S) =  S  N f(S)  i  S x i  i  N\S (1-x i )

Multilinear Extension of f For f: 2 N  R + define F:[0,1] N  R + as F(x) =  S  N f(S)  i  S x i  i  N\S (1-x i ) F is smooth submodular ([Vondrak’08])  F/  x i ≥ 0 for all i (monotonicity)  2 F/  x i  x j ≤ 0 for all i,j (submodularity)

Optimizing F(x) [Vondrak’08] Theorem: For any down-monotone polytope P  [0,1] n max F(x) s.t x  P can be optimized to within a (1- 1/e) approximation if we can do linear optimization over P Algorithm: Continuous-Greedy

Generic Approach Want to solve: max f(S) s.t S  I Relaxation: max F(x) s.t x  P(I ) P(I) is a polytope that captures/relaxes S  I Can solve to within (1-1/e) with continuous greedy How to round? F is a non-linear function

Rounding in Matroids Let M=(S,I) be a matroid P(M) is the independent set polytope of M B(M) is the base polytope of M Algorithm: Run continuous greedy to obtain a point x  B(M) such that F(x) ≥ (1-1/e) OPT Round x to a vertex of B(M) (a base)

Rounding in Matroids [CCPV’07] Theorem: Given any point x in B(M), there is a polynomial time algorithm to round x to a vertex x* (hence a base of M) such that F(x*) ≥ F(x). “Pipage” rounding technique of [Ageev-Sviridenko] adapted to matroids

New Rounding Method [C-Vondrak-Zenklusen’09] Randomized Swap-Rounding: Express x =  m j=1 β i B i (convex comb. of bases) B = B 1, β = β 1 For k = 2 to m do Randomly Merge β B and β k B k into (β + β k ) B’ Set B = B’, β = (β + β k ) Output B

Merging two Bases Merge B’ and B’’ into a random B that looks “half” like B’ and “half” like B’’

Merging two Bases B’ B’’ B’  B’’ ij Base ExchangeTheorem: B’-i+j and B’’-j+i are both bases

Merging two Bases B’ B’’ B’  B’’ ij prob ½ B’ B’’ B’  B’’ ii B’ B’’ B’  B’’ jj

Swap Rounding for Matroids Theorem: Swap-Merging with input x in B(M) outputs a random base B such that 1.E[f(B)] ≥ F(x) and 2.Pr[ f(B) < (1- δ ) F(x)] ≤ exp(- F(x) δ 2 /8) (concentration for lower tail of submod functions) and 3.For any vector a  [0,1] n, let μ = ax then Pr[a 1 B < (1- δ ) μ ] ≤ exp(- μ δ 2 /2) Pr[a 1 B > (1+ δ ) μ ] ≤ ( e δ / (1+ δ ) δ ) μ (concentration for linear functions) Almost like independent random rounding of x

Applications Can handle matroid constraint plus packing constraints x  P(M) and Ax ≤ b (1-1/e) approximation for submodular functions subject to a matroid plus O(1) knapsack/packing constraints (or many “loose” packing constraints) Simpler rounding and proof for “thin” spanning trees in ATSP application ([Asadpour etal’10])...

Other Tools On-going work [C-Vondrak-Zenklusen] Extension of continuous greedy to handle multiple submodular functions simultaneously Depending rounding via swap method for polyhedra matroid intersection matchings and b-matchings in non-bipartite graphs... Many applications of dependent randomized rounding[Arora-Frize-Kaplan, Srinivasan’01,...., Asadpour etal,..]

Illustrative Application Maximum Bipartite Flow in Networks with Adaptive Channel Width [Azar-Madry-Moscibroda- Panigrahy-Srinivasan’09] Problem motivated by capacity allocation in wireless networks (1-1/e) approximation via a specialized LP and complicated analysis An easy O(log n) approximation

Problem Definition Base stations Clients θ (B,C) : threshold capacity between B and C α (C): max flow that C desires from base stns β (B): total capacity of B to serve clients For each base station B, decide an operating point τ (B) ≤ β (B) If τ (B) > θ (B,C) then u(B,C) capacity of link (B,C) is 0, otherwise u(B,C) = τ (B) Maximize flow from base stations to clients β (B) α (C) u(B,C)

Reduction Base stations Clients β (B) α (C) t B Copies of B one for each τ (B) u(τ (B),C) N : all copies on left f : 2 N  R + where f(S) = flow from S to t Partition Matroid constraint: can pick only one copy for each B

Summary Substantial progress on submodular function maximization problems Increased awareness and more applications (New) tools and connections: continuous greedy, dependent rounding, local search,... Several open problems still remain

Open Problems Unconstrained for non-neg f. Close gap bet 0.41 and ½ max f(S) s.t S in intersection of p matroids p=1 we have 1-1/e approx which is tight unless P=NP for fixed p ≥ 2, 1/(p+ ε ) approx, otherwise 1/(p+1) hardness is Ω (p/log p) for large p when f is modular close gaps, most interesting for small p How to round F(x) for more than 1 matroid? Don’t know integrality gap for p=2! max f(x) s.t x in P(M) and Ax ≤ b, A is k-column-sparse. Want Ω (1/k) approx. Known without matroid constraint [Bansal-Korula-Nagarajan-Srinivisan’10]