Submodularity Reading Group Polymatroid M. Pawan Kumar http://www.robots.ox.ac.uk/~oval/

Submodular Function Ground set S Function f over power set of S f(T) + f(U) ≥ f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

Diminishing Returns Define df(s|T) = f(T ∪{s}) - f(T) Gain by adding s to T If f is submodular, df (s|T) is non-increasing df (s|T) ≥ df(s|U), for all T ⊆ U

A Clarification on Notation Vector: x (in bold font) An element of the vector: xi (non-bold x) Consider ground set S = {1, 2, …, n} x(U) where U ⊆ S: ∑i∈U xi

Polymatroid Set S Submodular function f Real vector x of size |S|x1 Pf = {x ≥ 0, x(U) ≤ f(U) for all U ⊆ S} Polymatroid EPf = {x(U) ≤ f(U) for all U ⊆ S} Extended Polymatroid

Primal Problem max wTx x ∈ EPf Assume f(null set) ≥ 0 Otherwise EPf is empty f(null set) can be set to 0 Why? Decreasing f(null set) maintains submodularity

Primal Problem max wTx x ∈ EPf Assume w ≥ 0 Otherwise the optimal solution is infinity Why?

Greedy Algorithm max wTx x ∈ EPf Order s1,s2,…,sn ∈ S such that w(si) ≥ w(si+1) Define Ui = {s1,s2,..,si} xGi = f(Ui) – f(Ui-1) xG ∈ EPf Proof?

Proof Sketch We have to show that xG(A) ≤ f(A) for all A ⊆ S Trivial when A = null set Mathematical induction on |A|

Proof Sketch Let k be the largest index such that sk ∈ A Clearly |A| ≤ |Uk| xG(A) = xG(A\{sk}) + xGk ≤ f(A\{sk}) + xGk Induction Why? = f(A\{sk}) + f(Uk) - f(Uk-1) Definition Why? ≤ f(A) Submodularity Why?

Greedy Algorithm max wTx x ∈ EPf Order s1,s2,…,sn ∈ S such that w(si) ≥ w(si+1) Define Ui = {s1,s2,..,si} xGi = f(Ui) – f(Ui-1) xG is optimal Proof?

Dual Problem min ∑A yA f(A) max wTx yA ≥ 0, for all A ⊆ S x ∈ EPf ∑A yAvA = w Let us first try to find a feasible dual solution

Greedy Algorithm Order s1,s2,…,sn ∈ S such that w(si) ≥ w(si+1) Define Ui = {s1,s2,..,si} yGUi = w(si) - w(si+1) yG is feasible yGS = w(sn) Proof? yGA = 0, for all other A

Proof Sketch Trivially, yG ≥ 0 Consider si ∈ S ∑A∋si yGA = ∑j≥i yGUj = w(si) ∑A yAvA = w

Optimality Primal feasible solution xG Dual feasible solution yG Primal value at xG = Dual value at yG Proof?

Proof Sketch wTxG = ∑s∈S w(s)xGs = ∑i∈{1,2,…,n} w(si)(f(Ui) - f(Ui-1)) = ∑i∈{1,2,…,n-1} f(Ui)(w(si) - w(si+1)) + f(S)w(sn) = ∑A yGA f(A)

Optimality Primal feasible solution xG Dual feasible solution yG Primal value at xG = Dual value at yG Therefore, xG is an optimal primal solution And, yG is an optimal dual solution