1. Discrete Optimization Lecture 5 – Part 1 M. Pawan Kumar pawan.kumar@ecp.fr Slides available online http://mpawankumar.info

2. Submodular Functions Unary Submodular Functions Pairwise Submodular Functions Submodular Energy Function Outline

3. Submodular Function Set S Function f over power set of S f(T) + f(U) ≥ f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

4. Supermodular Function Set S Function f over power set of S f(T) + f(U) ≤ f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

5. Modular Function Set S Function f over power set of S f(T) + f(U) = f(T ∪ U) + f(T ∩ U) for all T, U ⊆ S

6. Modular Function f(T) = ∑ s ∈ T w(s) + K Is f modular? All modular functions have above form? YES Prove at home

7. Diminishing Returns Define d f (s|T) = f(T ∪ {s}) - f(T) Gain by adding s to T If f is submodular, d f (s|T) is non-increasing

8. Diminishing Returns f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪ {s,t}) for all U ⊆ S and distinct s,t ∈ S\U Necessary condition for submodularityProof? Gain by adding s to T Define d f (s|T) = f(T ∪ {s}) - f(T)

9. Diminishing Returns Sufficient condition for submodularityProof? Gain by adding s to T f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪ {s,t}) for all U ⊆ S and distinct s,t ∈ S\U Define d f (s|T) = f(T ∪ {s}) - f(T)

10. Proof Sketch Consider T, U ⊆ S We have to prove f(T) + f(U) ≥ f(T ∪ U) + f(T ∩ U) We will use mathematical induction on |TΔU|

11. Proof Sketch |TΔU| = 1 Proof follows trivially Either U ⊆ T or T ⊆ U T ∪ U = U and T ∩ U = TLet T ⊆ U

12. Proof Sketch |TΔU| = 2 If U ⊆ T or T ⊆ U, then proof follows trivially If not, then simply use the condition f(U ∪ {s}) + f(U ∪ {t}) ≥ f(U) + f(U ∪ {s,t}) for all U ⊆ S and distinct s,t ∈ S\U

13. Proof Sketch |TΔU| ≥ 3 Assume, wlog, |T \ U| ≥ 2 |T Δ ((T \{t}) ∪ U)| < |T Δ U| Let t ∈ T\U Why? f(T ∪ U) - f(T) ≤ f((T\{t}) ∪ U) - f(T\{t}) Induction assumption

14. Proof Sketch |TΔU| ≥ 3 Assume, wlog, |T \ U| ≥ 2 |(T\{t}) Δ U| < |T Δ U| Let t ∈ T\U Why? f((T\{t}) ∪ U) - f(T\{t}) ≤ f(U) - f(T ∩ U) Induction assumption

15. Proof Sketch |TΔU| ≥ 3 f(T ∪ U) - f(T) ≤ f(U) - f(T ∩ U) Hence Proved

16. Submodular Functions Unary Submodular Functions Pairwise Submodular Functions Submodular Energy Function Outline

17. Unary Submodular Function Set S = {1, 2, …, n} Function f a over power set of {a} f a (null set) = θ a (0) Consider a ∈ S f a ({a}) = θ a (1) Unary potentials When is f a submodular? Always

18. Submodular Functions Unary Submodular Functions Pairwise Submodular Functions Submodular Energy Function Outline

19. Pairwise Submodular Function Set S = {1, 2, …, n} Function f ab over power set of {a,b} f ab (null set) = θ ab (0,0) Consider a,b ∈ S f ab ({a}) = θ ab (1,0) Pairwise potentials When is f ab submodular? f ab ({b}) = θ ab (0,1)f ab ({a,b}) = θ ab (1,1)

20. Pairwise Submodular Function Set S = {1, 2, …, n} Function f ab over power set of {a,b} f ab (null set) = θ ab (0,0) Consider a,b ∈ S f ab ({a}) = θ ab (1,0) Pairwise potentials f ab ({b}) = θ ab (0,1)f ab ({a,b}) = θ ab (1,1) θ ab (0,0) + θ ab (1,1) ≤ θ ab (0,1) + θ ab (1,0)

21. Submodular Functions Unary Submodular Functions Pairwise Submodular Functions Submodular Energy Function Outline

22. Energy Function Set S = {1, 2, …, n} Energy function E(x) ∑ a θ a (x a ) Consider x ∈ {0,1} n Assume θ ab (0,0) + θ ab (1,1) ≤ θ ab (0,1) + θ ab (1,0) + ∑ (a,b) θ ab (x a,x b ) Submodular Energy Function min x E(x)