1. Parallel Double Greedy Submodular Maxmization Xinghao Pan, Stefanie Jegelka, Joseph Gonzalez, Joseph Bradley, Michael I. Jordan

2. Submodular Set Functions F: 2 V → R, such that for all and Diminishing Returns Property

3. Submodular Examples clustering graph partitioning Max / Min Cut S Graph Algorithms (Krause & Guestrin 2005) Sensing (Kempe, Kleinberg, Tardos 2003, Mossel & Roch 2007) Network Analysis Document Summarization (Lin & Bilmes 2011)

4. ? Concurrency Control Double Greedy Optimal ½ - approximation Bounded overhead Coordination Free Double Greedy Bounded error Minimal overhead Submodular Maximization max F(A), A ⊆ V Monotone (increasing) functions [ Positive marginal gains ] Non-monotone functions Greedy (Nemhauser et al, 1978) (1-1/e) - approximation Optimal polytime G REE D I (Mirzasoleiman et al, 2013) (1-1/e) 2 / p – approximation 1 MapReduce round (Kumar et al, 2013) 1 / (2+ε) – approximation O(1/ε) MapReduce rounds Double Greedy (Buchbinder et al, 2012) ½ - approximation Optimal polytime Sequential Parallel / Distributed

5. Double Greedy Algorithm Set A Set B u v w x y z u v w xyz

6. Double Greedy Algorithm Set A Set B u v w x y z u v w xyz

7. Double Greedy Algorithm Set A Set B u v w x y z + A- B 01 p( | A, B ) = u rand u v w xyz Marginal gains Δ + (u|A) = F(A ∪ u) – F(A), Δ - (u|B) = F(B \ u) – F(B).

8. Double Greedy Algorithm Set A Set B u v w x y z + A- B 01 p( | A, B ) = u rand u v w xyz Marginal gains Δ + (u|A) = F(A ∪ u) – F(A), Δ - (u|B) = F(B \ u) – F(B).

9. Double Greedy Algorithm Set A Set B u w x y z + A- B 01 p( | A, B ) = v rand u v w xyz Marginal gains Δ + (v|A) = F(A ∪ v) – F(A), Δ - (v|B) = F(B \ v) – F(B). v v

10. Double Greedy Algorithm Set A Set B u w x y z u w xyz Return A

11. Parallel Double Greedy Algorithm Set A Set B u v w x y z u v w xyz CPU 1 CPU 2

12. Parallel Double Greedy Algorithm Set A Set B u v w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u | ?) = ? Δ - (u | ?) = ? Δ + (v | ?) = ? Δ - (v | ?) = ? vv ? ?

13. Concurrency Control Double Greedy Set A Set B u v w x y z u v w x y z CPU 1 CPU 2 uu v v Maintain bounds on A, B  Enable threads to make decisions locally

14. Concurrency Control Double Greedy Set A Set B w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y

15. Concurrency Control Double Greedy Set A Set B w x y z u v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u rand Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y u

16. Concurrency Control Double Greedy Set A Set B w x y z v w x y z CPU 1 CPU 2 uu Δ + (u|A) ∈ [Δ + min (u|A), Δ + max (u|A)] Δ - (u|B) ∈ [Δ - min (u|B), Δ - max (u|B) ] vv Maintain bounds on A, B  Enable threads to make decisions locally Δ + (v|A) ∈ [Δ + min (v|A), Δ + max (v|A)] Δ - (v|B) ∈ [Δ - min (v|B), Δ - max (v|B) ] + A- B 01 p( | A, B ) = u Uncertaint y + A- B 01 p( | A, B ) = v Uncertaint y rand Δ + (v|A) = F(A ∪ v) – F(A) Δ - (v|B) = F(B \ v) – F(B) v rand Common, Fast Rare, Slow

17. Properties of CC Double Greedy Theorem: CC double greedy is serializable. Corollary: CC double greedy preserves optimal approximation guarantee of ½OPT. Lemma: CC has bounded overhead. Expected number of blocked elements set cover with costs: < 2τ sparse max cut:< 2τ |E| / |V| Correctness Concurren cy Set ASet B w D E z x y x y w uu τ

18. Empirical Validation Multicore up to 16 threads Set cover, Max graph cut Real and synthetic graphs GraphVerticesEdges IT-2004Italian web-graph41 Million1.1 Billion UK-2005UK web-graph39 Million0.9 Billion Arabic-2005Arabic web-graph22 Million0.6 Billion FriendsterSocial sub-network10 Million0.6 Billion Erdos-RenyiSynthetic random20 Million2.0 Billion ZigZagSynthetic expander25 Million2.0 Billion

19. CC Double Greedy Coordination Increase in Coordination % elements failed + blocked Only 0.01% needs blocking!

20. Concurrency Ctrl. Runtime and Strong-Scaling Sequential

21. Conclusion Always fastNear optimal Coordination Free Double Greedy Usually fastAlways optimal Concurrency Control Double Greedy Always slowAlways optimal Sequential Double Greedy ScalabilityApproximation Paper @ NIPS 2014: Parallel Double Greedy Submodular Maximization.

22. BACKUP SLIDES

23. Concurrency Control Theorem: serializable. preserves optimal approximation bound ½OPT. Lemma: bounded overhead. Coordination Free Lemma: approximation bound ½OPT - error Correctness Concurren cy no overhead Correctness Concurren cy set cover with costs: sparse maxcut: from same dependencies (uncertainty region) set cover with costs: sparse maxcut: Set ASet B w D E z x y x y w uu τ

24. Adversarial Setting Processing order Overlapping covers  Increased coordination

25. Adversarial Setting – Ring Set Cover Faster Larger Error Always fast Possibly slow Always optimal Possibly wrongCoord Free Conc Ctrl