Distributed Verification and Hardness of Distributed Approximation Atish Das Sarma Stephan Holzer Danupon Nanongkai Gopal Pandurangan David Peleg 1 Weizmann Google Research Liah Kor Roger Wattenhofer ETH Zurich U. of Vienna & Georgia Tech Nanyang Technological University & Brown University ETH ZurichWeizmann Amos Korman U. Paris 7

PLAN 2 Result summary Techniques Overview From communication complexity to distributed algo. lower bound

Distributed network 3

Distributed network Distributed network A graph G of n nodes, diameter D 4 n= 4, D=

Main issue: LOCALITY and BANDWIDTH 5 ?

Time complexity = number of rounds log n

Example: Spanning tree in O(D) time

Weighted distributed network 8 ?

Fundamental problems Spanning Tree Spanning Tree – Broadcasting, Aggregation, etc Minimum Spanning Tree Minimum Spanning Tree – Efficient broadcasting, leader election, etc. Shortest path Shortest path – Routing, etc. Steiner tree Steiner tree – Multicasting, etc. Many other graph problems. 9

How fast can we compute distributively? 10

Three points of this work 1. A new generic technique to prove lower bounds of distributed algorithms. – Works for approximation algorithms. – Connection to communication complexity 2. New bounds for many problems. Tight in some cases. 3. A systematic study of distributed verification. 11

12 Distributed algorithms for the above problems require   (n 1/2 +D) time Distributed algorithms for the above problems require   (n 1/2 +D) time

Two main ingredients 1.Verification  Approximation 2.Connection to communication complexity. 13

14 Showcase Minimum Spanning Tree

Time of Distributed Algorithms ProblemsUpper boundLower bound Spanning tree (ST) O(D)  (D) MSTO(D + n  )  (D + n 1/2 )  -approx. MST  (D + (n /  ) 1/2 ) MST VerificationO(D + n  )  (D + n 1/2 ) 15 [trivial] [Garay, Kutten, Peleg FOCS’93][Peleg, Rubinovich FOCS’99] [Elkin STOC’04] [Kor, Korman, Peleg STACS’11]

Time of Distributed Algorithms ProblemsUpper boundLower bound Spanning tree (ST) O(D)  (D) MSTO(D + n  )  (D + n 1/2 )  -approx. MST  (D + (n /  ) 1/2 ) MST VerificationO(D + n  )  (D + n 1/2 ) ST VerificationO(D + n  ) 16 [trivial] [Garay, Kutten, Peleg FOCS’93][Peleg, Rubinovich FOCS’99] [Elkin STOC’04] [Kor, Korman, Peleg STACS’11]

17 Implication of our results

Time of Distributed Algorithms ProblemsUpper boundLower bound Spanning tree (ST) O(D)  (D) MSTO(D + n  )  (D + n 1/2 )  -approx. MST  (D + (n /  ) 1/2 ) MST VerificationO(D + n  )  (D + n 1/2 ) ST VerificationO(D + n  ) 18 [trivial] [Garay, Kutten, Peleg FOCS’93][Peleg, Rubinovich FOCS’99] [Elkin STOC’04] [Kor, Korman, Peleg STACS’11]  (D + n 1/2 )

Previous lower bound proofs Deterministic : Count the number of states. Argue that the number is not enough. Randomized: Come up with a good input distributions. 19 Our proof Simple reduction from communication complexity. Avoid complication in proving randomized lower bounds.

PLAN 20 Result summary Techniques Overview From communication complexity to distributed algo. lower bound

21 Approx MST lower bound  (n 1/2 ) Distributed equality verification lower bound  (n 1/2 ) Distributed equality verification lower bound  (n 1/2 ) ST verification lower bound  (n 1/2 ) Distributed equality verification lower bound  (n 1/2 ) Distributed equality verification lower bound  (n 1/2 ) Direct equality verification lower bound  (n 1/2 ) Direct equality verification lower bound  (n 1/2 ) Well-known result in communication complexity Similar to hardness of TSP Similar to lower bounds of graph streaming algorithms Three steps of reduction Distributed Algorithms Communication Complexity simulation theorem simulation theorem

PLAN 22 Result summary Techniques Overview From communication complexity to distributed algo. lower bound

Communication complexity of EQUALITY 23

How many bits do they have to exchange? 24 Alice Bob x  {0, 1} 100 y  {0, 1} 100 x=y? Yes, x=y

25 One solution: Alice sends everything... time=100 Alice Bob x  {0, 1} 100 y  {0, 1} 100 x=y?

26 Theorem: Any algorithm needs ≥100 bits Alice Bob x  {0, 1} 100 y  {0, 1} 100 x=y?

Distributed time complexity of EQUALITY 27

28 Alice x  {0, 1} 100 Bob y  {0, 1} green nodes Alice and Bob are connected by many paths of length 100 ∞

29 Alice x  {0, 1} 100 Bob y  {0, 1} green nodes In each step, one edge can carry one bit on each direction ∞

How many steps do they need to check whether “x=y”? 30

31 Alice Bob 100 green nodes A: 100 steps because the network diameter is 100

Let’s make the diameter smaller 32

33 Alice Bob 100 green nodes 10 green nodes Now the diameter is 30 How many steps do we need?

Claim: Need > 50 steps. 34

Proof: Assume there is a distributed algorithm A that uses ≤ 50 steps 35

36 AliceBob x  {0, 1} 100 y  {0, 1} bits x=yx=yx=yx=y Contradiction

Proof: Assume there is a distributed algorithm A that uses ≤ 50 steps 37 Goal: Show that Alice & Bob can use A to compute EQUALITY using 50 bits

38 Alice x  {0, 1} 100 Bob y  {0, 1} 100 x=yx=y x=yx=y ? ? ? ?

39 AliceBob x  {0, 1} 100 y  {0, 1} 100 Alice’s network Bob’s network Run A

40 AliceBob x  {0, 1} 100 y  {0, 1} 100 x y ? ? ? ? Alice’s network Bob’s network 0 Step Run A

41 In step 0, Alice can run A on all machines except Bob’s

42 AliceBob x y ? ? ? ? 1 Step

43 AliceBob x y ? ? ? ? 1 Step

44 AliceBob x y ? ? ? ? 1 Step ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? b1b1b1b1 a1a1a1a1 b 1 = b 1 = bit sent by A run on Bob’s machine

45 AliceBob x y ? ? ? ? 1 Step ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? b1b1b1b1 a1a1a1a1 a1a1a1a1 b1b1b1b1 b 1 = b 1 = bit sent by A from Bob’s machine keep this

46 AliceBob x y ? ? ? ? 2 Step ? ? ? ? ? ? ? ? ? ? ? ? b2b2b2b2 a2a2a2a2 a2a2a2a2 b2b2b2b2 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? b 2 = b 2 = bit sent by A from Bob’s machine

47 AliceBob x y ? ? ? ? 3 Step ? ? ? ? ? ? ? ? ? ? ? ? b3b3b3b3 a3a3a3a3 a3a3a3a3 b3b3b3b3 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? b 3 = b 3 = bit sent by A from Bob’s machine

48 AliceBob x y ? ? ? ? 4 Step ? ? ? ? ? ? ? ? ? ? ? ? b4b4b4b4 a4a4a4a4 a4a4a4a4 b4b4b4b4 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

49 AliceBob x y ? ? ? ? 5 Step ? ? ? ? ? ? ? ? ? ? ? ? b5b5b5b5 a5a5a5a5 a5a5a5a5 b5b5b5b5 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?

50 AliceBob x y ? ? ? ? Step ? ? ? ? ? ? ? ? ? ? ? ? b 50 a 50 b 50 ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? A finishes x=y

51 AliceBob x  {0, 1} 100 y  {0, 1} bits x=yx=yx=yx=y Contradiction

Remarks 52

53 1. By replacing 100 by n 1/2, we can reduce distributed EQUALITY to ST verification x=y?Do red edges form a spanning tree?

2. Reduce diameter... 54

55 Alice Bob n 1/2 n 1/2 paths n 1/2 n 1/2 green nodes n 1/4 n 1/4 orange nodes n 1/4 n 1/4 green nodes Diameter = n 1/4

56 Alice Bob Diameter = log n n 1/2 n 1/2 paths n 1/2 n 1/2 green nodes

3. Getting randomized lower bound EQAULITY does not give randomized lower bound. Simulation theorem holds for all functions. Reduce from communication complexity of HAMILTONIAN CYCLE [Spieker, Raz FOCS’93] 57

Recap 1. A new generic technique to prove lower bounds of distributed algorithms. – Works for approximation algorithms. 2. New bounds for many problems. Tight in some cases. 3. A systematic study of distributed verification. 58

Open problems Tight bounds of shortest paths, mincut, minimum routing cost spanning tree, Steiner forest,... Lower bounds of algorithms on complete graphs? Complexity theory of distributed computing? 59

Thank you! Related talk at PODC Thank you! Related talk at PODC Today 5:10pm “A tight unconditional lower bound on distributed random walk computation” 60