1. CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS Fall 2011 Prof. Jennifer Welch CSCE 668 Set 5: Synchronous LE in Rings 1

2. Leader Election in Synchronous Rings CSCE 668Set 5: Synchronous LE in Rings 2  Here is a simple algorithm.  Group rounds into phases, each phase containing n rounds  In phase i, the processor with id i, if there is one, sends a message around the ring and is elected.

3. Example of Simple Synchronous Algorithm CSCE 668Set 5: Synchronous LE in Rings 3  n = 4, the smallest id is 7.  In phases 0 through 6 (corresponding to rounds 1 through 28), no message is ever sent.  At beginning of phase 7 (round 29), processor with id 7 sends message which is forwarded around the ring.

4. Analysis of Simple Algorithm CSCE 668Set 5: Synchronous LE in Rings 4  Correctness: Easy to see.  Message complexity: O(n), which is optimal  Time complexity: O(n*m), where m is the smallest id in the ring.  not bounded by any function of n, undesirable

5. Another Synchronous Algorithm CSCE 668Set 5: Synchronous LE in Rings 5  Works in a slightly weaker model than the previous synchronous algorithm:  processors might not all start at same round; a processor either wakes up spontaneously or when it first gets a message  uniform (does not rely on knowing n)

6. Another Synchronous Algorithm CSCE 668Set 5: Synchronous LE in Rings 6  A processor that wakes up spontaneously is active; sends its id in a fast message (one edge per round)  A processor that wakes up when receiving a msg is relay; never in the competition  A fast message carrying id m becomes slow if it reaches an active processor; starts traveling at one edge per 2 m rounds  A processor only forwards a msg whose id is smaller than any id is has previously sent  If a proc. gets its own id back, elects self

7. Analysis of Synchronous Algorithm CSCE 668Set 5: Synchronous LE in Rings 7  Correctness: convince yourself that the active processor with smallest id is elected.  Message complexity: Winner's msg is the fastest. While it traverses the ring, other msgs are slower, so they are overtaken and stopped before too many messages are sent.

8. Message Complexity CSCE 668Set 5: Synchronous LE in Rings 8  Divide msgs into three kinds: 1. fast msgs 2. slow msgs sent while the leader's msg is fast 3. slow msgs sent while the leader's msg is slow  Next, count the number of each type of msg.

9. Number of Type 1 Messages CSCE 668Set 5: Synchronous LE in Rings 9  Show that no processor forwards more than one fast msg:  Suppose p i forwards p j 's fast msg and p k 's fast msg. When p k 's fast msg arrives at p j :  either p j has already sent its fast msg, so p k 's msg becomes slow, or  p j has not already sent its fast msg, so it never will  Number of type 1 msgs is n. pkpk pjpj pipi

10. Number of Type 2 Messages CSCE 668Set 5: Synchronous LE in Rings 10 (slow sent while leader's msg is fast)  Leader's msg is fast for at most n rounds  by then it would have returned to leader  Slow msg i is forwarded n/2 i times in n rounds  Max. number of msgs is when ids are small as possible (0 to n-1 and leader is 0)  Number of type 2 msgs is at most ∑n/2 i ≤ n i=1 n-1

11. Number of Type 3 Messages CSCE 668Set 5: Synchronous LE in Rings 11 (slow msgs sent while leader's is slow)  Maximum number of rounds during which leader's msg is slow is n*2 L (L is leader's id).  No msgs are sent once leader's msg has returned to leader  Slow msg i is forwarded n*2 L /2 i times during n*2 L rounds.  Worst case is when ids are L to L + n-1  Number of type 3 msgs is at most ∑n*2 L /2 i ≤ 2n i=L L+n-1

12. Total Number of Messages CSCE 668Set 5: Synchronous LE in Rings 12  We showed  number of type 1 msgs is at most n  number of type 2 msgs is at most n  number of type 3 msgs is at most 2n  Thus total number of msgs is at most 4n = O(n).

13. Time Complexity of Synchronous Algorithm CSCE 668Set 5: Synchronous LE in Rings 13  Running time is O(n 2 x ), where x is smallest id.  Even worse than previous algorithm, which was O(n x)  Both algorithms have two potentially undesirable properties:  rely on numeric values of ids to count  number of rounds depends on minimum id, which may bear no relationship to n  Next result shows that to obtain linear msg complexity, an algorithm must rely on numeric values of the ids.

14. Comparison-Based LE Algorithms CSCE 668Set 5: Synchronous LE in Rings 14  We will show that if nodes cannot rely on the numeric values of the ids, then any synchronous LE algorithm has message complexity  (n log n).  How do we formalize "not relying on numeric values of ids"?  Need a definition of "comparison-based", similar to that for sorting algs.

15. Definition of Comparison-Based CSCE 668Set 5: Synchronous LE in Rings 15  First note that in the synchronous model, the behavior of an LE algorithm is totally determined by the distribution of the ids. Denote execution on ring R by exec(R).  An LE algorithm is comparison-based if, in any two "order-equivalent" rings R 1 and R 2, "matching" processors p i in R 1 and p j in R 2 have "similar" behaviors in exec(R 1 ) and exec(R 2 ).

16. Definition of Order-Equivalent CSCE 668Set 5: Synchronous LE in Rings 16  Rings R 1 = (x 1,x 2,…,x n ) and R 2 = (y 1,y 2,…,y n ) are order-equivalent if x i < x j if and only if y i < y j  Example: x1x1 x5x5 x4x4 x3x3 x2x2 y1y1 y2y2 y3y3 y4y4 y5y5

17. Definition of Matching Processors CSCE 668Set 5: Synchronous LE in Rings 17  Processors p i in R 1 and p j in R 2 are matching if they are the same distance from the processor with minimum id.  Example: p 0 (with id 18) in R 1 and p 1 (with id 5) in R 2, which are 2 hops from p 3 (resp., p 4 )

18. Definition of Similar Behaviors CSCE 668Set 5: Synchronous LE in Rings 18  Two processors have similar behaviors in two executions if, in every round  one processor sends a message to the left (right) if and only if the other one does content of message is irrelevant  one processor is elected if and only if the other one is

19. Synchronous Lower Bound CSCE 668Set 5: Synchronous LE in Rings 19 Theorem (3.18): For every n ≥ 8 that is a power of 2, there is a ring of size n on which any synchronous comparison-based algorithm sends  (n log n) msgs. Proof: For each n, construct a highly symmetric ring S n on which every comparison-based algorithm takes lots of messages.  Symmetry means many processors have order- equivalent neighborhoods and thus behave similarly (including sending msgs)

20. Proof Strategy CSCE 668Set 5: Synchronous LE in Rings 20  Define highly symmetric ring S n  Show that the number of "active" rounds when any c-b alg runs on S n is at least n/8 (L. 3.20)  Show that in the k-th active round, at least n/(2(2k+1)) msgs are sent (L. 3.21)  Do some arithmetic to show that ∑ n/(2(2k+1)) =  (n log n) k=1 n/8

21. Active Rounds CSCE 668Set 5: Synchronous LE in Rings 21  A round is active if at least one processor sends a msg  A proc. in a synchronous algorithm can potentially learn something even in an inactive round  cf. the simple synchronous alg  But in a comparison-based alg., a proc. can't learn about the order pattern of its ring in an inactive round (intuition)  Note that the k-th active round might be much later than the k-th round

22. A Highly Symmetric Ring S n CSCE 668Set 5: Synchronous LE in Rings 22  Let p i 's id be rev(i), the integer whose binary representation using log n bits is the reverse of i 's binary rep. Example:  For technical reasons, then multiply id by n+1 and add n p3p3 p2p2 p1p1 p0p0 2 3 0 1 4 199 14

23. Why Lots of Active Rounds CSCE 668Set 5: Synchronous LE in Rings 23 L. 3.20: Number of active rounds, T, in exec(S n ) is at least n/8. Proof: Suppose T < n/8. Let p i be the elected leader.  Number of T-neighborhoods order-equivalent to p i 's is at least n/(2(2T+1)) (L. 3.19, to be shown)  Since n ≥ 8 and T < n/8, algebra shows n/(2(2T+1)) > 1.  So there is p j  p i whose T-neighborhood is order- equivalent to p i 's.  Then p j is also elected (L. 3.17, to be shown), contradiction.

24. Why Enough Msgs Sent in Each Active Round CSCE 668Set 5: Synchronous LE in Rings 24 L. 3.21: At least n/(2(2k + 1)) msgs are sent in kth active round. Proof: Since the round is active, at least one proc., say p i, sends a msg.  There are at least n/(2(2k+1)) procs. whose k- neighborhood is order-equivalent to p i 's (L. 3.19, to be shown)  Each of those procs. sends a msg in kth active round, since p i does (L. 3.17, to be shown).

25. Why So Many O-E Neighborhoods CSCE 668Set 5: Synchronous LE in Rings 25 L. 3.19: Every k-nbrhood of S n has at least n/(2(2k+1)) order-equivalent k-nbrhoods. Proof Sketch: Let N be a k-nbrhood (sequence of 2k+1 ids in the ring).  Let j be smallest power of 2 larger than 2k+1.  Break S n into n/j segments of length j, with one segment encompassing N.  Claim: Each segment is order-equivalent to N.

26. Why Similar Behavior in Order-Equivalent Neighborhoods CSCE 668Set 5: Synchronous LE in Rings 26  Intuition is that two nodes in order-equivalent neighborhoods in the same ring will behave similarly, at least until differentiating information from beyond those neighborhoods has reached them.  Yet the definition of comparison-based algorithm only requires similar behavior for matching processors in different (albeit order-equivalent) rings.

27. Order-Equivalent Means Similar CSCE 668Set 5: Synchronous LE in Rings 27 L. 3.17: If p i and p j have order-equivalent k-nbrhoods, then p i and p j have similar behaviors through k-th active round. Proof Sketch: Construct another ring S n ' such that  p i in S n behaves same as p j in S n '  p j in S n ' behaves similarly to p j in S n

28. Order-Equivalent Means Similar CSCE 668Set 5: Synchronous LE in Rings 28  In more detail, construct S n ' such that  p i 's k-nbrhood in S n equals p j 's in S n ' so behavior is same through k-th active round  S n ' and S n are order-equivalent  p j in S n ' is matching to p j in S n so behavior is similar through k-th active round  Can be done since ids in S n are spaced (multiplied by n+1, plus n)  So p i and p j have similar behavior in S n through k- th active round

29. Anonymous Rings Revisited CSCE 668Set 5: Synchronous LE in Rings 29  Leader election is impossible in anonymous rings  No way to break symmetry  No deterministic algorithm which works in every execution  Another way to break symmetry, which works in some (but not all) executions is to use randomization.

30. Randomized Algorithm CSCE 668Set 5: Synchronous LE in Rings 30  In each computation step, the processor receives an additional input to its state transition function, a random number.

31. Revised LE Problem Definition CSCE 668Set 5: Synchronous LE in Rings 31  Weakened problem definition compared to original:  At most one leader is elected in every state of every admissible execution  same as previous definition  At least one leader is elected "with high probability".  weaker than previous definition  But what does "with high probability" mean?

32. Randomized LE Algorithm CSCE 668Set 5: Synchronous LE in Rings 32  Assume synchronous model  Initially:  set id to 1 with probability 1 - 1/n and to 2 with probability 1/n  send id to left  When msg M is received:  if M contains n ids then if id is unique maximum in M then elect self else not elected  else append id to M and send to left

33. Analysis of Randomized LE Alg. CSCE 668Set 5: Synchronous LE in Rings 33  Uses O(n 2 ) msgs  There is never more than one leader  Sometimes there is no leader  leader is only elected if there is exactly one processor that sets its id to 2  How often is there no leader, i.e., what is the probability?  Need some more definitions…

34. Random Choices and Probabilities CSCE 668Set 5: Synchronous LE in Rings 34  Since system is synchronous, an admissible execution of the algorithm is determined solely by the initial random choices.  Call this collection random choices RC = where each r i is either 1 or 2.  Let exec(RC) be the resulting execution.  Definition: For any predicate P on executions, Pr[P] is the probability of {RC : exec(RC) satisfies P} I.e., the proportion of random choices resulting in an execution that satisfies P.

35. Probability of Electing a Leader CSCE 668Set 5: Synchronous LE in Rings 35  Let P be the predicate "there is a leader".  Pr[P] = probability RC contains exactly one 2

36. Improving the Probability of Electing a Leader CSCE 668Set 5: Synchronous LE in Rings 36  If procs. notice there is no leader, they can try again.  Each iteration of the basic algorithm is a phase.  Keep trying until success.  Random choices that define an execution consist of an infinite sequence of 1's and 2's, one for each proc.  It is possible that algorithm doesn't terminate.

37. Probability of Not Terminating CSCE 668Set 5: Synchronous LE in Rings 37  Probability of terminating in a particular phase is (1 - 1/n) n-1  Probability of not terminating in a particular phase is 1 - (1 - 1/n) n-1  Probability of not terminating in k phases is (1 - (1 - 1/n) n-1 ) k since phases are independent.  Last expression goes to 0 as k increases.

38. Expected Number of Phases CSCE 668Set 5: Synchronous LE in Rings 38  Definition: Expected value of a random variable T is E[T] =  k Pr[T = k]  Let T be number of phases until termination.  Pr[T = k] = Pr[first k-1 phases fail & kth succeeds] = (1 - (1 - 1/n) n-1 ) k-1 (1 - 1/n) n-1 = (1 - p) k-1 p, where p = (1 - 1/n) n-1  This is a geometric random variable with expected value p -1 < e.  So expected number of phases is < 3. k