1. UBE529 Distributed Algorithms Self Stabilization

2. 2 Self-Stabilization Formalizing the notion of self-stabilization A toy problem: “Rotating Privilege on A Ring” n The very first self-stabilization algorithm n More for theoretical interest A practical problem: “Self-Stabilizing Spanning Tree Construction” n Very useful in multicast (BitTorrent-style data streaming)

3. 3 Introduction Self-stabilization: Tolerate ‘data faults’ n Example: Parent pointers in a spanning tree getting corrupted Assume that the code does not get corrupted System state: legal or illegal Faults may result in an illegal system state Self-Stabilizing system: Irrespective of the initial state always reaches a legal state in finite time

4. 4 parent value of these two nodes no longer valid Motivation for Self-Stabilization (Motivation on the book is not very practical) Distributed systems can get into illegal state due to n Topology changes n Failures / reboots n Malicious processes n Generally called “faults” A multicast tree: Each node records who its parent and children are

5. 5 Motivation for Self-Stabilization Distributed systems can get into illegal state due to n Topology changes n Failures / reboots n Malicious processes n Generally called “faults” Mobile ad hoc networks: Maintaining the shortest route back to sink A A should now have an improved route back to sink

6. 6 Defining Self-Stabilization The state (i.e., data state in all processes) of a distributed system is either legal or illegal n Definition based on application semantics n The code on each process is assumed to be correct all the time A distributed algorithm is self-stabilizing if n Starting from any (legal or illegal) state, the protocol will eventually reach a legal state if there are no more faults n Once the system is in a legal state, it will only transit to other legal states unless there are faults Intuitively, will always recover from faults and once recovered, will stay recovered forever Self-stabilizing algorithm typically runs in background and never stops

7. 7 Mutual Exclusion Legal state: Exactly one machine in the system is ‘privileged’ Assume there are N machines 0 … N-1 Each machine is a K-State machine n Label the possible states from the set {0…K-1} There is one special machine called the bottom machine L, S, R = States of left machine, self, right machine respectively

8. 8 Algorithm Bottom: Privileged if L=S Other machines: Privileged if L  S

9. 9 Algorithm: A move by bottom machine

10. 10 Algorithm: A move by a normal machine

11. 11 Another Example

12. 12 Implementation Each process needs to query its left neighbor Instead of periodic queries use a TOKEN for message efficiency What if the token gets lost ? n Bottom machine maintains a timer n If it does not receive a token for a long time it regenerates the token n Multiple tokens do not affect the correctness of the algorithm

13. 13 //Program for the bottom node public class StableBottom extends Process implements Lock { int myState = 0; int leftState = 0; int next; Timer t = new Timer(); boolean tokenSent = false; public StableBottom(Linker initComm) { super(initComm); next = (myId + 1) % N; } public synchronized void initiate() { t.schedule(new RestartTask(this), 1000, 1000); } public synchronized void requestCS() { while (leftState != myState) myWait(); } public synchronized void releaseCS() { myState = (leftState + 1) % N; } public synchronized void sendToken() { if (!tokenSent) { sendMsg(next, "token", myState); tokenSent = true; } else tokenSent = false; } public synchronized void handleMsg(Message m, int src, String tag) { if (tag.equals("token") ) { leftState = m.getMessageInt(); notify(); Util.mySleep(1000); sendMsg(next, "token", myState); tokenSent = true; } else if (tag.equals("restart") ) sendToken() }

14. 14 //Program for a normal node public class StableNormal extends Process implements Lock { int myState = 0; int leftState = 0; public StableNormal(Linker initComm) { super(initComm); } public synchronized void requestCS() { while (leftState == myState) myWait(); } public synchronized void releaseCS() { myState = leftState; sendToken(); } public synchronized void sendToken() { int next = (myId + 1) % N; sendMsg(next, "token", myState); } public synchronized void handleMsg(Message m, int src, String tag) { if (tag.equals("token")) { leftState = m.getMessageInt(); notify(); Util.mySleep(1000); sendToken(); }

15. 15 Diskstra’s 2nd Algorithm for Mutual Exclusion Bottom : if (B + 1 = R) then B := B + 2 ; Normal : if (L=S+1) or (R=S+1) then S := S+1; Top : if (L=B) and (T!=B+1) then T :=B+1; 3 states per machine {0,1,2} An array of processors

16. 16 Second Alg. Example

17. 17 Use of First Alg : The Rotating Privilege Problem A ring of n processes, each process can only communicate with neighbors There is a privilege in the system n At any time, only one node may have the privilege (you can think of this as a token) n The node with the privilege may for example, have exclusive access to some resource n The privilege needs to “rotate” among the nodes so that each node has a chance

18. 18 The Rotating Privilege Algorithm Each process i has a local integer variable V_i n 0  V_i  k where k is some constant no smaller than n 12 0 3 9 Example: n = 5 and k = 12

19. 19 12 0 3 9 Red process’s action: Retrieve value L of my clockwise neighbor; Let V be my value; if (L == V) { // I have the privilege // complete whatever I want to do; V = (V+1) % k; } Each process executes each action repeatedly – we will assume each action happens instantaneously (for this algorithm only) Green process’s action: Retrieve value L of my clockwise neighbor; Let V be my value; if (L != V) { // I have the privilege // complete whatever I want to do; V = L; }

20. 20 0 0 0 0 0 1 1 1 1 1 1 1 1 0 0 2 2 2 1 2

21. 21 What’s Interesting about the Algorithm This problem is mainly for theoretical interests What is interesting about it: n Regardless of the initial values of the processes, eventually the system will get into a legal state and stay in legal states n Self-stabilizing!

22. 22 Legal States We say that a process makes a “move” if it has the privilege and changes its value System in legal state if exactly one machine can make a move n Easy to prove that in any state, at least one machine can make move Lemma: The following are legal states and are the only legal state n All n values same OR n Only two different values forming two consecutive bands, and one band starts from the red process To prove these are the only legal states, consider the value V of the red process and the value L of its clockwise neighbor n Case I: V=L n Case II: V  L

23. 23 Legal States  Legal States Theorem: If the system is in a legal state, then it will stay in legal states n Our assumption on instantaneous actions will simplify this proof n We can consider actions one by one

24. 24 Illegal States  Legal States Lemma: Let P be a green process, and let Q be P’s clockwise neighbor. If Q makes i moves, then P can make at most i+1 move. Lemma: Let Q be the red process. If Q makes i moves, then system-wide there can be at most the following number of moves: Lemma: Let Q be the red process, and consider a sequence of n^2 moves in the system. Q makes at least one move in the sequence

25. 25 Illegal States  Legal States Lemma 1: Regardless of the starting state, the system eventually reach a state T where the red process has a different value from all other process (though the system may not stay in such states) n Proof: Let Q be the red process. If in the starting state Q has the same value as some other process, then there must be an integer j (0  j  k-1) that is not the value of any process. Q will eventually take j as its value. n (It takes Q any most n moves to do so.)

26. 26 Illegal States  Legal States Lemma 2: If the system is in a state T where the red process has a different value from all other process, then the system will eventually each a state where all processes have the same value (though the system make not stay in such states) Theorem: Regardless of the initial states of the system, the system will eventually reach a legal state. n Proof: From Lemma 1 and Lemma 2.

27. 27 Self-stabilizing Dominating Partition (Hedetniemi) R1 : if x(i) = 0  (  j E N(i)) (x(j) = 0) then x(i) = 1 R2 : if x(i) = 1  (  j E N(i)) (x(j) = 1) then x(i) = 0

28. 28 Hedetniemi Example All transformations are by R1

29. 29 Hedetniemi MIS Algorithm R1 : if s(i) = 0  (  j E N(i)) (s(j) = 0) then s(i) = 1 R2 : if s(i) = 1  (There exists j E N(i)) (s(j) = 1) then s(i) = 0

30. 30 Self Stabilizing Spanning Tree Algorithm Given n processes connected by an undirected graph and one special process P1, construct a spanning tree rooted at P1 n Not all processes can communicated with all processes directly A very useful / practical algorithm n Can also be used to compute shortest path P1

31. 31 Self Stabilizing Spanning Tree Algorithm Each process maintains two variables n parent: Who my parent is n dist: My distance to root Runs in the background n parent and dist are continuously updated At any given point of time, the values of the two variables can be wrong n Due to “faults” such as topology change resulted from node movement P1

32. 32 Self Stabilizing Spanning Tree Algorithm (3, P5) (9, P3) P1 P2 P3 P4 P5 P6 P7 P8 (8, P4) (6, P7) (5, P7) (1, P1) (2, P3) Red values are initially incorrect values; Green values are values that have become correct (0, P8) On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed)

33. 33 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (9, P3) P1 P2 P3 P4 P5 P6 P7 P8 (8, P4) (6, P7) (5, P7) (1, P1) (2, P3) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

34. 34 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (6, P7) (5, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

35. 35 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (1, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (0, P8)

36. 36 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (1, P7) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (2, P4)

37. 37 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (2, P5) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (2, P4)

38. 38 Self Stabilizing Spanning Tree Algorithm On P1 (executed periodically): n dist = 0; parent = -1; On all other processes (executed periodically): n Retrieve dist from all neighbors n Set my own dist = 1 + (the smallest dist received) n Set my own parent = my neighbor with the smallest dist (tie break if needed) (0, -1) (1, P1) P1 P2 P3 P4 P5 P6 P7 P8 (1, P1) (2, P6) (2, P5) (3, P4) (1, P1) Red values are initially incorrect values; Green values are values that have become correct (3, P4)

39. 39 Self-stabilizing spanning tree Maintain a spanning tree rooted at the ‘root’ node A data fault may corrupt the ‘parent’ pointer at any node Recalculate parent pointers regularly

40. 40 Algorithm dist maintains the distance of a node from the root

41. 41 Algorithm The root periodically sets parent to -1(null) and dist to 0 A non-root reads dist from all neighbors and points its parent to the node with the least distance from the root

42. 42 Correctness Proof Define a phase to be the minimum time period where each process has executed its code at least once (called “has taken an action”) n Some process may execute its code more than once n The definition of a phase here is different from a round in synchronous systems ! Let A_i to be the length of the shortest path from process i to the root, let dist_i to be the value of dist on process i n dist_i is not allowed to be negative

43. 43 Correctness Proof Lemma: At the end of phase 1, dist_1 = 0 and dist_i  1 for any i  2 Lemma: At the end of phase 2, n Any process i whose A_i = 0, we have dist_i = 0; n Any process i whose A_i = 1, we have dist_i = 1; n Any process i whose A_i  2, we have dist_i  2;

44. 44 Correctness Proof Lemma: At the end of phase r, n Any process i whose A_i  r-1, we have dist_i = A_i; n Any process i whose A_i  r, we have dist_i  r; Prove by induction: assume the lemma holds at phase r, now consider phase r+1, we need to prove n Any process i whose A_i  r-1, we have dist_i = A_i; n Any process i whose A_i = r, we have dist_i = A_i; n Any process i whose A_i  r+1, we have dist_i  r+1;

45. 45 Correctness Proof Consider all t actions taken during phase r+1 n We will use an induction on t This proof is tricky if this is your first self-stabilization proof n A process may take multiple actions in a phase ! n Processes may take actions in parallel – cannot assume a serialization of all actions ! The proof technique is typical for proving self-stabilization n Step 1: Prove that the t actions will not roll back what is already achieved so far (no backward move) n Step 2: Prove that at some point, each node will achieve more (forward move) n Step 3: Prove that the t actions will not roll back the effects of the forward move (no backward move after the forward move)

46. 46 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 1: The t actions will not change the green conditions Proof: Induction on t and consider action (t+1) by some process. (Cannot assume action (t+1) happens after the t actions.) Regardless of what values the process draws from its neighbors, the action will not end up violating the condition.

47. 47 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 1: The t actions will not change the green conditions satisfied at the beginning of phase r Proof (continued): True because a level A_i process only have neighbors from level A_i – 1, A_i, and A_i + 1.

48. 48 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 2: For each process, at some point during phase r+1, it will satisfy the red conditions Proof: By definition of a phase, each process will take at least one action during phase r+1

49. 49 A_i  r - 1 A_i = r A_i  r - 1 …… for nodes with already know: phase r want to show: phase r+1 A_i  r-1 dist_i = A_i A_i = r dist_i  r dist_i = A_i A_i  r+1dist_i  rdist_i  r+1 Step 3: For each process, after it first satisfies the red condition, it will continue to satisfy the red condition for the remainder of the phase Proof: Trivial – but do need to enumerate three cases

50. 50 Correctness Proof Theorem: After H rounds, A_i = dist_i on all processes n H being the length of the shortest path from the most far away process to the root Theorem: After H rounds, the dist and parent values on all processes are correct n Proof: Each process has a single parent pointer except the root. So the graph has n nodes and n-1 edges. Each process has a path to the root, thus the graph is connected.

51. 51 Acknowledgements This part is heavily dependent on the course : CS4231 Parallel and Distributed Algorithms, NUS by Dr. Haifeng Yu and Vijay Hargh Elements of Distributed Computing Book.