Chapter 2 Self-Stabilization

Chapter 2 Self-Stabilization
Self-Stabilization Shlomi Dolev MIT Press , 2000 2-1 1

Chapter 2: roadmap 2.1 Definitions of Computational Model
2.2 Self-Stabilization Requirements 2.3 Complexity Measures 2.4 Randomized Self-Stabilization 2.5 Example: Spanning-Tree Construction 2.6 Example: Mutual Exclusion 2.7 Fair Composition of Self-Stabilization Algorithms 2.8 Recomputation of Floating Output 2.9 Proof Techniques 2.10 Pseudo-Self-Stabilization Chapter 2 - Definitions, Techniques and Paradigms 2-2 2

What is a Distributed System?
Communication networks Multiprocessor computers Multitasking single processor A Distributed System is modeled by a set of n state machines called processors that communicate with each other Chapter 2 - Definitions, Techniques and Paradigms 2-3

The Distributed System Model
Denote : Pi - the ith processor neighbor of Pi - a processor that can communicate with it Node i = Processor i Pi Pj Link Pi<->Pj = Pi can communicate with Pj How to Represent the Model? Ways of communication message passing - fits communication networks and all the rest shared memory - fits geographically close systems Chapter 2 - Definitions, Techniques and Paradigms 2-4 4

Asynchronous Distributed Systems – Message passing
A communication link which is unidirectional from Pi to Pj transfers message from Pi to Pj For a unidirectional link we will use the abstract qij (a FIFO queue) receive m2 P1 P2 P3 q13 = () q32 = () q21 = (m2,m10) P1 P2 P3 q13 = () q32 = (m1) q21 = (m10) P1 P2 P3 q13 = () q32 = (m1) q21 = (m2, m10) send m1 Chapter 2 - Definitions, Techniques and Paradigms 2-5 5

Asynchronous Distributed Systems - Message passing
System configuration (configuration) : Description of a distributed system at a particular time. A configuration will be denoted by c = (s1,s2,…,sn,q1,2,q1,3,…,qi,j,…,qn,n-1) , where si =State of Pi qi,j (ij) the message queue P1 P2 P3 q13 = () q32 = () q21 = (m2,m10) m1 Chapter 2 - Definitions, Techniques and Paradigms 2-6 6

Asynchronous Distributed Systems – Shared Memory
P1 P2 r12 Processors communicate by the use of shared communication registers The configuration will be denoted by c = (s1,s2,…,sn,r1,2,r1,3,…ri,j,…rn,n-1) where si = State of Pi ri= Content of communication register i Chapter 2 - Definitions, Techniques and Paradigms 2-7

The distributed System – A Computation Step
In shared memory model … P1 P2 m1 r12: x P1 P2 r12: m1 P1 writes P1 P2 r12: m1 P2 reads m1 And in message passing model … loss21(m3) P1 P2 q12 = (m1) q21 = (m2,m3,m4) P1 P2 q12 = (m1) q21 = (m2,m4) P1 receives P1 P2 q12 = (m1) q21 = (m4) m2 Chapter 2 - Definitions, Techniques and Paradigms 2-8

The Interleaving model
Scheduling of events in a distributed system influences the transition made by the processors The interleaving model - at each given time only a single processor executes a computation step Every state transition of a process is due to communication-step execution A step will be denoted by a c1 a c2 denotes the fact that c2 can be reached from c1 by a single step a Chapter 2 - Definitions, Techniques and Paradigms 2-9

The distributed System – more definitions
Step a is applicable to configuration c iff  c’ : c a c’ . In the message passing system - queue qi,j contains m in ck, and in ck+1 m is removed from qi,j and placed in Pt and this is the only difference between ck& ck+1 An execution E = (c1,a1,c2,a2,…) , an alternating sequence such that ci-1 a  ci (i>1) A fair execution - every step that is applicable infinitely often is executed infinitely often In the message passing model - a message which is sent infinitely often will be received infinitely often – number of losses is finite Chapter 2 - Definitions, Techniques and Paradigms 2-10

Synchronous Distributed Systems
A global clock pulse (pulse) triggers a simultaneous step of every processor in the system Fits multiprocessor systems in which the processors are located close to one another The execution of a synchronous system E = (c1,c2,…) is totally defined by c1, the first configuration in E 1 4 3 2 q12 = (m12) q14 = (m14) q21 = (m21) q31 = (m31) q32 = (m32) intermediate state send receive 1 4 3 2 q12 = () q14 = () q21 = () q31 = () q32 = () 1 4 3 2 q12 = () q14 = () q21 = () q31 = () q32 = () * This models message passing, shared memory is analogous with write -> read Chapter 2 - Definitions, Techniques and Paradigms 2-11

Legal Behavior A desired legal behavior is a set of legal executions denoted LE ctsafe c’safe c’’safe step legal execution step ci c’ c’’ cm cl c2safe c1safe cksafe c * A desired legal behavior is a set of legal executions denoted LE *Defined for a particular system and a particular task Should have a suffix that appears in LE *A configuration c is safe with regard to task LE and an algorithm if every fair execution of the algorithm that starts from c belongs to LE *An algorithm is self-stabilizing for a task LE if every fair execution of the algorithm reaches a safe configuration with relation to LE A self-stabilizing system can be started in any arbitrary configuration and will eventually exhibit a desired “legal” behavior Chapter 2 - Definitions, Techniques and Paradigms 2-12 12

Time complexity The first asynchronous round (round) in an execution E is the shortest prefix E’ of E such that each processor executes at least one step in E’, E=E’E’’. The number of rounds = time complexity A Self-Stabilizing algorithm is usually a do forever loop The number of steps required to execute a single iteration of such a loop is O(), where  is an upper bound on the number of neighbors of Pi Asynchronous cycle (cycle) the first cycle in an execution E is the shortest prefix E’ of E such that each processor executes at least one complete iteration of it’s do forever loop in E’, E=E’E’’. Note : each cycle spans O() rounds The time complexity of synchronous algorithm is the number of pulses in the execution mkly Chapter 2 - Definitions, Techniques and Paradigms 2-13 13

Space complexity The space complexity of an algorithm is the total number of (local and shared) memory bits used to implement the algorithm Chapter 2 - Definitions, Techniques and Paradigms 2-14 14

Randomized Self-Stabilization
Assumptions and definitions Processor activity is managed by a scheduler The scheduler’s assumption - at most one step is executed in every given time The scheduler is regarded as an adversary The scheduler is assumed to have unlimited resources and chooses the next activated processor on-line A scheduler S is fair if, for any configuration c with probability 1, an execution starting from c in which processors activated by S is fair Chapter 2 - Definitions, Techniques and Paradigms 2-15

Randomized Self-Stabilization - Assumptions and definitions ..
An algorithm is randomized self-stabilizing for task LE if, starting with any system configuration and considering any fair scheduler, the algorithm reaches a safe configuration within a finite number of rounds Randomized algorithms are often used to break symmetry in a system of totally identical processors Chapter 2 - Definitions, Techniques and Paradigms 2-16

2.2 Self-Stabilization Requirements 2.3 Complexity Measures 2.4 Randomized Self-Stabilization 2.5 Example: Spanning-Tree Construction 2.6 Example: Mutual Exclusion 2.7 Fair Composition of Self-Stabilization Algorithms 2.8 Recomputation of Floating Output 2.9 Proof Techniques 2.10 Pseudo-Self-Stabilization Chapter 2 - Definitions, Techniques and Paradigms 2-17

Spanning-Tree Construction
1 3 2 The root writes 0 to all it’s neighbors The rest – each processor chooses the minimal distance of it’s neighbors, adds one and updates it’s neighbors Chapter 2 - Definitions, Techniques and Paradigms 2-18

Spanning-Tree Algorithm for Pi
01 Root: do forever for m := 1 to  do write rim := 0,0 03 od 04 Other: do forever 05 for m := 1 to  do write lrmi := read(rmi) 06 FirstFound := false 07 dist := 1 + minlrmi.dis 1  m    08 for m := 1 to  09 do 10 if not FirstFound and lrmi.dis = dist -1 11 write rim := 1,dist FirstFound := true 13 else 14 write rim := 0,dist 15 od 16 od = # of processor’s neighbors i= the writing processor m= for whom the data is written lrji (local register ji) the last value of rji read by Pi Chapter 2 - Definitions, Techniques and Paradigms 2-19

Spanning-Tree, System and code
Demonstrates the use of our definitions and requirements The system – We will use the shared memory model for this example The system consists n processors A processor Pi communicates with its neighbor Pj by writing in the communication register rij and reading from rji Chapter 2 - Definitions, Techniques and Paradigms 2-20

Spanning-Tree, System and code
The output tree is encoded by means of the registers 1 4 5 8 6 2 3 7 r21: parent = 1 dis = 1 r58: parent = 8 dis = 3 r73: parent = 2 The output tree is encoded by means of the registers as follows: * Each register rij contains binary parent field, if Pj is the parent of Pi in the BFS tree then the value is 1 * In addition each register rij has a distance field (dis) that holds the distance from the root to Pi Chapter 2 - Definitions, Techniques and Paradigms 2-21 21

Spanning-Tree Application
RUN Chapter 2 - Definitions, Techniques and Paradigms 2-22

Spanning-Tree, is Self-Stabilizing
The legal task ST - every configuration encodes a BFS tree of the communication graph Definitions : A floating distance in configuration c is a value in rij.dis that is smaller than the distance of Pi from the root The smallest floating distance in configuration c is the smallest value among the floating distance  is the maximum number of links adjacent to a processor * A configuration is a vector of the processors states and a vector of communication registers values The task ST of legitimate sequences is defined as the set of all configuration sequences in which every configuration encodes a BFS tree of the communication graph Definitions : *A floating distance in some configuration c is a value in a register rij.dis that is smaller than the distance of Pi from the root *The smallest floating distance in some configuration c is the smallest value among the floating distance * is the maximum number of links adjacent to a processor Chapter 2 - Definitions, Techniques and Paradigms 2-23 23

Spanning-Tree, is Self-Stabilizing
For every k > 0 and for every configuration that follows  + 4k rounds, it holds that: (Lemma 2.1) If there exists a floating distance, then the value of the smallest floating distance is at least k The value in the registers of every processor that is within distance k from the root is equal to its distance from the root Note that once a value in the register of every processor is equal to it’s distance from the root, a processor Pi chooses its parent to be the parent in the first BFS tree, this implies that : The algorithm presented is Self-Stabilizing for ST Chapter 2 - Definitions, Techniques and Paradigms 2-24

Mutual Exclusion The root changes it’s state if equal to it’s neighbor
The rest – each processor copies it’s neighbor’s state if it is different Chapter 2 - Definitions, Techniques and Paradigms 2-25

Dijkstra’s Algorithm 01 P1: do forever 02 if x1=xn then
x1:=(x1+1)mod(n+1) 04 Pi(i  1): do forever if xi  xi-1 then xi:=xi-1 Chapter 2 - Definitions, Techniques and Paradigms 2-26

Mutual-Exclusion Application

Dijsktra’s alg. is Self-Stabilizing
A configuration of the system is a vector of n integer values (the processors in the system) The task ME : exactly one processor can change its state in any configuration, every processor can change its state in infinitely many configurations in every sequence in ME A safe configuration in ME and Dijkstra’s algorithm is a configuration in which all the x variables have the same value * The task ME is defined by the set of all configuration sequences in which exactly one processor can change its state in any configuration and every processor can change its state in infinitely many configurations in every sequence in ME Chapter 2 - Definitions, Techniques and Paradigms 2-28 28

Dijkstra’s alg. is Self-Stabilizing
A configuration in which all x variables are equal, is a safe configuration for ME (Lemma 2.2) For every configuration there exists at least one integer j such that for every i xi j (Lemma 2.3) For every configuration c, in every fair execution that starts in c, P1 changes the value of x1 at least once in every n rounds (Lemma 2.4) For every possible configuration c, every fair execution that starts in c reaches a safe configuration with relation to ME within O(n2) rounds (Theorem 2.1) * A configuration c in which all the x variables have the same value is a safe configuration for ME and Dijsktra’s algorithm (Lemma 2.2) * For every possible configuration c, there exists at least one integer jn such that for every i xi j in c (Lemma 2.3) * For every possible configuration c, in every fair execution that starts in c, the special processor P1 changes the value of x1 ate least once in every n rounds (Lemma 2.4) Chapter 2 - Definitions, Techniques and Paradigms 2-29 29

Fair Composition -Some definitions
The idea – composing self-stabilizing algorithms AL1,...,ALk so that the stabilized behavior of AL1,AL2,...,ALi is used by ALi+1 ALi+1 cannot detect whether the algorithms have stabilized, but it is executed as if they have done so The technique is described for k=2 : Two simple algorithms server & client are combined to obtain a more complex algorithm The server algorithm ensures that some properties will hold to be used by the client algorithm The technique is described for k=2 : * Two simple algorithms, called a server algorithm and a client algorithm, are combined to obtain a more complex algorithm * The server algorithm ensures that some properties will hold, and these properties are later used by the client algorithm Chapter 2 - Definitions, Techniques and Paradigms 2-30 30

Fair Composition -more definitions
Assume the server algorithm AL1 is for a task defined by a set of legal execution T1, and the client algorithm AL2 for T2 Let Ai be the state set of Pi in AL1 and Si= AiBi the state set of Pi in AL2 ,where whenever Pi executes AL2 , it modifies only the Bi components of AiBi For a configuration c  S1 … Sn , define the A-projection of c as the configuration (ap1, … , apn)  A1 … An The A-projection of an execution - consist of the A-projection of every configuration of the execution * Assume server algorithm AL1 is for a task defined by a set of legal execution T1, and that the client algorithm AL2 is for a task defined by a set of legal execution T2 * Let Ai be the state set of Pi in AL1 and Si= AiBi the state set of Pi in AL2 ,where whenever Pi executes AL2 , it modifies only the Bi components of AiBi * For a configuration c  S1 … Sn , define the A-projection of c as the configuration (ap1, … , apn)  A1 … An * The A-projection of an execution is defined analogously to consist of the A-projection of every configuration of the execution Chapter 2 - Definitions, Techniques and Paradigms 2-31 31

Fair Composition -more definitions …
AL2 is self-stabilizing for task T2 given task T1 if any fair execution of AL2 that has an A-projection in T1 has a suffix in T2 AL is a fair composition of AL1 and AL2 if, in AL, every processor execute steps of AL1 and AL2 alternately Assume AL2 is self-stabilizing for task T2 given task T1. If AL1 is self-stabilizing for T1, then the fair composition of AL1 and AL2 is self-stabilizing for T2 (Theorem 2.2) * We say that algorithm AL2 is self-stabilizing for task T2 given task T1 if any fair execution of AL2 that has an A-projection in T1 has a suffix in T2 * An algorithm AL is a fair composition of AL1 and AL2 if, in AL, every processor execute steps of AL1 and AL2 alternately * Note: for an execution E at AL, the A-projection of E is a sub-execution of E corresponding to a fair execution of the server algorithm AL1 * Theorem (2.2) : Assume that AL2 is self-stabilizing for task T2 given task T1. If AL1 is self- stabilizing for T1, then the fair composition of AL1 and AL2 is self-stabilizing for T2 Chapter 2 - Definitions, Techniques and Paradigms 2-32 32

Example : Mutual exclusion for general communication graphs
Will demonstrate the power of fair composition method What will we do ? Compose the spanning-tree construction algorithm with the mutual exclusion algorithm Server 1 3 2 Client The server algorithm is the spanning-tree construction The client algorithm is the mutual exclusion Chapter 2 - Definitions, Techniques and Paradigms 2-33 33

Modified version of the mutual exclusion algorithm
Designed to stabilize in a system in which a rooted spanning tree exists and in which only read/write atomicity is assumed Euler tour defines a virtual ring P1 lr4,1 lr2,1 lr3,1 r1,4 r1,3 r1,2 r4,1 r2,1 r3,1 P4 P2 P3 lr1,4 lr1,2 lr1,3 Chapter 2 - Definitions, Techniques and Paradigms 2-34

Mutual exclusion for tree structure (for Pi)
01 Root: do forever 02 lr1,i := read (r1,i) 03 if lr,i = ri,1 then 04 (* critical section*) 05 write ri,2 := (lr1,i + 1 mod (4n -5)) 06 for m := 2 to  do 07 lrm,i = read(rm,i ) 08 write ri,m+1 := lrm,i 09 od 10 od 11 Other: do forever 12 lr1,i := read (r1,i) 13 if lr1,i  ri,2 14 (* critical section*) 15 write ri,2 := lr1,i 16 for m := 2 to  do 17 lrm,i := read (rm,i) 18 write ri,m+1 := lrm,i 19 od 20 od Chapter 2 - Definitions, Techniques and Paradigms 2-35

mutual exclusion for communication graphs
Mutual-exclusion can be applied to a spanning tree of the system using the ring defined by the Euler tour Note – a parent field in the spanning-tree defines the parent of each processor in the tree When the server reaches the stabilizing state, the mutual-exclusion algorithm is in an arbitrary state, from there converges to reach the safe configuration Let i = i1, i2, … , i be the arbitrary ordering of the tree edges incident to a non-root node vi V, where the first edge is the edge leading to the parent of vi in the tree *…(page 25) he mutual-exclusion algorithm can be applied to a spanning tree of the system using the ring defined by the Euler tour on the tree * Note - the value of the parent fields of the server algorithm (spanning-tree) eventually defines the parent of each processor in the tree * Until the server algorithm reaches the stabilized state the clients execution may not be as considered during the design of the algorithm. Consequently, once the tree is fixed, the self- stabilizing mutual-exclusion algorithm is in an arbitrary state from which it converges to reach a safe configuration Chapter 2 - Definitions, Techniques and Paradigms 2-36 36

Recomputation of Floating Output -The Idea
OK Output BAD c1 c2 ci ck BAD BAD floating output variables What do we need to prove ? * The recomputation of the floating output is a way to convert a non-stabilizing algorithm AL that computes a fixed distributed output into a self-stabilizing algorithm (the idea is to execute AL repeatedly) * Each time the execution of AL is over, the output is written in special output variables called floating output variables What do we need to prove ? * From every possible configuration, an execution of AL will eventually begin from a predefined initial state * Any two executions of AL that begin from the initial state have identical output From every possible configuration, an execution of AL will eventually begin from a predefined initial state Any two executions of AL that begin from the initial state have identical output Chapter 2 - Definitions, Techniques and Paradigms 2-38 38

Recomputation of Floating Output Example: Synchronous Consensus
Demonstrates the core idea behind the recomputation of the floating output Assumptions We will ignore synchronization issues We will assume that the system is a message- passing synchronous system Define – The synchronous consensus task - a set of (synchronous) executions SC in which the output variable of every processor is 1 iff there exists an input value with value 1 Chapter 2 - Definitions, Techniques and Paradigms 2-39

Non-stabilizing synchronous consensus algorithm
This algorithm is not a self stabilizing algorithm 01 initialization 02 pulse := 0 03 Oi = Ii 04 while pulsei  D do 05 upon a pulse 06 pulsei := pulsei + 1 07 send (Oi) 08 forall Pj  N(i) do receive (Oj) 09 if Oi = 0 and  Pj  N(i) | Oj = 1 then 10 Oi := 1 1 1 1 1  i : Ii = 0 Chapter 2 - Definitions, Techniques and Paradigms 2-40

Self-stabilizing synchronous consensus algorithm
01 upon a pulse 02 if (pulsei mod (D +1)) = 0 then 03 begin 04 Oi = oi (*floating output is assigned by the result*) 05 oi = Ii (*initializing a new computation*) 06 end 07 send (oi , pulsei) 08 forall Pj  N(i) do receive (oj , pulsej) 09 if oi = 0 and  Pj  N(i) | oj = 1 then 10 oi := 1 11 pulsei := max {{pulsei } {pulsej | Pj  N(i)}} +1 Chapter 2 - Definitions, Techniques and Paradigms 2-41

Self-stabilizing synchronous consensus algorithm
The output variable Oi of every processor Pi is used as a floating output, which is eventually fixed and correct The variable oi of every processor Pi is used for recomputing the output of the algorithm  i : Oi = 1 1  i : Oi = 0 1 1 1 1  i : Ii = 0 Chapter 2 - Definitions, Techniques and Paradigms 2-42

Variant Function Used for proving convergence
step c c1 c2 c3 csafe steps |VF(c)|  |VF(c1)|  |VF(c2)|  |VF(c3)|  …  |VF(csafe)|  …  bound Used for proving convergence Can be used to estimate the number of steps required to reach a safe configuration The idea : Use a function over a configuration set whose value is bounded Prove that this function monotonically decreases/ increases when processors execute a step Show after the function reaches a certain threshold, the system is in a safe configuration Chapter 2 - Definitions, Techniques and Paradigms 2-44 44

Variant Function - Example: self stabilizing Maximal Matching
Every processor Pi tries to find a matching neighbor Pj Program for Pi : 01 do forever 02 if pointeri = null and ( Pj  N(i) | pointerj = i ) then 03 pointeri = j 04 if pointeri = null and ( Pj  N(i) | pointerj  i ) and 05 ( Pj  N(i) | pointerj = null ) then 06 pointeri = j 07 if pointeri = j and pointerj = k and k  i then 08 pointeri = null 09 od Chapter 2 - Definitions, Techniques and Paradigms 2-45

Maximal Matching Application

Variant Function - Example: self stabilizing Maximal Matching
The algorithm should reach a configuration in which pointeri = j implies that pointerj =i We will assume the existence of a central daemon The set of legal executions MM for the maximal matching task includes every execution in which the values of the pointers of all the processors are fixed and form a maximal matching * The maximal matching algorithm should reach a configuration cl in which the existence of a pointer of Pi that points to Pj implies the existence of a pointer of Pj that points to Pi * To simplify the discussion, let us assume the existence of a central daemon that activates one processor at a time * The set of legal executions MM for the maximal matching task includes every execution in which the values of the pointers of all the processors are fixed and form a maximal matching Chapter 2 - Definitions, Techniques and Paradigms 2-47 47

Variant Function - Example: self stabilizing Maximal Matching- definitions
Program for Pi : 01 do forever 02 if pointeri = null and ( Pj  N(i) | pointerj = i ) then 03 pointeri = j 04 if pointeri = null and ( Pj  N(i) | pointerj  i ) and 05 ( Pj  N(i) | pointerj = null ) then 06 pointeri = j 07 if pointeri = j and pointerj = k and k  i then 08 pointeri = null 09 od matched free waiting Given a configuration cl, we say that a processor Pi is: matched in cl, if Pi has a neighbor Pj such that pointeri = j and pointerj = i single in cl, if pointeri = null and every neighbor of Pi is matched waiting in cl, if Pi has a neighbor such that pointeri = j and pointerj = null free in cl, if pointeri = null and there exists a neighbor Pj, such that Pj is not matched chaining in cl, if there exists a neighbor Pj for which pointeri = j and pointerj = k , k  i chaining single Chapter 2 - Definitions, Techniques and Paradigms 2-48 48

Variant Function - Example: self stabilizing Maximal Matching- proving correctness
The variant function VF(c) returns a vector (m+s,w,f,c) m - matched, s – single, w – waiting, f – free, c - chaining Values of VF are compared lexicographically VF(c) = (n,0,0,0)  c is a safe configuration with relation to MM and to our algorithm Once a system reaches a safe configuration, no processor changes the value of its pointer Note that: for every configuration cl for which VF(c) = (n,0,0,0) is a safe configuration with relation to MM and to our algorithm for every safe configuration VF(c) = (n,0,0,0) Once a system reaches the safe configuration, no processor changes the value of its pointer In every non-safe configuration, there exists at least one processor that can change the value of its pointer when it is activated by the central daemon Chapter 2 - Definitions, Techniques and Paradigms 2-49 49

Variant Function - Example: self stabilizing Maximal Matching- proving correctness
In every non-safe configuration, there exists at least one processor that can change the value of its pointer Every change of a pointer-value increases the value of VF The number of such pointer-value changes is bounded by the number of all possible vector values. The first three elements of the vector (m+s,w,f,c) imply the value of c, thus there at most O(n3) changes. The fact that m+s+w+f+c = n implies that the number of possible vector values is O(n3) The value of n and the first three elements of the vector (m+s, w, f, c) imply the value of c . Therefore the system reaches a safe configuration within O(n3) pointer-value changes Chapter 2 - Definitions, Techniques and Paradigms 2-50 50

Convergence Stairs A1 A2 ..... Ak Ai – predicate
c1c2…cj A1 cj+1…cl A2 cl+1…ci ..... cmcm+1 Ak Definitions : Ai+1 refines predicate Ai if Ai holds whenever Ai+1 holds The term attractor is often used for such Ai predicate The idea : prove that the self-stabilizing algorithm converges to fulfill k >1 predicates A1,A2, … ,Ak such that, for every 1  i  k, Ai+1 is a refinement of Ai The “stairs” : prove that, from some point of the execution, every configuration satisfies Ai then proving that an execution in which Ai holds reaches a configuration after which every configuration satisfies Ai+1 until Ak which is the predicate for a safe configuration csafe Ai – predicate for every 1  i  k, Ai+1 is a refinement of Ai Chapter 2 - Definitions, Techniques and Paradigms 2-51 51

Convergence Stairs - Example: Leader election in a General Communication Network
Program for Pi, each processor reads it’s neighbors leader and chooses the candidate with the lowest value : 01 do forever 02 candidate,distance =  ID(i), 0  03 forall Pj  N(i) do begin 05 leaderi[j],disi[j] := read leaderj, disj  06 if (disi[j]  N) and ((leaderi[j]  candidate) or ((leaderi[j] = candidate) and (disi[j]  distance))) then candidate,distance :=  leaderi[j],disi[j] + 1 end 10 write leaderi ,disi := candidate,distance 11 od Program for Pi , each processor reads its neighbors leader and chooses the canidate with the lowest value: Chapter 2 - Definitions, Techniques and Paradigms 2-52 52

Convergence Stairs - Example: Leader election in a General Communication Networks
We assume that every processor has a unique identifier in the range 1 to N The leader election task is to inform every processor of the identifier of a single processor in the system, this single processor is the leader Floating identifier - an identifier that appears in the initial configuration, when no processor in the system with this identifier appears in the system * We assume that every processor has a unique identifier in the range 1 to N , where N is the upper bound of the number of processors in the system * The leader election task is to inform every processor of the identifier of a single processor in the system, this single processor with the elected identifier is the leader * The term floating identifier is used to describe an identifier that appears in the initial configuration, when no processor in the system with this identifier appears in the system (we use distance variables and the upper bound N to eliminate floating identifiers) Chapter 2 - Definitions, Techniques and Paradigms 2-53 53

Convergence Stairs - Example: Leader election, proving correctness
We will use 2 convergence stairs : A1 - no floating identifiers exists A2 (for a safe configuration) every processor chooses the minimal identifier of a processor in the system as the identifier of the leader To show that A1 holds, we argue that, if a floating identifier exists, then during O() rounds, the minimal distance of a floating identifier increases * We will use 2 convergence stairs : First - predicate A1 on system configurations verifying that no floating identifiers exists Second - predicate A2 for a safe configuration, a predicate that verifies that every processor chooses the minimal identifier of a processor in the system as the identifier of the leader * To show that A1 holds, we argue that, if a floating identifier exists, then during O() rounds, the minimal distance of a floating identifier increases Chapter 2 - Definitions, Techniques and Paradigms 2-54 54

Convergence Stairs - Example: Leader election, proving correctness ...
After the first stair only the correct ids exist, so the minimal can be chosen From that point, every fair execution that starts from any arbitrary configuration reaches the safe configuration Notice : if A1 wasn’t true, we couldn’t prove the correctness Notice : if A1 wasn’t true, we couldn’t prove the correctness, it is possible that the minimal identifier z - which is the first (arbitrary) configuration of the system - is not an identifier of a processor in the system Chapter 2 - Definitions, Techniques and Paradigms 2-55 55

Scheduler-Luck Game Purpose : proving the upper bounds of the time complexity of randomized distributed algorithms by probability calculations Using the sl-game method . It tries to avoid considering every possible outcome of the randomized function used by the randomized algorithm Chapter 2 - Definitions, Techniques and Paradigms 2-56

The sl-game Given a randomized algorithm AL, we define a game between 2 players, scheduler and luck scheduler ● Given a randomized algorithm AL, we define a game between 2 players, scheduler and luck The goal of the scheduler is to prevent the algorithm AL from fulfilling its task, it can choose the activation interleaving of the processors luck may determine the result of the randomized function invoked The rules … In each turn of the game, the scheduler chooses the next processor to be activated, which then makes a step. If, during this step,the activated processor uses a random function, then luck may intervene luck Chapter 2 - Definitions, Techniques and Paradigms 2-57 57

The sl-game(2) Luck’s strategy is used to show that the algorithm stabilizes Some definitions : cp = fi=1pi f is # of lucks intervenes pi - probability of the ith intervention luck should have a (cp,r)-strategy to win the game If luck has a (cp,r)-strategy,then AL reaches a safe configuration within, at most, r/cp expected number of rounds (Theorem 2.4) Some definitions : cp = fi=1pi where f is the number of times that luck intervenes and pi is the probability of the ith intervention luck has a (cp,r)-strategy to win the game if it has a strategy to reach a safe configuration in the game in an expected number of at most r rounds, and with interventions that yield a combined probability of no more than cp We can get to the conclusion that (Theorem 2.4): If luck has an (cp,r)-startegy,then AL reaches a safe configuration within, at most, r/cp expected number of rounds Chapter 2 - Definitions, Techniques and Paradigms 2-58 58

SL-Game, Example: Self Stabilizing Leader election in Complete Graphs
The next algorithm works in complete graph systems, communication via shared memory Program for Pi : 01 do forever 02 forall Pj  N(i) do 03 li[j] = read( leaderj ) 04 if (leaderj = 0 and { j  i|li[j] = 0}) or (leaderi = 1 and { j  i|li[j] = 1}) then write leaderi := random({0,1}) 07 end The next algorithm works in complete graph systems in which every processor can communicate with every other processor via shared memory Chapter 2 - Definitions, Techniques and Paradigms 2-59 59

Random Leader election in Complete Graphs

SL-Game, Example: Self Stabilizing Leader election in Complete Graphs
Task LE - the set of executions in which there exists a single fixed leader throughout the execution A configuration is safe if it satisfies the following: for exactly one processor, say Pi, leaderi = 1 and  j  i li[j] = 0 for every other processor Pj  Pi leaderj = 0 and lj[i] = 1 In any fair execution E that starts with a safe configuration, Pi is a single leader and thus E  LE Task LE to be the set of executions in which there exists a single fixed leader throughout the execution A configuration is safe if it satisfies the following: for exactly one processor, say Pi, leaderi = 1 and  j  i li[j] = 0 for every other processor Pj  Pi leaderj = 0 and lj[i] = 1 In any fair execution E that starts with a safe configuration, Pi is a single, leader and thus E  LE Chapter 2 - Definitions, Techniques and Paradigms 2-61 61

SL-Game, Example: Self Stabilizing Leader election in Complete Graphs - proof
The algorithm stabilizes within 2O(n) expected number of rounds (Lemma 2.6) using theorem 2.4 we will show that the number of round is expected to be 2n2n we present an (1/2n, 2n)-strategy for luck to win the sl-game Luck’s strategy is as follows: whenever some processor Pi tosses a coin, luck intervenes; if for all j  i, leaderj = 0, then luck fixes the coin toss to be 1; otherwise, it fixes the coin toss to be 0 Lemma 2.6 : The algorithm stabilizes within 2O(n) expected number of rounds * using theorem 2.4 we will show that the number of round is expected to be 2n2n * we present an (1/2n, 2n)-strategy for luck to win the sl-game * Luck’s strategy is as follows: whenever some processor Pi tosses a coin, luck intervenes; if for all j  i, leaderj = 0, then luck fixes the coin toss to be 1; otherwise, it fixes the coin toss to be 0 Chapter 2 - Definitions, Techniques and Paradigms 2-62 62

SL-Game, Example: Self Stabilizing Leader election in Complete Graphs - proof...
The algorithm is not self-stabilizing under fine- atomicity (Lemma 2.7) By presenting a winning strategy for the scheduler that ensures that the system never stabilizes Starting with all leader registers holding 1, the scheduler will persistently activate each processor until it chooses 0 and before it writes down. Then it lets them write. Analogously, scheduler forces to choose them 1. Lemma 2.7 : The algorithm is not self-stabilizing under fine-atomicity (in which a coin-toss is separate atomic step) . * By presenting a winning strategy for the scheduler (that guarantees that the obtained schedule is a fair schedule with probability 1) that ensures that the system never stabilizes Chapter 2 - Definitions, Techniques and Paradigms 2-63 63

Neighborhood Resemblance
Purpose : proving memory lower bounds silent self-stabilizing algorithm - self-stabilizing algorithm in which the communication between the processors is fixed from some point of the execution The technique can be applied to silent self- stabilizing algorithms The idea : using a configuration c0 (safe configuration) and construct a non-safe configuration c1 in which every processor has the same neighborhood and therefore cannot distinguish c0 from c1 * Purpose : proving memory lower bounds - proving that it is impossible to achieve certain tasks with less than a certain amount of memory * We define silent self-stabilizing algorithm as a self-stabilizing algorithm if the communication between the processors is fixed from some point of the execution * The technique can be applied on silence self- stabilizing algorithms * The idea : The neighborhood-resemblance technique uses a configuration c0 that is claimed to be safe configuration and constructs a non-safe configuration c1 in which every processor has the same neighborhood and therefor cannot distinguish c0 from c1 Chapter 2 - Definitions, Techniques and Paradigms 2-64 64

Reminder … Spanning-Tree Algorithm for Pi
01 Root: do forever 02 for m := 1 to  do write rim := 0,0 03 od 04 Other: do forever 05 for m := 1 to  do write lrmi := read(rmi) 06 FirstFound := false 07 dist := 1 + minlrmi.dis 1  m    08 for m := 1 to  09 do 10 if not FirstFound and lrmi.dis = dist -1 11 write rim := dist,1 FirstFound := true 13 else 14 write rim := 0,dist 15 od 16 od 1 3 2 Chapter 2 - Definitions, Techniques and Paradigms 2-65

Neighborhood Resemblance - Example: Spanning-Tree Construction
We will show that implementing the distance field requires (log d) bits in every communication register, where d is the diameter of the system Note - the Spanning-Tree Construction is indeed a silent self-stabilizing algorithm * We will show that implementing the distance field requires (log d) bits in every communication register, where d is the diameter of the system * Note - the Spanning-Tree Construction is indeed a silent self-stabilizing algorithm (once the system stabilizes, the content of each communication register is fixed ) Chapter 2 - Definitions, Techniques and Paradigms 2-66 66

Neighborhood Resemblance, Example: Spanning Tree Construction
ak bk Non-tree with special processor Graph P2 P1 ai Q1 bi ei Q2 ej ak ek bk Tree ak ek bk ai bi Non-tree without special processor Chapter 2 - Definitions, Techniques and Paradigms 2-67

What is Pseudo-Self-Stabilization ?
step c2safe c1safe cksafe c ci c’ c’’ cm cl ctsafe c’safe c’’safe ci c’’safe clsafe * The Concept : Pseudo self-stabilizing algorithms converge from any initial state to execution, in which they exhibit a legal behavior; but they still may deviate from this legal behavior a finite number of times The algorithm exhibits a legal behavior; but may deviate from this legal behavior a finite number of times Chapter 2 - Definitions, Techniques and Paradigms 2-69 69

An Abstract Task An abstract task - variables and restrictions on their values The token passing abstract task AT for a system of 2 processors; Sender (S) and Receiver (R). S and R have boolean variable tokenS and tokenR Given E = (c1,a1,c2,a2,…) one may consider only the values of tokenS and tokenR in every configuration ci to check whether the token-passing task is achieved * An abstract task is defined by a set of variables and a set of restrictions on their values * Let us define the token passing abstract task AT for a system of 2 processors; Sender (S) and Receiver (R). S and R have boolean variable tokenS and tokenR respectively * An algorithm is pseudo-self-stabilizing for an abstract task AT if every infinite execution of the algorithm has a suffix satisfying the restriction of AT Chapter 2 - Definitions, Techniques and Paradigms 2-70 70

Pseudo-self-Stabilization
Denote : ci|tkns the value of the boolean variables (tokenS , tokenR ) in ci E|tkns as (c1|tkns, c2|tkns, c3|tkns, …) We can define AT by E|tkns as follows: there is no ci|tkns for which tokenS=tokenR=true It is impossible to define a safe configuration in terms of ci|tkns, since we ignore the state variables of R/S Given E = (c1,a1,c2,a2,…) one may consider only the values of tokenS and tokenR in every configuration ci to check whether the token-passing task is achieved Denote : ci|tkns the value of the boolean variables (tokenS , tokenR ) in ci E|tkns as (c1|tkns, c2|tkns, c3|tkns, …) We can define AT by E|tkns as follows: there is no ci|tkns for which tokenS=tokenR=true Note that it is impossible to define a safe configuration in terms of ci|tkns, since we ignore the state variables of R/S Chapter 2 - Definitions, Techniques and Paradigms 2-71 71

Pseudo-Self-Stabilization
legal tokenR = true tokenS = false tokenR = false tokenS = true The system is not in a safe configuration, and there is no time bound for reaching a safe configuration Only after a message is lost does the system reach the safe configuration Illegal tokenR = true tokenS = true legal Only after a message is lost does the system reach the lower cycle in the figure, and a safe configuration tokenR = true tokenS = false tokenR = false tokenS = true Chapter 2 - Definitions, Techniques and Paradigms 2-72 72

Pseudo-Self-Stabilization, The Alternating Bit Algorithm
A data link algorithm used for message transfer over a communication link Messages can be lost, since the common communication link is unreliable The algorithm uses retransmission of messages to cope with message loss frame distinguishes the higher level messages, from the messages that are actually sent (between S and R) The term frame distinguish the higher level messages that must be transferred, from the messages that are actually sent (between S and R) in order to transfer the higher-level messages Chapter 2 - Definitions, Techniques and Paradigms 2-73 73

Pseudo-Self-Stabilization, The Data Link Algorithm
The task of delivering a message is sophisticated, and may cause message corruption or even loss The layers involved Physical Layer Data link Layer Tail Packet Frame Network Layer Head The task of delivering a message from one processor in the network to another remote processor is sophisticated, and may cause message corruption or even loss There are several layers involved Physical Layer Data Link Layer (which concerns us) Network Layer Chapter 2 - Definitions, Techniques and Paradigms 2-74 74

Pseudo-Self-Stabilization, The Data Link Algorithm
. m3 m2 m1 S R . m3 m2 m1 fetch The flow of a message: S R . m3 m2 m1 f1 send S R . m3 m2 m1 f1 receive The flow of a message from the senders queue to the receiver : fetch -> insert to frame and send -> receive frame -> deliver to and send notification to the sender S R . m3 m2 m1 deliver S R . m3 m2 m1 f2 send Chapter 2 - Definitions, Techniques and Paradigms 2-75 75

Pseudo-Self-Stabilization, Back to The Alternating Bit Algorithm
The abstract task of the algorithm: S has an infinite queue of input messages (im1,im2,…) that should be transferred to the receiver in the same order without duplications, reordering or omissions. R has an output queue of messages (om1,om2,…). The sequence of messages in the output queue should always be the prefix of the sequence of messages in the input queue The abstract task of the algorithm: S has an infinite queue of input messages (im1,im2,…) that should be transferred to the receiver in the same order without duplications, reordering or omissions. R has an output queue of messages (om1,om2,…). The sequence of messages in the output queue should always be the prefix of the sequence of messages in the input queue Chapter 2 - Definitions, Techniques and Paradigms 2-76 76

The alternating bit algorithm - Sender
01 initialization 02 begin i := 1 bits := 0 send(bits,imi) (*imi is fetched*) 06 end (*end initialization*) 07 upon a timeout 08 send(bits,imi) 09 upon frame arrival 10 begin 11 receive(FrameBit) 12 if FrameBit = bits then (*acknowledge arrives*) begin 14 bits := (bits + 1) mod 2 15 i := i + 1 16 end 17 send(bits,imi) (*imi is fetched*) 18 end Chapter 2 - Definitions, Techniques and Paradigms 2-77

The alternating bit algorithm - Receiver
01 initialization 02 begin j := 1 bitr := 1 05 end (*end initialization*) 06 upon frame arrival 07 begin 08 receive(FrameBit , msg) 09 if FrameBit  bitr then (*a new message arrived*) begin 11 bitr := FrameBit 12 j := j + 1 13 omj := msg (*omj is delivered*) 14 end 15 send(bitr) 16 end Chapter 2 - Definitions, Techniques and Paradigms 2-78

Denote L = bits,qs,r,bitr,qr,s , the value of the of this label sequence is in [0*1*] or [1*0*] where qs,r and qr,s are the queue messages in transit on the link from S to R and from R to S respectively We say that a single border between the labels of value 0 and the labels of value 1 slides from the sender to the receiver and back to the sender Once a safe configuration is reached, there is at most one border in L, where a border is two consecutive but different labels Chapter 2 - Definitions, Techniques and Paradigms 2-79

<m3 ,0>..<m2 ,1> .. <m1 ,0>
The Alternating Bit Algorithm, borders sample Suppose we have two borders If frame m2 gets lost, receiver will have no knowledge about it X S R <m3 ,0>..<m2 ,1> .. <m1 ,0> bitS = 0 bitR = 1 <m3 ,0>…<m1 ,0> Chapter 2 - Definitions, Techniques and Paradigms 2-80

Denote L(ci) - the sequence L of the configuration ci A loaded configuration ci is a configuration in which the first and last values in L(ci) are equal Chapter 2 - Definitions, Techniques and Paradigms 2-81

The algorithm is pseudo self-stabilizing for the data-link task, guaranteeing that the number of messages that are lost during the infinite execution is bounded, and the performance between any such two losses is according to the abstract task of the data-link Chapter 2 - Definitions, Techniques and Paradigms 2-82 82

Chapter 3: Motivating Self-Stabilization
Converging to a desired behavior from any initial state enables the algorithm to converge from an arbitrary state caused by faults Why should one have interest in self- stabilizing algorithms? Its applicability to distributed systems Recovering from faults of a space shuttle. Faults may cause malfunction for a while. Using a self- stabilizing algorithm for its control will cause an automatically recovery, and enables the shuttle continue in its task Chapter 3 - Motivating Self-Stabilization 3-84 84

What is a Self-Stabilizing Algorithm ?
This question will be answered using the “Stabilizing Orchestra” example The Problem: The conductor is unable to participate – harmony is achieved by players listening to their neighbor players Windy evening – the wind can turn some pages in the score, and the players may not notice the change Chapter 3 - Motivating Self-Stabilization 3-85

The “Stabilizing Orchestra” Example
Our Goal: To guarantee that harmony is achieved at some point following the last undesired page turn Imagine that the drummer notices a different page of the violin next to him … (solutions and their problems): The drummer turns to its neighbors new page – what if the violin player noticed the difference as well ? Both the drummer and violin player start from the beginning - what if the player next to the violin player notices the change only after sync between the other 2 ? Chapter 3 - Motivating Self-Stabilization 3-86

The “Stabilizing Orchestra” Example – the Self-Stabilizing Solution
Every player will join the neighboring player who is playing the earliest page (including himself) Note that the score has a bounded length. What happens if a player goes to the first page of the score before harmony is achieved? This case is discussed in details in chapter 6. In every long enough period in which the wind does not turn a page, the orchestra resumes playing in synchrony Chapter 3 - Motivating Self-Stabilization 3-87

Chapter 3: roadmap 3.1 Initialization of a Data-Link Algorithm in the Presence of Faults 3.2 Arbitrary Configuration Because of Crashes 3.3 Frequently Asked Questions Chapter 3 - Motivating Self-Stabilization 3-88

The sender sends sequences of bits to the receiver
The Data Link Algorithm The task of delivering a message is sophisticated, and may cause message corruption or even loss Physical Layer Data link Layer Tail Packet Frame Network Layer Head The layers involved: Physical Layer Data link Layer Tail Packet Frame Network Layer Head The layers involved: Tail Packet Frame Head The sender sends sequences of bits to the receiver The task of delivering a message from one processor in the network to another remote processor is sophisticated, and may cause message corruption or even loss There are several layers involved Physical Layer Data Link Layer (which concerns us) Network Layer Chapter 3 - Motivating Self-Stabilization 3-89 89

The alternating-bit algorithm
Is used to cope with possibility of frame corruption or loss 01 initialization 02 begin i := 1 bits := 0 send(bits,imi) (*imi is fetched*) 06 end (*end initialization*) 07 upon a timeout 08 send(bits,imi) 09 upon frame arrival 10 begin 11 receive(FrameBit) 12 if FrameBit = bits then begin 14 bits := (bits + 1) mod 2 15 i := i + 1 16 end 17 send(bits,imi) (*imi is fetched*) 18 end Sender j := 1 bitr := 1 05 end (*end initialization*) 06 upon frame arrival 07 begin 08 receive(FrameBit , msg) 09 if FrameBit  bitr then begin 11 bitr := FrameBit 12 j := j + 1 13 omj := msg 14 end 15 send(bitr) 16 end Receiver Every message from the sender is repeatedly sent in a frame to the receiver until acknowledges arrives acknowledgement Send acknowledgement Chapter 3 - Motivating Self-Stabilization 3-90

The alternating-bit algorithm – run sample
<m2 ,1> <m1 ,0> <0> bits = 1 bitR = 0 S received ack. R received m1 again S R <m2 ,1> . . . <0> <0> bits = 0 bitR = 0 R received m1 again Upon a timeout … S R <m1 ,0> <m1 ,0> <0> bits = 0 bitR = 0 S R <m1 ,0> <m1 ,0> bits = 0 bitR = 1 Upon a timeout … S R <m2 ,1> <m2 ,1> <0> bits = 1 bitR = 0 Upon a timeout … S R <m2 ,1> <0> <1> bits = 1 bitR = 1 R received m2 S R <m1 ,0> <0> bits = 0 bitR = 0 R received m1 S R <m2 ,1> <m2 ,1> <1> bits = 1 bitR = 1 S R <m1 ,0> bits = 0 bitR = 1 Once the sender receives an acknowledgment <1>, no frame with sequence number 0 exists in the system Chapter 3 - Motivating Self-Stabilization 3-91

There Is No Data-link Algorithm that can Tolerate Crashes
It is usually assumed that a crash causes the sender/receiver to reach an initial state No initialization procedure exists such that we can guarantee that every message fetched by the sender, following the last crash, will arrive at its destination The next Execution will demonstrate this point. Denote: CrashR – receiver crash CrashS – sender crash CrashX causes X to perform an initialization procedure Chapter 3 - Motivating Self-Stabilization 3-92

The Pumping Technique Reference Execution (RE) = CrashS, CrashR, sendS(fs1), receiveR(fs1), sendR(fr1), receiveS(fr1), sendS(fs2), … , receiveS(frk) The idea : repeatedly crash the sender and the receiver and to replay parts of the RE in order to construct a new execution E’ S R fsk fs2 fs1 S sends fs1 receives fr1 , sends fs2 receives fr2, … , receives fr(k-1) and sends fsk If these k frames are lost, no information about the message exists in the system S R fr1 fr frk R receives fs1, sends fr1, receives fs2, sends fr2 , … , receives fsk and sends frk We let S send fsi and receive fri (i from 1 to k) S R S R fs1 S sends fs1 S R fr1 R receives fs1 and sends fr1 S R CrashS Suppose CrashS and CrashR occurred CrashR S R CrashS fr1 fr fr(k-1) S crashes . m2 m1 S R CrashS S crashes fr1 S R fs2 fs1 S sends fs1 receives fr1 and sends fs2 S R fr1 fr2 R receives fs1, sends fr1, receives fs2 and sends fr2 S R fs2 fs1 CrashR R crashes S R fr1 fr frk CrashR Now S and R crash CrashS S R fsk fs2 fs1 During the replay the sender fetches several messages that are identical to m. Note that it is possible for the network layer to use the services for the data link layer to use the services of the data-link layer ..... Continue with the same technique m2 m2 m1 Chapter 3 - Motivating Self-Stabilization 3-93 93

Conclusion ! It is possible to show that there is no guarantee that the kth message will be received We want to require that eventually every message fetched by the sender reaches the receiver, thus requiring a Self- Stabilizing Data-Link Algorithm Chapter 3 - Motivating Self-Stabilization 3-94

Arbitrary configuration because of crashes
A combination of crashes and frame losses can bring a system to any arbitrary states of processors and an arbitrary configuration Chapter 3 - Motivating Self-Stabilization 3-96

Any Configuration Can be Reached by a Sequence of Crashes
The pumping technique is used to reach any arbitrary configuration starting with the reference execution Reference Execution (RE) = CrashS, CrashR, sendS(fs1), receiveR(fs1), sendR(fr1), receiveS(fr1), sendS(fs2), … , receiveS(frk) The technique is used to accumulate a long sequence of frames Chapter 3 - Motivating Self-Stabilization 3-97

..... Reaching an Arbitrary Configuration
Our first goal – creating an execution in which RE appears i times in a row (RE)i Denote : FrE (FsE) – the sequence of frames sent by the receiver (sender) in RE FirE (FisE) = the sequence Fr(s)E Fr(s)E … Fr(s)E (i times) S R fr1 fr frk First we use the Pumping Technique to receive RE S R F isE For any finite i, the technique can be extended to reach a configuration in which FisE appears in qs,r S R fsk, … , fs2, fs1, fs1, S sends fs1, receives fr1, sends fs2, receives fr2, … , sends fsk, receives frk, S R FsE fs2 fs1 S sends fs1, receives fr1, … , sends fsk, receives frk S R FsE FsE S received the first FrE, crashed and received the second S R fr1 FrE R receives fs1 sends fr1 … receives fsk and sends frk S R fs2 fs1 FrE S sends fs1, receives fr1, sends fs2 S R fs1 fr1 fr frk S sends fs1 S R CrashS fs1 fr1 fr frk S crashes S R FsE fr1 R receives fs1 and sends fr1 S R CrashS fr1 FrE S crashes S R CrashS fs2 fs1 FrE S crashes S R FsE fr1 CrashR R crashes ..... Continue with the same technique Chapter 3 - Motivating Self-Stabilization 3-98

Reaching an Arbitrary Configuration
Our second goal – achieving ca (an arbitrary configuration) Denote k1 (k2)- the number of frames in qs,r (qr,s) in ca i = k1+k2+2 S' R F k1+1sE qr,s S replays RE using the first FrE until it reaches its desired state (loosing the frames sent by it and the leftovers of Fk2rE that are not in qr,s) S' R' qs,r qr,s We do the same with R, reaching the arbitrary configuration ca S R F k1+1sE F k2+1rE R replays RE k2+1 times S R F isE Using the previous technique we accumulate FisE Chapter 3 - Motivating Self-Stabilization 3-99

Crash-Resilient Data-Link Algorithm,With a Bound on the Number of Frames in Transit
Crashes are not considered severe type of faults (Byzantine are more severe - chapter 6) The algorithm uses the initialization procedure, following the crashes of S and R bound – the maximal number of frames that can be in transit S R . . .<clean ,bound+1>. . . <ackClean ,bound+1> When the sender receives the first <ackClean,bound+1> it can be sure that the only label in transit is bound+1, and can initialize the alternating bit algorithm (similarly R can initialize as well) S R <clean ,2>. . .<clean ,1> <ackClean ,1> S received <ackClean,1>, then sends repeatedly <clean,2> until it will receive <ackClean,2> S R <clean ,1> S ,in after-crash state, invokes a clean procedure S R CrashS S crashes ..... Continue until S receives <ackClean,bound+1> S R <clean ,1>. . . <clean ,1> <ackClean ,1> S R <clean ,1> <clean ,1> S R . . . <clean ,1> . . . <ackClean ,1> S R <mnew ,0> <ackClean ,1> bits = 0 bitR = 1 Chapter 3 - Motivating Self-Stabilization 3-100

Crash-Resilient Data-Link Algorithm – R crashes
<msg ,FrameBit> bitR =FrameBit R received msg and assigned FrameBit to bitR it then delivers msg to the output queue – The Problem : extra copy of msg in the output queue S R CrashR R crashes S R <msg ,FrameBit> bitR = i Chapter 3 - Motivating Self-Stabilization 3-101

Crash-Resilient Data-Link Algorithm – R crashes
Can we guarantee at most one delivery, and exactly-once delivery after the last crash? bitR initialization should assure that a message fetched after the crash will be delivered A solution: S sends each message in a frame with label 0, until Ack. arrives and then sends the same message with label 1 until an Ack. arrives R delivers a message only with label 1 that arrives immediately after label 0 Chapter 3 - Motivating Self-Stabilization 3-102

What is the Rational behind assuming that the states of the processors can be corrupted while the processors’ programs cannot ? If the program is subjected to corruption, any configuration is possible. The Byzantine model allows 1/3 of processors to execute corrupted programs The program is stored in a long-term memory device which makes it possible to Reload program statements periodically Protect the memory segment using a read-only memory device What is the Rational behind assuming that the states of the processors can be corrupted while the processors’ programs cannot ? ???????? 1)The program is stored in a long-term memory device which makes it possible to 1)Reload program statements periodically 2)Protect the memory segment using a read-only memory device 2)There is an uncorrupted copy of the program in a secondary device (hardware or memory device) Chapter 3 - Motivating Self-Stabilization 3-104 104

Safety Properties Safety and Liveness properties should be satisfied by a distributed algorithm Safety ensures avoiding bad configurations Liveness ensures achieving the systems’ goal The designer of a self-stabilizing algorithm wants to ensure that even if the safety property is violated, the system execution will reach a suffix in which both properties hold What use is an algorithm that doesn’t ensure that a car never crashes? If the faults are severe enough to make the algorithm reach an arbitrary configuration, the car may crash no matter what the algorithm is chosen Chapter 3 - Motivating Self-Stabilization 3-105

Safety Properties … A safety property for a car controller might be: never turn into a one-way road A self Stabilization controller will recover from this non-legal init (by turning the car) When no specification exists the car can continue driving on this road and crash with other cars Chapter 3 - Motivating Self-Stabilization 3-106

Processors Can Never be Sure that a Safe Configuration is Reached
What use is an algorithm in which the processors are never sure about the current global state? The question confuses the assumptions (transient faults occurrence) with the algorithm that is designed to fit the severe assumptions. A self-stabilizing algorithm can be designed to start in a particular (safe) state A self-stabilizing algorithm is at least good as a non-self-stabilizing one for the same task, and is in fact much better !!! Chapter 3 - Motivating Self-Stabilization 3-107

Chapter 4: roadmap 4.1 Token Passing: Converting a Central Daemon to read/write 4.2 Data-Link Algorithms: Converting Shared Memory to Message Passing 4.3 Self-Stabilizing Ranking: Converting an Id-based System to a Special-processor System 4.4 Update: Converting a Special Processor to an Id- based Dynamic System 4.5 Stabilizing Synchronizers: Converting Synchronous to Asynchronous Algorithms 4.6 Self-Stabilizing Naming in Uniform Systems: Converting Id-based to Uniform Dynamic Systems

Token Passing: Converting a Central Daemon to read/write
Distributed Daemon – activates a selected set of processors simultaneously to execute a computation step The Daemon chooses a set of processors state1 1 Each processor in the set simultaneously reads from its neighbors… state2 2 state4 … and then all write their new state 3 4 state3 Central Daemon – is a special case of Distributed Daemon in which the set is of exactly 1 processor Synchronous System – is a special case in which the set consists all the processors in the system 110

The use of Central Daemon
The literature in self-stabilizing is rich in algorithms that assume the existence of powerful schedulers WHY ? Dijkstras’ choices in the first work in the field The assumption of the existence of a daemon enables the designer to consider only a subset of the possible execution set An algorithm designed to work in read/write atomicity can be used in any system in which there exists a daemon but the reverse wont work 111

All the above facts are our motivation for designing a compiler:
AL for T (daemon) (read/write) An algorithm designed to stabilize in the presence of distributed daemon must stabilize in a system with central daemon

Converting How does the compiler work ?
1. Compose a spanning tree using the Spanning Tree Construction algorithm 2. Construct an Euler tour on the tree to create a virtual ring for the Mutual Exclusion algorithm 3. A processor that enters the critical section reads the state of its neighbors changes state and writes, then it exits the critical section Al for T (read/write) = Al for T (daemon) ◦ Mutual Exclusion

Converting Shared Memory to Message Passing
Designing a self-stabilization algorithm for asynchronous message-passing systems is more subtle than the same task in shared memory systems Main difficulty : The messages stored in the communication links No bound on message delivery time No bound on number of messages that can be in link There are infinitely many initial configurations from which the system must stabilize

Converting Shared Memory to Message Passing
Our main goal is designing of compiler: Compiler AL for T (read/write) (message passing) First goal in designing of such a compiler is a self-stabilizing data-link algorithm

Definition of Self-Stabilizing Data-Link Algorithm
Data-Link Algorithm : Messages fetched by sender from network layer should be delivered by receiver to network layer without duplications, omissions or reordering One of the implementations of data-link task is the token-passing algorithm Token-passing task is a set of executions TP The legal execution of TP is the sequence of configurations in which : - No more than one processor holds the token - Both the sender and receiver hold the token in infinitely many configurations

Unbounded solution of TP task
Sender: 01 upon timeout 02 send (counter ) 03 upon message arrival 04 begin 05 receive (MsgCounter) 06 if MsgCounter ≥ counter then 07 begin 08 counter := MsgCounter + 1 09 send (counter) 10 end 11 else send (counter) 12 end A timeout mechanism is used to ensure that the system will not enter to communication-deadlock configuration Each message has integer label called MsgCounter Sender ( and Receiver ) maintains an unbounded local variable called counter

Unbounded solution of TP task
Receiver: 13 upon message arrival 14 begin 15 receive (MsgCounter) 16 if MsgCounter ≠ counter then 17 counter := MsgCounter 18 send (counter) 19 end Token arrives Token released In safe configuration of TP and the algorithm - counter values of all messages and values of the counters of sender and receiver, have the same value (lemma 4.1)

The algorithm is self-stabilizing
For every possible configuration c , every fair execution that starts in c reaches a safe configuration with relation to TP (Theorem 4.1) Question : Whether the unbounded counter and label can be eliminated from the algorithm ? Answer : NO

Lower Bound on the System Memory
The memory of the system in configuration c is the number of bits for encoding state of sender, receiver, and messages in transit. Weak-Exclusion task (WE): In every legal execution E, there exists a combination of steps, a step for each processor, so that these steps are never executed concurrently. We will prove that there is no bound on system memory for WE task .

Lower Bound on the System Memory
Theorem: For any self-stabilizing message driven protocol for WE task and for any execution E’ in WE all the configurations are distinct. Hence for any t > 0, the size of at least one of the first t configurations in E is at least log2(t)

Receiver acknowleging…
Proof of Theorem Any execution E’ in which not all the configurations are distinct has circular sub- execution E = (c1,a2,….,cm) where (c1=cm) Sender sending… Receiver acknowleging… c1 s1 r1 Sender Ps Receiver Pr qs,r(c 1) qr,s(c1) qSr,s(E) qSs,r(E) E cm=

Proof - Building CE and cinit
Let E – be circular sub-execution Si - sequence of steps of Pi CE – set of circular executions Each execution in CE – merge of Si’s , while keeping their internal order We obtain initial configuration of Ec in CE from c1 of E, and sequence of messages sent during E cinit r1 s1 qr,s(c 1) qs,r(c 1) qSs,r(E) qSr,s(E)

Proof - Possible execution in CE
cinit r1 s1 qr,s(c 1) qs,r(c 1) qSs,r(E) qSr,s(E) Receiver recieves… qRs,r( E) Receiver sends… qSr,s( E) qr,s(c 1) qSr,s(E) E steps Sender recieves… qRr,s( E) Sender sends… qSs,r( E) qr,s(c 1) qs,r(c 1) cinit r1 s1 qSs,r(E) qSr,s(E)

Proof – cont… cinit c Sender steps during E is Ssender = {a1,a2…am}
Receiver steps during E is Sreceiver = { b1,b2…bk } For any pair <ai, bj> there exists E’’CE in which there is configuration c, such that ai and bj is applicable in c  c is not safe configuration a1 a2 … ai-1 b1 b2 … bj-1 c ai & bj

Proof – cont… If self-stabilizing algorithm AL for WE task have circular sub-execution E  exists infinite fair execution E∞ of AL , none of whose configurations is safe for WE E’: cinit c1 cinit cinit E’ from CE(E) E’ from CE(E) …

… Proof – cont… Let assume in contradiction that c1 is safe
Then let’s extend E’ to E∞ by E1 , E2 , … , Ek - executions from CE(E) E∞ : cinit c1 … E1• E2• … • Ek c’ For each pair < ai, bj > there is c’ in E∞ so that both ai, bj applicable in c’  c’ is not safe The proof is complete!

Bounded-Link Solution
Let cap be the bound on number of the messages in transit The algorithm is the same as presented before with counter incremented modulo cap+1 08 counter := (MsgCounter + 1)mod(cap+1) Sender must eventually introduce a counter value that not existing in any message in transit

Randomized Solution The algorithm is the same as original one with counter chosen randomly 08 label := ChooseLabel(MsgLabel) At least three labels should be used The sender repeatedly send a message with particular label L until the a message with the same label L arrives The sender chooses randomly the next label L’ from the remaining labels so that L’ ≠ L

Self-Stabilizing Simulation of Shared Memory
The heart of the simulation is a self-stabilizing implementation of the read and write operations The simulation implements these operations by using a self-stabilizing, token passing algorithm The algorithm run on the two links connecting any pair of neighbors In each link the processor with the larger ID acts as the sender while the other as the receiver (Remind: all processors have distinct IDs)

Self-Stabilizing Simulation of Shared Memory - cont…
Every time Pi receives a token from Pj. Pi write the current value of Rij in the value of the token Write operation of Pi into rij is implemented by locally writing into Rij Read operation of Pi from rji is implemented by: Pi receives the token from Pj Pi receives the token from Pj. Return the value attached to this token

Self-Stabilizing Simulation of Shared Memory - Run
Write Operation: Pi write x to rij Read Operation: Pi read from value y from rji t+1 | value = z Pj Pi Rij Rji Rji = Y Rij=X t+1 | value = Y t | value = z t+1 | value = y 3. Pj receive token from Pi and write Rji to value of the token 4. Pi receive token from Pj and read the value of the token 2. Pi send token to Pj 1. Pi receive token from Pj Pi writes to Rij

Converting an Id-based System to a Special-processor System
Our goal is to design a compiler that converts self-stabilizing algorithm for a unique ID system to work in special processor system. The ranking (compiler) task is to assign each of the n processors in the system with a unique identifier in the range 1 to n.

Converting an Id-based System to a Special-processor System
We will form the self stabilizing ranking algorithm by running 3 self-stabilizing algorithms one after the other: Self-Stabilizing spanning tree construction (section 2.5) Self-Stabilizing counting algorithm Self-Stabilizing naming algorithm Special processor system Unique Id’s Spanning tree const. Counting algorithm Naming

The correctness proofs by induction on the height of a processor
Self-Stabilizing Counting Algorithm Assuming rooted spanning tree system in which every processor knows its parent and children's Pi has a variable counti that hold the number of processor in sub-tree where Pi is a root The correctness proofs by induction on the height of a processor

Self-Stabilizing Counting Algorithm
01 Root: do forever 02 sum:= 0 03 forall Pj  children(i) do lrji := read(rji) sum := sum + lrji.count 06 od 07 counti = sum + 1 08 od 09 Other: do forever 10 sum:= 0 11 forall Pj  children(i) do 12 lrji := read(rji) sum := sum + lrji.count 14 od 15 counti = sum + 1 16 write ri,parent.count := counti 17 od Calculate counti: sum the values of rji registers of his child’s and 1 (himself) Write local count value to communication register

Self-Stabilizing Naming Algorithm
The naming algorithm uses the value of the count fields from counting algorithm. Algorithm assign unique identifiers to the processors. The identifier of a processor is stored in the IDi variable. Proof of the stabilization by induction on the distance of the processors from the root

Self-Stabilizing Naming Algorithm
01 Root: do forever 02 IDi:= 1 03 sum := 0 04 forall Pj  children(i) do lrji := read(rji) write rij.identifier := Id sum 06 sum := sum + lrji.count 08 od 09 od 10 Other: do forever 11 sum:= 0 12 lrparent,i := read(rparent,I) 13 IDi := lrparent,I.identifier 14 forall Pj  children(i) do 15 lrji := read(rji) write rij.identifier := Idi sum 17 sum := sum + lrji.count 18 od 19 od

Ranking Task Run 1 1 8 2 2 6 6 3 3 4 4 5 5 7 7 Counti := 8 IDi := 1
r.id=2 r.id= r.id=6 r.id= Counti := 2 IDi := Counti := 2 IDi := 2 Counti := IDi := r.id=4 r.id= 8 2 2 Counti := 1 IDi := 8 Counti := IDi := Counti := 1 IDi := Counti := IDi := Counti := 3 IDi := Counti := 3 IDi := 6 r.id= r.id=3 6 6 Counti := 2 IDi := Counti := IDi := Counti := 2 IDi := 4 r.id= r.id=7 3 3 4 4 Counti := 1 IDi := 3 Counti := IDi := Counti := 1 IDi := r.id=5 r.id= r.id=8 r.id= 5 5 7 7 Counti := IDi := Counti := 1 IDi := 5 Counti := 1 IDi := Counti := 2 IDi := 7 Counti := IDi := Counti := 2 IDi :=

Counting Algorithm for non-rooted tree
01 do forever 02 forall Pj  N(i) do lrji := read (rji) 03 sumi := 0 04 forall Pj  N(i) do 05 sumj := 0 06 forall Pk  N(i) do 07 if Pj != Pk then sumj := sumj +lrki.count 09 od 10 counti[j] := sumj + 1 11 sumi := sumi + sumj 12 write rij.count := counti[j] 13 od 14 counti = sumi + 1 15 od Each processor Pi has a variable counti[j] for every neighbor Pj The value of counti[j] is the number of processors in subtree of T to which Pi belongs and Pj doesn’t. The correctness proof is by induction on the height of the registers rij

Counting Algorithm for non-rooted tree Run
Counti[j]=6 Counti[j]=3 Counti[j]=5 Counti[j] =4 Counti[j]=6 Counti[j]=5 Counti[j]=5 Counti[j]=2 Counti[j]=1 Counti[j]=3 Counti[j]=4 Counti[j]=7 Counti[j] =3 Counti[1]=6 Counti[j] =1 Counti[j]=3 Counti[j]=2 Counti[1]=2 Counti[1]=7 Counti[1]=2 Counti[1]=1 Counti[1]=7

Update - Converting a Special Processor to an Id-based Dynamic System
The task of the update algorithm in id-based system is to inform each processor of the other processors that are in its connected component. As a result every processor in the connected component knows the maximal identifier in the system, and a single leader is elected. The update algorithm is a self-stabilizing leader election algorithm within O(d) cycles (pulses). The motivation for the restriction that the update must work in id-based system can be viewed by examining Dijkstra self stabilizing ME algorithm for a ring of processors… 145

Dijkstra proof Dijkstra proved that without a special processor, it is impossible to achieve ME in a self-stabilizing manner The impossibility proof is for composite number of identical processors connected in a ring activated by a central daemon ME requires that processor Pi should execute the critical section if and only if it is the only processor that can change its state (by reading its neighbors’ states) at that execution point

Dijkstra proof Consider P1,P3 starting in the same state S0, P2,P4 starting in the same state S0’ And execution order is: S0 S1 P1 ,P3 ,P2 ,P4 … p1 Then, p1 and p3 will be at S1, p2 and p4 will be at S1’. We can see that symmetry in state is preserved p4 p2 S0’ S1’ S0’ S1’ p3 S0 S1

Conclusions from Dijkstra’s proof
Whenever P1 has permission to execute critical so does P3. (Like that for P2 and P4). With no central daemon the impossibility result of self-stabilizing ME algorithm holds also for a ring of prime number of identical processors. We start in a configuration that all the processors are in the same state, and the contents of all the registers are identical. An Execution that preserve that symmetry forever is one that every processor reads all neighbors registers before any writing is done. The restriction for designing the update algorithm in an id-based (not identical) system is thus clear.

Each processor Pi holds a BFS tree rooted at Pi.
The Update algorithm outlines The algorithm constructs n directed BFS trees. One for every processor. Each processor Pi holds a BFS tree rooted at Pi. When a node Pj with distance k+1 from Pi has more than one neighbor at distance k from Pi, Pj is connected to the neighbor (parent) with the maximal ID. Each Pi reads from his δ neighbors their set of tuples <j,x> where j represents a processor id, and x is the distance of j from the processor that holds this tuple. Pi tuples are computed from these neighbors’ sets, adapting for each ji the tuple <j,x> with the smallest x and adding 1 to x. Pi also adds the tuple <i,0> indicating the distance from itself. At the end of the iteration, Pi removes the floating (false) tuples.

Update algorithm 01 do forever 02 Readseti :=Ø 03 forall PjN(i) do
Readseti := Readseti  read(Processorsj) 05 Readseti := Readseti \\ <i,*> 06 Readseti := Readseti ++ \\<*,1> 07 Readseti := Readseti  {<i,0>} 08 forall Pj  processors(Readseti) do Readseti := Readseti\\ NotMinDist(Pj, Readseti) 10 write Processorsi := ConPrefix(Readseti) 11od

Chapter 6: Convergence in the Presence of Faults - Motivation
Processors crash and software (and hardware) may contain flaws Byzantine and crash failures are both well studied models Algorithms that tolerate failures are of practical interest The focus of this presentation is the integration of self-stabilization with other fault models chapter 6 - Convergence in the Presence of Faults 1-152 152

Byzantine Faults “Byzantine” – permanent faults
The type of faults is not known in advance Processors can exhibit arbitrary “malicious”, “two faced” behavior Models the possibility of code corruption chapter 6 - Convergence in the Presence of Faults 1-153

Byzantine Fault Model A Byzantine processor “fights” against the rest of the processors in order to prevent them from reaching their goal A Byzantine processor can send any message at any time to each of its neighbors If 1/3 or more of the processors are Byzantine it is impossible to achieve basic tasks such as consensus in distributed systems The Byzantine fault model assumes that the processor is controlled by an adversary that “fights” against the rest of the processors in order to prevent them from reaching their goal If more than 1/3 of the processors are Byzantine it is impossible to achieve basic tasks such as consensus in distributed systems No existence of a non-stabilizing algorithm => no existence of a self-stabilizing algorithm for this task chapter 6 - Convergence in the Presence of Faults 1-154 154

We will examine a six processor ring
At least 1/3 of the processors are Byzantine  No convergence  P’1 and P’2 have the same input, since P’3 and P3 may be Byzantine they must choose 1  P2 and P3 have the same input, since P’1 and P1 may be Byzantine they must choose 0 P1 P2 P3 P’1 P’2 P’3 i=0 i=1 P1 P2 P3 i=0 P’1 P’2 P’3 i=1 choose the same value. when the non-faulty processors have the same input, that input must be chosen. Assume there is a distributed algorithm AL that achieves consensus in the presence of a single Byzantine processor in the next system     We will examine a six processor ring Note that AL is designed to be executed on a system with only 3 processors c=1 c=0 c = ? Contradiction !! P’1 and P3 must decide on one input BUT P3 must choose 0 and P’1 must choose 1 Consider a system of three processors p1,p2,p3 that are connected to one another. Each processor has a single input bit. The task of the non faulty processors is to choose the same value. More over – when the non-faulty processors have the same input, that input must be chosen. Assume there is a distributed algorithm AL that achieves consensus in the presence of a single Byzantine processor i = input value c = consensus chapter 6 - Convergence in the Presence of Faults 1-155 155

At least 1/3 of the processors are Byzantine  No convergence
We have just seen the impossibility result for 3 processors, but is it a special case? Is it possible to reach consensus when the number of processors is 3f, where f>1 is the number of Byzantine processors? No! chapter 6 - Convergence in the Presence of Faults 1-156

At least 1/3 of the processors are Byzantine  No convergence
Proof: (by reduction) Divide the system into 3 clusters (group) of processors, one of which contains all the Byzantine processors. Replace each cluster by a super processor that simulates the execution of the cluster. The existence of an algorithm for the case 3f, f>1 , implies existence for f=1, which we have proved impossible. chapter 6 - Convergence in the Presence of Faults 1-157

The Use of Self-Stabilization
What happens if… For a short period, 1/3 or more of the processors are faulty or perhaps temporarily crashed? Messages from a non-faulty processor are lost? Such temporary violations can be viewed as leaving a system in an arbitrary initial state Such temporary violations can be viewed as leaving a system in an arbitrary initial state that could have been chosen by a malicious adversary and from which the algorithm resumes Self–Stabilizing algorithms that cope with Byzantine and transient faults and stabilize in spite of these faults are presented, and demonstrate the generality of the self-stabilization concept! chapter 6 - Convergence in the Presence of Faults 1-158 158

Chapter 6: roadmap 6.1 Digital Clock Synchronization
6.2 Stabilization in Spite of Napping 6.3 Stabilization in Spite of Byzantine Faults 6.4 Stabilization in the Presence of Faults in Asynchronous Systems chapter 6 - Convergence in the Presence of Faults 1-159

Digital Clock Synchronization - Motivation
Multi processor computers Synchronization is needed for coordination – clocks Global clock pulse & global clock value Global clock pulse & individual clock values Individual clock pulse & individual clock values Fault tolerant clock synchronization Multi processor computers Have potential to be resilient to faults Faults # grows with the # of processors Fault-tolerance is vitally important Synchronization is needed for coordination – clocks Global clock pulse & global clock value Global clock pulse & individual clock values Individual clock pulse & individual clock values Fault tolerant clock synchronization chapter 6 - Convergence in the Presence of Faults 1-160 160

Digital Clock Synchronization
In every pulse each processor reads the value of it’s neighbors clocks and uses these values to calculate its new clock value . The Goal (1) identical clock values (2) the clock values are incremented by one in every pulse The problem is defined for a system of n identical processors connected to a global common clock pulse. Each processor maintains a digital clock value. In every pulse each processor executes a step in which it reads the value of it’s neighbors’ clocks and uses these values to calculate its new clock value . chapter 6 - Convergence in the Presence of Faults 1-161 161

Digital Clock Sync – Unbounded version
01 upon a pulse 02 forall Pj  N(i) do send (j,clocki) 03 max := clocki 04 forall Pj  N(i) do 05 receive(clockj) 06 if clockj  max then max := clockj 07 od 08 clocki := max + 1 A simple induction can prove that this version of the algorithm is correct: If Pm holds the max clock value, by the i’th pulse every processor of distance i from Pm holds the maximal clock value chapter 6 - Convergence in the Presence of Faults 1-162

Digital Clock Synchronization – Bounded version
Unbounded clocks is a drawback in self- stabilizing systems The use of 264 possible values does not help creating the illusion of “unbounded”: A single transient fault may cause the clock to reach the maximal clock value … chapter 6 - Convergence in the Presence of Faults 1-163

Digital Clock Sync – Bounded version (max)
01 upon a pulse 02 forall Pj  N(i) do send (i,clocki) 03 max := clocki 04 forall Pj  N(i) do 05 receive(clockj) 06 if clockj  max then max := clockj 07 od 08 clocki := (max + 1) mod ((n +1)d +1) The Boundary M = ((n+1)d+1) Why is this algorithm correct? The number of different clock values can only decrease, and is reduced to a single clock value Note that for ease of description the values we choose for M are not the minimal possible value The number of clock values can only decrease, and is reduced to a single clock value chapter 6 - Convergence in the Presence of Faults 1-164 164

For Example: M = ((n+1)d+1) = 4*2+1 = 9 Pulse Round 1 Round 2 Pulse
3 1 2 4 5 6 7 8 M = ((n+1)d+1) = 4*2+1 = 9 p3 p1 Pulse Round 1 Round 2 Pulse Round 3 p1 p2 p3 p2 4 5 3 6 1 6 1 8 5 8 5 6 4 3 chapter 6 - Convergence in the Presence of Faults 1-165

Why is this algorithm correct? If all the clock values are less than M-d we achieve sync before the modulo operation is applied m-i m-1 3 2 1 . m-d . m-1 3 2 1 m-2 m-d m-d-i After d pulses there must be convergence and the max value is less than m Note that for ease of description the values we choose for M are not the minimal possible value chapter 6 - Convergence in the Presence of Faults 1-166 166

… Why is this algorithm correct? If not all the clock values are less than M-d By the pigeonhole principle, in any configuration there must be 2 clock values x and y such that y-x  d+1, and there is no other clock value between After M-y+1 pulses the system reaches the configuration in which all clock values are less than M-d Note that for ease of description the values we choose for M are not the minimal possible value chapter 6 - Convergence in the Presence of Faults 1-167 167

Digital Clock Sync – Bounded version (min)
The Boundary M = 2d+1 Why is this algorithm correct? If no processor assigns 0 during the first d pulses – sync is achieved (can be shown by simple induction) Else A processor assigns 0 during the first d pulses, d pulses after this point a configuration c is reached such that there is no clock value greater than d: the first case holds 01 upon a pulse 02 forall Pj  N(i) do send (j,clocki) 03 min := clocki 04 forall Pj  N(i) do 05 receive(clockj) 06 if clockj  min then min := clockj 07 od 08 clocki := (min + 1) mod (2d +1) chapter 6 - Convergence in the Presence of Faults 1-168

Digital clocks with a constant number of states are impossible
Consider only deterministic algorithm: There is no uniform digital clock- synchronization algorithm that uses only a constant number of states per processor. Thus, the number of clock values in a uniform system must be related to the number of processors or to the diameter. chapter 6 - Convergence in the Presence of Faults 1-169

A special case will imply a lower bound for the general case A processor can read only the clock of a subset of its neighbors In a undirected ring every processor has a left and right neighbor, and can read the state of its left neighbor sit+1= f(si-1t, sit) sit - state of Pi in time t, f - the transition function |S| - the constant number of states of a processor The proof shows that in every step, the state of every processor is changed to the state of its right processor The proof shows that a configuration exists for a sufficiently large ring such that the states of the processor rotate: in every step, the state of every processor is changed to the state of its right processor chapter 6 - Convergence in the Presence of Faults 1-170 170

In each pulse, the states are rotated one place left.
Digital clocks with a constant number of states are impossible s1 s2 . . . s3 = f(s1, s2) sl+2 = f(sl, sl+1) . . . sk sj sj+1 sk-1 . . . sk sj sj+1 sk-1 Pulse sk+1 = sj sk+2 = sj+1 . . . sk sj sj+1 sk-1 sk+1 = sj sk+2 = sj+1 . . . sk sj sj+1 sk+1 = sj sk+2 = sj+1 . . . sk sj sj+1 There must be a sequence of states sj,sj+1,…, sk-1,sk that is, a subset of this infinite sequence such that f(sk-1,sk) = sj and f(sk,sj) = sj+1 Use s1 and s2 to construct an infinite sequence of states such that si+2 = f(si,si+1) sk+1 = sj sk+2 = sj+1 . . . sk sj sj+1 In each pulse, the states are rotated one place left. chapter 6 - Convergence in the Presence of Faults 1-171

Since the states of the processors encodes the clock values, and the set of states just rotates around the ring, We must assume that all the states encode the same clock. On the other hand, the clock value must be increments in every pulse. Contradiction. chapter 6 - Convergence in the Presence of Faults 1-172

Stabilizing in Spite of Napping
Wait-free self-stabilizing clock-synchronization algorithm is a clock-sync. Algorithm that copes with transient and napping faults Each non-faulty operating processor ignores the faulty processors and increments its clock value by one in every pulse Given a fixed integer k, once a processor Pi works correctly for at least k time units and continues working correctly, the following properties hold: Adjustment Pi does not adjust its clock Agreement Pis clock agrees with the clock of every other processor that has also been working correctly for at least k time units chapter 6 - Convergence in the Presence of Faults 1-174

Note that this approach wont work using bounded clock values
Algorithms that fulfill the adjustment-agreement – unbounded clocks Simple example for k=1, using the unbounded clocks In every step – each processor reads the clock values of the other processors, and chooses the maximal value (denote by x) and assigns x+1 to its clock Note that this approach wont work using bounded clock values After an execution of P1, it’s clock holds the maximal clock value, and wont adjust its clock as long as it doesn’t crash 11 9 3 5 P1 After an execution of P1, it’s clock holds the maximal clock value, and wont adjust its clock as long as it doesn’t crash 10 9 3 5 P1 After an execution of P1, it’s clock holds the maximal clock value, and wont adjust its clock as long as it doesn’t crash 7 9 3 5 P1 7 9 3 5 P1 max 5 The clock value never changes until the napping processor with max value starts to work 7 max 5 8 max 5 chapter 6 - Convergence in the Presence of Faults 1-175

Algorithms that fulfill the adjustment-agreement – bounded clock values
Using bounded clock values (M) The idea – identifying crashed processors and ignoring their values Each processor P has: P.clock  {0… M-1} Q P.count[Q]  {0,1,2} P is behind Q if P.count[Q]+1 (mod 3) = Q.count[P] P Q P.count[Q] 1 2 Q.count[P] 1 2 chapter 6 - Convergence in the Presence of Faults 1-176

Algorithms that fulfill the adjustment-agreement – bounded solution
The implementation is based on the concept of the “rock, paper, scissors” children’s game VS VS VS 2 1 VS VS 1 > 1 2 VS 2 chapter 6 - Convergence in the Presence of Faults 1-177

Algorithms that fulfill the adjustment-agreement – bounded solution
The program for P: Read every count and clock Find the set R that are not behind any other processor If R   then P finds a processor K with the maximal clock value in R and assigns P.clock := K.clock + 1 (mod M) For every processor Q, if Q is not behind P then P.count[Q] := P.count[Q] + 1 (mod 3) chapter 6 - Convergence in the Presence of Faults 1-178

Self-stabilizing Wait-free Bounded Solution – Run Sample
8 4 2 1 P1 P2 P3 P4 R 7 P1 P2 P3 P4 R 5 2 1 P1 P2 P3 P4 R 5 6 2 P1 P2 P3 P4 R 8 4 2 1 P1 P2 P3 P4 Active processor K = 2 Simple connection “behind” connection chapter 6 - Convergence in the Presence of Faults 1-179

The algorithm presented is wait-free and self-stabilizing
The algorithm presented is a wait-free self- stabilizing clock-synchronization algorithm with k=2 (Theorem 6.1) All processors that take a step at the same pulse, see the same view Each processor that executes a single step belongs to R, in which all the clock values are the same  the agreement requirement holds Every processor chooses the maximal clock value of a processor in R, and increments it by 1 mod M  the adjustment requirement holds The proof assumes an arbitrary start configuration  the algorithm is both wait-free and self-stabilizing Assuming Pi executes more than k steps: - R is not empty (after Pi’s first step) - Following Pi’s second step the clock values of the processors that belong to R are the same chapter 6 - Convergence in the Presence of Faults 1-180 180

Enhancing the fault tolerance
Using self-stabilizing algorithm  if temporary violation, of the assumptions on the system, occur the system synchronizes the clocks when the assumptions hold again Byzantine processor may exhibit a two- faced behavior,sending different messages to its neighbors If starting in an arbitrary configuration, during the future execution more than 2/3 of the processors are non-faulty, the system will reach a configuration within k rounds in which agreement and adjustment properties hold chapter 6 - Convergence in the Presence of Faults 1-182

Self Stabilizing clock synchronization algorithm
Complete communication graph f = # of Byzantine faults Basic rules: Increment – Pi finds n-f-1 clock values identical to its own The action – (increment clock value by 1) mod M Reset – fewer than n-f-1 are found The action – set Pi’s clock value to 0 After the 2nd pulse, there are no more than 2 distinct clock values among the non- faulty processors No distinct supporting groups for 2 values may coexist chapter 6 - Convergence in the Presence of Faults 1-183

Suppose 2 such values exist: x and y.
No distinct supporting groups for 2 values may coexist Suppose 2 such values exist: x and y. p1 p2 x y n-f processors gave (x-1) n-f processors gave (y-1) Since n>3f the number of non-faulty processors is at least: 2n-4f>2n-n-f=n-f There are at least n-2f non-faulty processors with (x-1) There are at least n-2f non-faulty processors with (y-1) There are at least 2n-4f non-faulty processors chapter 6 - Convergence in the Presence of Faults 1-184

How can a Byzantine processor prevent reaching 0, simultaneously even after M-1 rounds
 1 P1 P2 P3 P4 will reset  1 P1 P2 P3 P4 will reset  1 P1 P2 P3 P4 will reset This strategy can yield an infinite execution in which the clock values of the non-faulty processors will never be synchronized n-f-1 = 2  f= 1 chapter 6 - Convergence in the Presence of Faults 1-185

       The randomized algorithm
As a tool to ensure the the set of clock values of the non-faulty processors will eventually, with high probability, include only a single clock If a processor reaches 0 using “reset”, and has the possibility to increment it’s value, it tosses a coin  1 P1 P2 P3 P4 Note that NO reset was done  the values were incremented automatically  2 1 P1 P2 P3 P4  1 P1 P2 P3 P4  1 P1 P2 P3 P4 randomized  1 P1 P2 P3 P4  1 P1 P2 P3 P4  1 P1 P2 P3 P4 randomized  1 P1 P2 P3 P4 randomized chapter 6 - Convergence in the Presence of Faults 1-186

Digital clocks in the presence of Byzantine processors
01 upon a pulse 02 forall Pj  N(i) do send (j,clockj) 03 forall Pj  N(i) do 04 receive (clockj) (*unless a timeout*) 05 if |{j|i j, clocki  clockj}| < n – f – 1 then 06 clocki := 0 07 LastIncrementi := false 08 else 09 if clocki  0 then 10 clocki := (clocki + 1) mod M 11 LastIncrementi := true 12 else if LastIncrementi = true then clocki := 1 else clocki := random({0,1}) if clocki = 1 then LastIncrementi := true else LastIncrementi := false used since Byzantine neighbor may not send a message Indicates a reset or an increment operation chapter 6 - Convergence in the Presence of Faults 1-187

The randomized algorithm
If no sync is gained after a sequence of at most M successive pulses all non-faulty processors hold the value 0 At least 1 non faulty processor assigns 1 to its clock every M successive pulses In expected M·22(n-f) pulses, the system reaches a configuration in which the value of every non-faulty processor’s clock is 1 (Theorem 6.2) Proving using the scheduler-luck game  The expected convergence time depends on M What if M=264 ? chapter 6 - Convergence in the Presence of Faults 1-188

Parallel Composition for Fast Convergence
The purpose : achieving an exponentially better convergence rate while keeping the max clock value of no smaller than 264 The technique can be used in a synchronous system In every step Pi will execute several independent versions of a self-stabilizing algorithm Compute it’s output using the output of all versions chapter 6 - Convergence in the Presence of Faults 1-189

Using the Chinese remainder theorem : (DE Knuth. The Art of Computer Programming vd.2. Addison-Wesely, 1981) Let m1,m2, … ,mr be positive integers that are relatively prime in pairs, i.e., gcd(mj, mk)=1 when jk. Let m= m1m2•••mr, and let a,u1,u2, … ,ur be integers. Then there is exactly one integer u that satisfies the conditions a  u  a+m and u  uj (mod mj) for 1  j  r (Theorem 6.3) chapter 6 - Convergence in the Presence of Faults 1-190

Choose : a=0 r primes 2,3,..,r such that 2·3·5···m(r-1)M 2·3·5···mr The lth version uses the lth prime ml , for the value Ml (as the clock bound) A message sent by Pi contains r clock values (one for each version) The expected convergence time for all the versions to be synchronized is less than (m1+m2+ … +mr)•22(n-f) chapter 6 - Convergence in the Presence of Faults 1-191

GAS Parallel Composition for Fast Convergence 7 5 3 2 2 3 5 7 1 2 3 5
The Chinese remainder theorem states that: Every combination of the parallel version clock corresponds to a unique clock value in the range 2357… GAS Start filling 7 5 3 2 2 3 5 7 1 2 3 5 7 2 3 5 7 1 2 3 5 7 2 3 5 7 1 4 chapter 6 - Convergence in the Presence of Faults 1-192

Stabilization in the Presence of Faults in Asynchronous Systems
For some tasks a single faulty processor may cause the system not to stabilize Example – counting the number of processors in a ring communication graph, in presence of exactly 1 crashed processor. Eventually each processor should encode n-1 P2 P3 P4 P1 P2 P3 P4 P1 3 P2 P3 r12 = x P4 P1 r43=z 3 2 P2 P3 r12 = x P4 P1 3 r43=z 3 Assume the existence of a self- stabilizing algorithm AL that does the job in the presence of exactly one crashed processor It is NOT possible to design a self-stabilizing algorithm for the counting task ! P1 r12 = x r13 = z Conclusion: c’ is not a safe configuration  the system never reaches a safe configuration The system will reach c’ in which all P2-4 encode 3 Lets consider a system with 4 processors We can stop P4 until P2 and P3 encode 2 P3 P2 2 chapter 6 - Convergence in the Presence of Faults 1-194

Chapter 7 – Local Stabilization
Self-Stabilization Shlomi Dolev MIT Press , 2000 Draft of January 2004 Shlomi Dolev, All Rights Reserved © Chapter 7 - Local Stabilization 195 195

Chapter 7: roadmap 7.1 Superstabilization
7.2 Self-Stabilizing Fault-Containing Algorithms 7.3 Error-Detection Codes and Repair Chapter 7 - Local Stabilization 196

Dynamic System & Self Stabilization
Algorithms for dynamic systems are designed to cope with failures of processors with no global re-initialization. Such algorithms consider only global states reachable from a predefined initial state under a restrictive sequence of failures and attempt to cope with such failures with as few adjustments as possible. Self Stabilization Self-stabilizing algorithms are designed to guarantee a particular behavior finally. Traditionally, changes in the communications graph were ignored. Superstabilizing algorithms combine the benefits of both self-stabilizing and dynamic algorithms Chapter 7 - Local Stabilization 197

Definitions A Superstabilizing Algorithm: Must be self-stabilizing
Must preserve a “passage predicate” Should exhibit fast convergence rate Passage Predicate - Defined with respect to a class of topology changes (A topology change falsifies legitimacy and therefore the passage predicate must be weaker than legitimacy but strong enough to be useful). Chapter 7 - Local Stabilization 198

Passage Predicate - Example
In a token ring: A processor crash can lose the token but still not falsify the passage predicate Passage Predicate Legitimate State At most one token exists in the system. (e.g. the existence of 2 tokens isn’t legal) Exactly one token exists in the system. Chapter 7 - Local Stabilization 199

Evaluation of a Super-Stabilizing Algorithm
Time complexity The maximal number of rounds that have passed from a legitimate state through a single topology change and ends in a legitimate state Adjustment measure The maximal number of processors that must change their local state upon a topology change, in order to achieve legitimacy Chapter 7 - Local Stabilization 200

Motivation for Super-Stabilization
A self-stabilizing algorithm that does not ignore the occurrence of topology changes (“events”) will be initialized in a predefined way and react better to dynamic changes during execution Question: Is it possible, for the algorithm that detects a fault, when it occurs, to maintain a “nearly legitimate” state during convergence? Chapter 7 - Local Stabilization 201

Motivation for Super-Stabilization
While transient faults are rare (but harmful), a dynamic change in the topology may be frequent. Thus, a super-stabilizing algorithm has a lower worst- case time measure for reaching a legitimate state again, once a topology change occurs. In the following slides we present a self-stabilizing and a super-stabilizing algorithm for the graph coloring task. Chapter 7 - Local Stabilization 202

Graph Coloring The coloring task is to assign a color value to each processor, such that no two neighboring processors are assigned the same color. Minimization of the colors number is not required. The algorithm uses Δ+1 colors, where Δ is an upper bound on a processor’s number of neighbors. For example: Chapter 7 - Local Stabilization 203

Graph Coloring - A Self-Stabilzing Algorithm
01 Do forever 02 GColors := 0 03 For m:=1 to δ do 04 lrm:=read(rm) 05 If ID(m)>i then GColors := GColors U lrm.color 07 od 08 If colori GColors then 09 colori:=choose(\\ GColors) 10 Write ri.color := color 11 od Colors of Pi’s neighbors Gather only the colors of neighbors with greater ID than Pi’s. If Pi has the color of one of its neighbors with a higher ID, it chooses another color and writes it. Chapter 7 - Local Stabilization 204

Graph Coloring - Self-Stabilzing Algorithm - Simulation
GColors = { Blue } GColors = {} Phase I Id = 1 Id = 3 Id = 5 Id = 4 Id = 2 Chapter 7 - Local Stabilization 205

GColors = { Blue , green , Red }
Graph Coloring - Self-Stabilzing Algorithm - Simulation GColors = { Green } GColors = {} GColors = { Blue , green , Red } GColors = { Blue , green } Phase II Id = 3 Id = 1 Id = 5 Id = 4 Id = 2 Chapter 7 - Local Stabilization 206

GColors = { Blue , green , Red }
Graph Coloring - Self-Stabilzing Algorithm - Simulation GColors = { Green } GColors = {} GColors = { Blue , green , Red } GColors = { Blue , green } Phase III Id = 3 Id = 1 Id = 5 Id = 4 Id = 2 Stabilized Chapter 7 - Local Stabilization 207

Graph Coloring - Self-Stabilizing Algorithm (continued)
What happens when a change in the topology occurs ? If a new neighbor is added, it is possible that two processors have the same color. It is possible that during convergence every processor will change its color. Example: i=1 GColors {blue} i=1 GColors {red} i=2 GColors {blue} i=2 GColors {red} i=3 GColors {red} i=4 GColors {blue} i=5 GColors ø Stabilized But in what cost ? Chapter 7 - Local Stabilization 208

Graph Coloring – Super-Stabilizing Motivation
Every processor changed its color but only one processor really needed to. If we could identify the topology change we could maintain the changes in its environment. We’ll add some elements to the algorithm: AColor – A variable that collects all of the processor neighbors’ colors. Interrupt section – Identify the problematic area. - A symbol to flag a non-existing color. Chapter 7 - Local Stabilization 209

Graph Coloring – A Super-Stabilizing Algorithm
01 Do forever 02 AColors :=  03 GColors :=  04 For m:=1 to δ do 05 lrm:=read(rm) 06 AColors := AColors U lrm.color 07 If ID(m)>i then GColors := GColors U lrm.color 08 od 09 If colori = ┴ or colori  GColors then 10 colori:=choose(\\ AColors) 11 Write ri.color := color 12 od 13 Interrupt section 14 If recoverij and j > i then 15 Colori := ┴ 16 Write ri.color := ┴ All of Pi neighbors’ colors Activated after a topology change to identify the critical processor recoveri,j is the interrupt which Pi gets upon a change in the communication between Pi and Pj Chapter 7 - Local Stabilization 210

Graph Coloring - Super-Stabilizing Algorithm - Example
Notice that the new algorithm stabilizes faster than the previous one. Let us consider the previous example, this time using the super-stabilizing algorithm: Color4 = r4.color = i=4 GColors = {blue} AColors = {blue,red} Stabilized In O(1). Chapter 7 - Local Stabilization 211

Graph Coloring – Super-Stabilizing Proof
Lemma 1: This algorithm is self-stabilizing. Proof by induction: After the first iteration: The value doesn’t exist in the system. Pn has a fixed value. Assume that Pk has a fixed value i<k<n. If Pi has a Pk neighbor then Pi does not change to Pk’s color, but chooses a different color. Due to the assumptions we get that Pi’s color becomes fixed for 1≤i≤n, so the system stabilizes. Chapter 7 - Local Stabilization 212

Graph Coloring – Super-Stabilizing
Passage Predicate – The color of a neighboring processor is always different in every execution that starts in a safe configuration, in which only a single topology change occurs before the next safe configuration is reached Chapter 7 - Local Stabilization 213

Super-stabilizing vs. Self-Stabilizing
Graph Coloring – Super-Stabilizing Super-stabilizing Time – Number of cycles required to reach a safe configuration following a topology change. Super-stabilizing vs. Self-Stabilizing O(1) O(n) Chapter 7 - Local Stabilization 214

Graph Coloring – Super-Stabilizing
Adjustment Measure – The number of processors that changes color upon a topology change. The super-stabilizing algorithm changes one processor color, the one which had the single topology change Chapter 7 - Local Stabilization 215

Self-Stabilizing Fault-Containing Algorithms
Fault model : Several transient faults in the system. This fault model is less severe than dynamic changes faults, and considers the case where f transient faults occurred, changing the state of f processors. The goal of self-stabilizing fault containing algorithms : From any arbitrary configuration, a safe configuration is reached. Starting from a safe configuration followed by transient faults that corrupt the state of f processors, a safe configuration is reached within O(f) cycles. Chapter 7 - Local Stabilization 217 217

A Self-Stabilizing Algorithm for Fixed Output Tasks
Our Goal: to design a self-stabilizing fault- containing algorithm for fixed-output fixed-input tasks. Fixed Input: the algorithm has a fixed input (like its fixed local topology), Ii will contain the input for processor Pi Fixed Output: the variable Oi will contain the output of processor Pi, the output should not change over time. A version of the update algorithm is a self stabilizing algorithm for any fixed-input fixed-output task. Chapter 7 - Local Stabilization 218

Fixed-output algorithm for Processor Pi
upon a pulse ReadSeti := Ø forall Pj  N(i) do ReadSeti := ReadSeti  read(Processorsj) ReadSeti := ReadSeti \\ <i,*,*> ReadSeti := ReadSeti ++ <*,1,*> ReadSeti := ReadSeti  {<i,0,Ii>} forall Pj  processors(ReadSeti) do ReadSeti := ReadSeti \\ NotMinDist(Pj, ReadSeti) write Processorsi := ConPrefix(ReadSeti) write Oi := ComputeOutput(Inputs(Processorsi)) Chapter 7 - Local Stabilization 219 219

Explaining the Algorithm
The algorithm is a version of the self-stabilizing update algorithm, it has an extra Ii variable in each <id, dis, Ii> tuple which contains the fixed input of the processor Pi Just like in the update algorithm, it is guaranteed that eventually each processor will have a tuple for all other processors Each processor will have all the inputs and will compute the correct output. But is this algorithm fault containing? Chapter 7 - Local Stabilization 220 220

Does this Algorithm have the Fault-containment Property?
Error Scenario (assuming output is OR of inputs): In a safe configuration P5 has a tuple <1,4,0> A fault has occurred and changed it to <1,1,1> Error propagation : P1 P2 P3 P4 P5 I=0 O1=0 O2=0 O3=0 O4=0 O5=0 <1,1,0> <1,2,0> <1,4,0> <1,3,0> 1 O4=0 <1,1,1> <1,3,0> O5=1 2 <1,2,1> O4=1 <1,4,0> O5=0 3 <1,3,1> O4=0 <1,3,0> O5=1 4 O5=0 <1,4,0> cycle : Conclusion : It doesn’t have this property. The system stabilizes only after O(d/2) cycles Chapter 7 - Local Stabilization 221 221

Fault Containment – Naive Approach
A processor that is about to change its output waits for d cycles before it does so This approach ensures : Stabilization from every arbitrary state Starting in a safe configuration followed by several simultaneous transient faults, only the output of the processors that experienced a fault can be changed and each such change is a change to correct output value. During this interval correct input values propagate towards the faulty processor Chapter 7 - Local Stabilization 222 222

Fault Containment – Naive Approach (cont.)
The above approach ensures self- stabilization, but has a serious drawback : The time it takes for the correct input values to propagate to the faulty processors and correct its output is O(d), which contradicts the second requirement of self stabilizing fault-containment This requirement states that a safe configuration is reached within O(f) cycles Lets consider a more sophisticated approach to meet all fault-containment requirements Chapter 7 - Local Stabilization 223 223

Designing a Fault-containing Algorithm
Evidence : The values of all the Ii fields in Processorsi. Each tuple will contain evidence in addition to its other 3 fields The additional Ai variable enables the processors that experienced a fault to learn quickly about the inputs of the remote processors When a processor experiences a fault, the Ai variables of most of the processors within distance 2f+1 or less from it are the set of correct inputs We should maintain this as an invariant throughout the algorithm for a time sufficient to let the faulty processors regain the correct input values Ii of the other processors Chapter 7 - Local Stabilization 224 224

Self-stabilizing Fault-containing Algorithm for Processor Pi
upon a pulse ReadSeti := Ø forall Pj  N(i) do ReadSeti := ReadSeti  read(Processorsj) if (RepairCounteri ≠ d + 1) then //in repair process RepairCounteri := min(RepairCounteri, read(RepairCounterj)) od ReadSeti := ReadSeti \\ <i,*,*,*> ReadSeti := ReadSeti ++ <*,1,*,*> ReadSeti := ReadSeti  <i,0,Ii,Ai> forall Pj  processors(ReadSeti) do ReadSeti := ReadSeti \\ NotMinDist(Pj, ReadSeti) write Processorsi := ConPrefix(ReadSeti) Chapter 7 - Local Stabilization 225 225

Self-stabilizing Fault-containing Algorithm – (cont.)
if (RepairCounteri = d + 1) then //not in repair process if (Oi ≠ ComputeOutputi(Inputs(Processorsi))) or (<*,*,*,A>  Processorsi | A ≠ Inputs(Processorsi)) then RepairCounteri := 0 //repair started else //in repair process RepairCounteri := min(RepairCounteri + 1, d + 1) write Oi := ComputeOutputi(RepairCounteri, MajorityInputs(Processorsi)) if (RepairCounteri = d + 1) then //repair over Ai := Inputs(Processorsi) Chapter 7 - Local Stabilization 226 226

Explaining the Algorithm
Initial state: RepairCounteri = d + 1. A change in the RepairCounteri variable occurs when Pi detects an error in its state, or when an error from another processor propagates towards it during the repair process ComputeOutput is applied on the majority of inputs of processors with distance ≤ RepairCounteri from Pi, thus assuring that eventually the distance 2f+1 is reached and that Oi is set correcly The value of Ai doesn’t change throughout the repair process, thus maintaining the invariant needed for the faulty processors to regain the correct input values and eventually present a correct output Chapter 7 - Local Stabilization 227 227

4 2 1 3 cycle : P1 P2 P3 P4 P5 P6 Error scenario :
A4:i1= 1 <1,1,0,A1> 2 <1,3,0,A1> A2:i1= 1 <1,2,1,A1> A2:i1= 1 A4:i1= 1 1 <1,1,1,A1> <1,3,1,A1> I=0 I=0 I=0 I=0 I=0 I=0 3 <1,2,0,A1> O=1 <1,3,1,A1> O=0 O=0 O=0 O=0 O=0 <1,0,0,A1> <1,1,0,A1> <1,2,0,A1> <1,3,0,A1> <1,4,0,A1> <1,5,0,A1> A2:i1= 0 A1:i1= 0 A4:i1= 0 A2:i1= 0 cycle : A3:i1= 0 A4:i1= 0 A5:i1= 0 Error scenario : Input of P1 at (P2 and P4) erroneously alters to 1 Input of P1 at (A2 and A4) evidences erroneously alters to 1 The erroneous input propagates to P3 and so do the evidences, causing it to calculate erroneous output based on majority inputs at distance ≤ 1 The output is fixed in P3 once the distance grows to ≤ 2. Chapter 7 - Local Stabilization 228 228

Error Scenario Initially the network graph is in a safe configuration
In the first cycle red processors experience several faults : The evidence considering the blue processor is erroneous. The distance to the blue processor and its input are also erroneous. In the next cycles the error propagates throughout the graph The output becomes erroneous at many processors, but convergence back to the safe state is quick Chapter 7 - Local Stabilization 229 229

Feel the Power (example) …
Regular Wrong evidence Source (blue) Wrong output and output … Cycle: 2 3 4 1 (repairCounter) … … Chapter 7 - Local Stabilization 230

Algorithm Analysis Ai of processor Pi stays unchanged for (d+1) cycles since the repair process started, which ensures that the fault factor doesn’t grow The majority-based output calculation is applied during the repair process After 2f+1 cycles the majority of inputs around the faulty processor is correct, applying correct output computation In that manner, after 2f+1 cycles the system’s output stabilizes with correct values, despite the continuing changes in processors’ internal states (for d+1 cycles) Chapter 7 - Local Stabilization 231 231

Conclusions Compared to the naive implementation, this algorithm significantly shortens the system stabilization time The price we pay for this improvement : Network load grows because each tuple includes an additional variable Ai of size O(n) (n – number of processors). During the algorithm execution, faulty output is allowed for non-faulty processors during short periods of time Chapter 7 - Local Stabilization 232 232

Chapter 8: roadmap 8.1 The Computation Power of Self-Stabilizing Systems 8.2 Queue Machine chapter 8 - Self-Stabilizing Computing 235

The Computation Power of Self-Stabilizing Systems
? l 1 l 2 l n l . l . l . Multi processor computers Have potential to be resilient to faults Faults # grows with the # of processors Fault-tolerance is vitally important Synchronization is needed for coordination – clocks Global clock pulse & global clock value Global clock pulse & individual clock values Individual clock pulse & individual clock values Fault tolerant clock synchronization l . l n chapter 8 - Self-Stabilizing Computing 236 236

Modeling the problem The single processor is modeled by a Universal-Turing-Machine* (UTM) The distributed system will simulate the UTM in a self-stabilized fashion The distributed system topology will be a simple chain of n processors, each processor represents a single working cell * note: The use of UTM eliminates the transition table size factor, since we use the code of the algorithm as part of the input The problem is defined for a system of n identical processors connected to a global common clock pulse. Each processor maintains a digital clock value. In every pulse each processor executes a step in which it reads the value of it’s neighbors’ clocks and uses these values to calculate its new clock value . chapter 8 - Self-Stabilizing Computing 237 237

Token passing with a constant amount of memory
The distributed system uses a version of Dijkstra’s second mutual exclusion algorithm: P1 Pn The algorithm simulates a token with a color field of three values chapter 8 - Self-Stabilizing Computing 238

Version of Dijkstra’s mutual exclusion algorithm:
01 P1 02 lr2,1.color  r2,1.color 03 if lr2,1.color  r1,2.color then 04 r1,2.color  (r1,2.color + 1) (mod 3) 01 Pi (i 1, n) 02 lri-1,i.color  ri-1,i.color 03 ri,i+1.color  lri-1,i.color 04 lri+1,i.color  ri+1,i.color 05 if lri+1,i.color  lri-1,i.color then 06 ri,i-1.color  lri+1,i.color 01 Pn 02 lrn-1, n.color  rn-1, n.color 03 rn, n -1.color  lrn-1, n.color chapter 8 - Self-Stabilizing Computing 239

Self-Stabilization of the algorithm
Lemma 8.1: In every fair execution, P1 changes the value of r1,2.color infinitely often Lemma 8.2: In every fair execution, a configuration is reached in which all the color fields of the communication registers and internal variables have identical values Note that for ease of description the values we choose for M are not the minimal possible value The number of clock values can only decrease, and is reduced to a single clock value chapter 8 - Self-Stabilizing Computing 240 240

Self-Stabilization of the algorithm
By using these Lemmas (8.1, 8.2) we can see the algorithm works as expected. Once the safe configuration has been achieved, the token will traverse the chain chapter 8 - Self-Stabilizing Computing 241

A simple running example
Lets look at a system with three processors that start at a random configuration: Ci is the state of the processor (the line in the code it’s about to execute). P1,C1 P2,C1 P3,C2 r1,2 1 lr2,1 2 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 Each step is an execution of some line of code chapter 8 - Self-Stabilizing Computing 242 242

P1 executes line 2: it reads the value of r2,1 into the local variable lr2,1 . P1,C2 P2,C1 P3,C2 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 chapter 8 - Self-Stabilizing Computing 243

P2 executes line 2: it reads the value of r1,2 into the local variable lr1,2 P1,C2 P2,C2 P3,C2 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 Each step is an execution of some line of code chapter 8 - Self-Stabilizing Computing 244 244

P2 executes line 3: it writes the value of lr1,2 into r2,3 P1,C2 P2,C3 P3,C2 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 chapter 8 - Self-Stabilizing Computing 245

P2 executes line 4: it reads the value of r3,2 into lr3,2 .Line 6 is not executed since lr1,2 ≠ lr3,2 . P1,C2 P2,C4 P3,C2 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 chapter 8 - Self-Stabilizing Computing 246

P3 executes line 2: it reads the value of r2,3 into the local variable lr2,3 P1,C2 P2,C4 P3,C2 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 2 lr2,3 1 Each step is an execution of some line of code chapter 8 - Self-Stabilizing Computing 247 247

P3 executes line 3: it writes the value of lr2,3 into r3,2 P1,C2 P2,C4 P3,C3 (a full round) r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 2 r3,2 1 lr2,3 chapter 8 - Self-Stabilizing Computing 248

P2 executes lines 2-6 and we gate a safe configuration… SAFE!!! P1,C2 P2,C2-6 P3,C1 r1,2 1 lr2,1 r2,1 1 r2,3 lr1,2 lr3,2 r3,2 1 lr2,3 Each step is an execution of some line of code chapter 8 - Self-Stabilizing Computing 249 249

The Simulation Each processor represents a working tape cell.
The processor’s input represents the initial configuration of the UTM. The processor’s output represents the UTM’s decision, i.e. output = 1 (0)  UTM accepted (rejected) the input using no more than n working tape cells. chapter 8 - Self-Stabilizing Computing 250

The Simulation Each processor maintains 3 local variables:
HeadMarki (1-bit) – true if the head is currently on the working tape cell this processor simulates. WorkSymboli (2-bit) – the current content of the tape cell. Counteri (2-bit) – used to bound the execution of the simulation, and initialize it from time to time. (will be explained soon) chapter 8 - Self-Stabilizing Computing 251

The number of configurations
A certain UTM exists , thus the number of states C is bounded by 56 [Hopcroft & Ullman 1979] which possesses: n memory bits which can contain 2n different tape contents. n possibilities for the location of the first ⊥. n possibilities for the location of the reading head Thus: cn2 2n is the total number of the UTM’s configurations. Note: for n  14 : 22n  cn2 2n. So a UTM execution that exceeds 22n steps (configurations) must repeat at least one configuration twice, resulting in an infinite loop. How to maintain a distributed counter that count up to 22n ? chapter 8 - Self-Stabilizing Computing 252

System initialization
All processors maintain a distributed counter using their local variable Counteri. The system is initialized when counter overflow occurs. (increasing the counter in the following example will end up in initialization) P3 P2 P1 11 Note that for ease of description the values we choose for M are not the minimal possible value In this example the value of the counter is 63. chapter 8 - Self-Stabilizing Computing 253 253

I am the token, moving left
A Simple Example Rest is 1  there is an overflow I am a processor MSB LSB P3 P2 P1 11 10 I am the token, moving left Carry:0 Reset:0 I am a counter chapter 8 - Self-Stabilizing Computing 254

A Simple Example The start configuration of this example, the token is moving to the left. P3 P2 P1 11 11 10 Carry:0 Reset:0 chapter 8 - Self-Stabilizing Computing 255

A Simple Example P3 P2 P1 11 11 10 Carry:0 Reset:0
chapter 8 - Self-Stabilizing Computing 256

A Simple Example P1 increments the counter without generating a carry
11 11 11 Carry:0 Reset:0 chapter 8 - Self-Stabilizing Computing 258

A Simple Example P1 increments the counter which results in generating a carry P3 P2 P1 11 11 00 Carry:1 Reset:0 chapter 8 - Self-Stabilizing Computing 262

A Simple Example P3 doesn’t have the option to pass on the carry (which means an overflow) and it start a reset. P3 P2 P1 00 00 00 Carry:1 Reset:1 chapter 8 - Self-Stabilizing Computing 264

A Simple Example i ≠ 1:HeadMarki = false Reset finished, and:
HeadMark1 = true i ≠ 1:HeadMarki = false P3 P2 P1 00 00 00 Carry:0 Reset:1 chapter 8 - Self-Stabilizing Computing 266

The Simulation When processor Pi, with HeadMarki = true, receives the token, it computes the next step, i.e. WorkSymboli, State, Right/Left, using: Current state, retrieved from the token. WorkSymboli. UTM’s transition table. The head movement is simulated by sending the token to the Right (or Left) neighbor and changing its HeadMark value to be true. When a processor reaches a decision state it notifies all other to write the correct output. chapter 8 - Self-Stabilizing Computing 267

I am the token, moving left
A Simple Example P1 P2 P3 HeadMark false Working Cell counter 01 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head Ø output HeadMark false Working Cell 1 counter 00 Move head Ø output Set head:true state: q1 I am the token, moving left A token arrives to P2, with “set head = true”. chapter 8 - Self-Stabilizing Computing 268

A Simple Example P1 P2 P3 HeadMark false Working Cell counter 01 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head R output HeadMark false Working Cell 1 counter 00 Move head Ø output P2 consults the UTM transition table, and decides to write 0, and move right. Since the token is moving left now (until it get to the end) P2 will remember to set the HeadMark when the token returns. chapter 8 - Self-Stabilizing Computing 269

Set head:false state: q26
A Simple Example P1 P2 P3 HeadMark false Working Cell counter 11 Move head Ø output HeadMark false Working Cell counter 00 Move head R output HeadMark false Working Cell 1 counter 00 Move head Ø output Set head:false state: q26 P1 is working and increase the counter, but doesn’t calculate a UTM step since the W/R head is not in his cell. chapter 8 - Self-Stabilizing Computing 270

Set head:false state: q26
A Simple Example P1 P2 P3 HeadMark false Working Cell counter 11 Move head Ø output HeadMark false Working Cell counter 00 Move head R output HeadMark false Working Cell 1 counter 00 Move head Ø output Set head:false state: q26 P2 remember that the W/R head should move right so he will change the Set head to be true chapter 8 - Self-Stabilizing Computing 271

A Simple Example P1 P2 P3 Set head:true state: q26 11 Ø 00 Ø 1 00 Ø
HeadMark false Working Cell counter 11 Move head Ø output HeadMark false Working Cell counter 00 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head Ø output Set head:true state: q26 chapter 8 - Self-Stabilizing Computing 272

A Simple Example P1 P2 P3 HeadMark false Working Cell counter 11 Move head Ø output HeadMark false Working Cell counter 00 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head L output Token arrived at P3, with “set head = true”. P3 consults the UTM transition table, and decides to write 1 change the state to 32, and move left (note; the token travels left now!). chapter 8 - Self-Stabilizing Computing 273

A Simple Example P1 P2 P3 Set head:true state: q32 11 Ø 1 00 Ø 00 Ø 1
HeadMark false Working Cell counter 11 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head Ø output HeadMark false Working Cell counter 00 Move head Ø output 1 Set head:true state: q32 chapter 8 - Self-Stabilizing Computing 274

A Simple Example P1 P2 P3 HeadMark false Working Cell counter 11 Move head Ø output HeadMark true Working Cell 1 counter 00 Move head Ø output HeadMark false Working Cell counter 00 Move head Ø output 1 Token arrived at P2, with “set head = true”. P2 consults the UTM transition table, and decides to accept. P2 will place a clear message in the token. chapter 8 - Self-Stabilizing Computing 276

Set head:true state: qaccept
A Simple Example P1 P2 P3 HeadMark false Working Cell counter 11 Move head Ø output HeadMark true Working Cell counter 00 Move head Ø output 1 HeadMark false Working Cell counter 00 Move head Ø output 1 Set head:true state: qaccept chapter 8 - Self-Stabilizing Computing 277

A Simple Example P1 P2 P3 Clear Result:1 00 Ø 1 00 Ø 1 00 Ø 1 HeadMark
false Working Cell counter 00 Move head Ø output 1 HeadMark false Working Cell counter 00 Move head Ø output 1 HeadMark false Working Cell counter 00 Move head Ø output 1 Clear Result:1 chapter 8 - Self-Stabilizing Computing 278

A Simple Example P1 P2 P3 Clear Result:1
HeadMark true Working Cell counter 10 Move head Ø output 1 HeadMark false Working Cell counter 00 Move head Ø output 1 HeadMark false Working Cell counter 00 Move head Ø output 1 Clear Result:1 Now, P1 will start the computation again. chapter 8 - Self-Stabilizing Computing 282

Chapter 8: roadmap 8.1 The Computation Power of Self-Stabilizing Systems 8.2 Queue Machine chapter 8 - Self-Stabilizing Computing 283

What is a Queue Machine? A queue machine Q is a finite state machine equipped with a queue. The queue initially contains a non-empty word from ∑ for some (finite) alphabet ∑ . In each step of the computation, Q performs: Reads and deletes a letter from the head of the queue. Adds zero or more letters from ∑ to the tail of the queue. Changes a state. The computation terminates when the queue becomes empty. + chapter 8 - Self-Stabilizing Computing 284

Token Controller - Definition
Token controller is a special type of a queue machine. Assume that the alphabet ∑ contains a subset T of token letters. A queue machine is a token controller if, starting with a non-empty queue of arbitrary content and in an arbitrary state of the finite-state machine, eventually there is exactly 2 different segments, each of them contains exactly one letter from T. chapter 8 - Self-Stabilizing Computing 285

Token Controller - Definition
The token controller is an abstraction of the token- passing task. The sender is the finite-state machine that receives (finite-length) messages and sends massages in response. The massages in transit form the content of the queue. Token controller messages chapter 8 - Self-Stabilizing Computing 286

Does a Token Controller exist ?
The Controller, if exists, cannot compute any estimation of the number of tokens in the queue. The Controller is finite and the queue may contain any number of messages hence the finite control cannot “remember” what he saw. The solution: all the information needed is stored within the queue. The controller makes decisions while the messages are passing. chapter 8 - Self-Stabilizing Computing 287

Token Controller Algorithm
01 do forever dequeue (color, CounterBit, Token) if color = TokenColor then (*token arrives*) begin if carry = 1 then enqueue (color, 1, false) TokenColor := (color + CounterXor +1)(mod 3) CounterXor := 0 carry := 1 Token := true end CounterXor := CounterXor  CounterBit NewCounterBit := carry  CounterBit carry := carry  CounterBit enqueue (TokenColor, NewCounterBit, Token) Token := false 16 od chapter 8 - Self-Stabilizing Computing 288

The algorithm uses three colors to color the messages it sends. TokenColor - the current color. Each message caries a bit - CounterBit. A segment of messages is a maximal sequence of messages in the queue with identical color. CounterBit1 is the first massage who left the controller. N (segment) - the integer whose binary form is: CounterBiti , CounterBiti -1 … CounterBit1 N=4 1 CounterBit1 is the least significant bit chapter 8 - Self-Stabilizing Computing 289

While a segment passes, the Controller performs: Change the color of each massage to TokenColor . Calculate XOR (N ( segment ) ). Increments N(segment) by binary addition (using the carry). When a segment with TokenColor arrives calculate the new color. chapter 8 - Self-Stabilizing Computing 290

Distributed counter Red = 0 Turquoise= 1 Orange = 2 The current
color of the controller Token color = 1 Carry = 0 counterXor = 0 … The color of the head massage: Color = 1 1 1 1 1 Token color = 2 Carry = 1 counterXor = 1 … chapter 8 - Self-Stabilizing Computing 291

Distributed counter (cont)
Token color = 2 Carry = 1 counterXor = 0 … 1 1 Token color = 0 Carry=1 counterXor = 0 … chapter 8 - Self-Stabilizing Computing 292

Distributed counter (summary)
Token color = 1 Carry = 1 counterXor = 0 … Starting at this configuration we will eventually arrived to configuration without green colored massages 1 1 1 1 … Token color = 1 Carry = 1 counterXor = 0 chapter 8 - Self-Stabilizing Computing 293

Distributed counter (summary)
1 … Token color = 1 Carry = 1 counterXor = 0 And we will eventually arrived to configuration with only one color at the messages. 1 Token color = 1 Carry = 1 counterXor = 0 chapter 8 - Self-Stabilizing Computing 294

pseudo-stabilized configuration
When in some configuration c, a message with TokenColor arrives, we say that c is a pseudo-stabilized configuration. A pseudo-stabilized configuration is a safe configuration when there is only one segment in c. chapter 8 - Self-Stabilizing Computing 295

The Number of Segments S4 S3 S2 S1 S'1 S3 S2 S4 New color = color(S2)
The Number of Segments Preserves chapter 8 - Self-Stabilizing Computing 296

The Number of Segments S4 S3 S2 S1 S1’+S2’ S3 S4 New color = color(S3)
The Number of Segments Reduces chapter 8 - Self-Stabilizing Computing 297

Token Controller, is Self-Stabilizing
The colors of the segments in the first pseudo-configuration define the periodic sequence that must be used to preserve the number of segments. If the sender chooses differently and the number of segments is greater then one, then the number of segments is reduced. Define Xor(x) as the parity of the binary representation of x. XorSeq = xor(0), xor(1), xor(2), xor(3)… = … is eventually aperiodic (Lemma 8.3). chapter 8 - Self-Stabilizing Computing 298

Lemma 8.3: The Infinite Sequence XorSeq is Eventually Aperiodic - proof
l and p numbers such that the suffix of XorSeq that follows the first l elements of XorSeq is periodic with period of length p. Case 1 : xor(p) = 1 . Let l1 = 2k such that 2k-1 > l and l1 > 2p . xor ( l1 + p ) = 0 , xor ( l1 ) = 1. Example: p = 11111, xor ( p ) = 1. l1 = , xor ( l 1) = 1. xor ( l1 + p ) = xor ( ) = 0 l l+p l+2p l1 l+3p … The suffix that follows l has no period p. chapter 8 - Self-Stabilizing Computing 299

Lemma 8.3 – proof (cont) Case 2 : xor (p) = 0 .
Let l2 = 2j such that 2j  p  2j+1 ( 2j is the most significant bit of p ). xor ( p + l2 ) = xor ( p ) = 0 . xor ( l1 + p + l2 ) = 1, while xor ( l1 + l2 ) = 0. Example: p = 11110, xor( p ) = 0. l1 = , l2 = xor ( p + l2 ) = xor(101110) = 0. xor ( l1 + l2 ) = xor ( ) = 0. xor ( l1 + p + l2 ) = xor ( ) = 1. chapter 8 - Self-Stabilizing Computing 300

Token Controller, is Self-Stabilizing
Theorem 8.1 : In any execution in which the number of segments in the pseudo-stabilized configuration is fixed, the sequence of colors produced by the algorithm is aperiodic. Proof : The sequence CounterXor is aperiodic for each segment. If the combined sequence is periodic then each sequence is periodic. CounterXor is aperiodic, so the sequence of colors choose by the algorithm is aperiodic. chapter 8 - Self-Stabilizing Computing 301

Chapter 2 Self-Stabilization

Similar presentations

Presentation on theme: "Chapter 2 Self-Stabilization"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Chapter 2 Self-Stabilization

Similar presentations

Presentation on theme: "Chapter 2 Self-Stabilization"— Presentation transcript:

Similar presentations

About project

Feedback