Convergence Refinement Murat Demirbas Anish Arora The Ohio State University

A basic question in fault-tolerance Is fault-tolerance preserved under program refinement ? In other words, given a high-level program A that is fault-tolerant, is a low-level implementation C of A yielded by a standard compiler or program transformer also fault-tolerant ?

A counterexample: A toy infinite loop int x=0; while( x==x ) { x=0; } This program outputs 0 always –it is live even if x is corrupted 0iconst_0 1istore_1 2goto 7 5iconst_0 6istore_1 7iload_1 8iload_1 9if_icmpeq 5 12return Corresponding byte code can lose liveness –if x is corrupted between lines 7 & 8 Java compiler

Another counterexample: Bid Server High-level server: – maintains best K bids – replaces minimum stored bid x with an incoming bid v iff x<v gracefully tolerates Byzantine failure of single bid location, – maintains (K-1) of best K bids Sorted-list implementation of server: – stores best K bids in increasing order – incoming bid is considered only if it is larger than head of list does not tolerate single Byzantine failure: – e.g., of head of list

So, what went wrong? Standard compilers and transformers yield refinements Intuitively, refinement means that every properly initialized computation of C is is a computation of A Refinements are clearly insufficient, because they don’t take into account states reached in the presence of faults E.g.: s0 s1s2s3 … s* F s0 s1s2s3 … s* F AC

Fault-tolerance refinement isn’t just refinement What happens when compilers / transformers are modified to refine even from non-initialized states?( Everywhere refinement ) Unfortunately, fault-tolerance preserving refinements are not always [ everywhere ] refinements [Leal 00] :  intuitively because in the presence of faults C may not use the identical “recovery” path that A would In this paper, we develop an alternative notion of refinement that  suffices for stabilization fault-tolerance preserving implementations  applies also to masking fault-tolerance preserving implementations

Outline of talk Definition Compositionality  role in wrapper-based design of stabilization An illustration  deriving Dijkstra’s stabilizing programs from simple token rings

Definition: Model A system is an automaton ( ,T,I)   : state space, T : set of transitions, I : set of initial states  Computation is a maximal sequence of states-transitions pairs C is stabilizing to A iff every computation of C has a suffix that is a suffix of some computation of A that starts at an initial state of A Note: C and A may have different state spaces, relative to an abstraction function that is total and onto from C to A

Definition: Convergence refinement [C  A] C is a convergence refinement of A iff C refines A, and every computation c of C is a “compressed” form of some computation a of A Compressed means that c can omit a finite number of states in a, except its initial state and final state (if any) a = s1 s2 s3 s4 s5 s6 c = s1 s4 s6  c = s1 s0 s4 s6 

Convergence refinement preserves stabilization C is a convergence refinement of A  C tracks A from all states with only a finite number of compressions  stabilization of A is preserved in C Theorem [C  A] and A is stabilizing to B  C is stabilizing to B

Outline of talk Definition Compositionality  role in wrapper-based design of stabilization An illustration  deriving Dijkstra’s stabilizing programs from simple token rings

Compositionality of convergence refinement A desirable property of convergence refinement is that if the union of two high-level components is stabilizing, then so is the union of their low-level convergence refinements Lemma If [ C  A ], A  W is stabilizing to A then [ C  W  A  W ] and hence C  W is also stabilizing to A Theorem If [ C  A ], [ W’  W ], A  W is stabilizing to A then [ C  W’  A  W ] and hence C  W’ is also stabilizing to A

Outline of talk Definition Compositionality  role in wrapper-based design of stabilization An illustration  deriving Dijkstra’s stabilizing programs from simple token rings

Deriving Dijkstra’s token ring programs After all these years of staring at Dijkstra’s program, why chose them as the illustration ?  No previous work has been able to derive Dijkstra’s programs as refinements of simple token rings Next Steps: 0 Chose a simple token ring program BTR 1. Add an abstract stabilization wrapper W to BTR 2. Convergence refine BTR into BTR4 and W into W’ BTR  W’ results in Dijkstra’s 4-state token ring program The paper contains derivations for K-state & 3-state programs also

Step 0: Bidirectional token ring, BTR Propagate up-token at j:  t.j   t.j ;  t.(j+1) Bounce up-token at N:  t.N   t.N ;  t.(N-1) Propagate down-token at j  t.j   t.j ;  t.(j-1) Bounce down-token at 0  t.0   t.0 ;  t.1

Step 1: Stabilization wrappers for BTR W1 ensures that there exists at least one token in the system (  j : j  N:  t.j   t.j )   t.N W2 ensures eventually at most one token exists in the system  t.j   t.j   t.j ;  t.j (BTR  W1  W2) is stabilizing to BTR Notice that W1 accesses to global state

Step 2: A 4 state implementation How to localize W1 ?  Implement bits  t.j and  t.j in terms of 2 bits: up.j and c.j  up.0 := true and up.N=false ensures that in any state there exists two adjacent processes that point to each other Up-token at j :  j and (j-1) point to each other  c values at j and (j-1) differ  formally,  t.j  c.j  c.(j-1)  up.(j-1)   up.j Down-token at j :  j and (j+1) point to each other  c values at j and j+1 are the same  formally,  t.j  c.j=c.(j+1)   up.(j+1)  up.j

BTR after the mapping Propagate up-token at j c.j  c.(j-1)  up.(j-1)   up.j  c.j = c.(j-1); up.j; c.(j+1)  c.j   up.(j+1) Bounce up-token at N c.N  c.(N-1)  up.(N-1)  c.N = c.(N-1); up.(N-1) Propagate down-token at j c.j=c.(j+1)   up.(j+1)  up.j   up.j; c.(j-1) = c.j  up.(j-1) Bounce down-token at 0 c.0=c.1   up.1  c.0 =  c.0;  up.1 Propagate up-token at j:  t.j   t.j ;  t.(j+1) Bounce up-token at N:  t.N   t.N ;  t.(N-1) Propagate down-token at j  t.j   t.j ;  t.(j-1) Bounce down-token at 0  t.0   t.0 ;  t.1

Wrappers after the mapping Applying the mapping to W1, we get (  j : j  N: up.j )  c.(N-1)  c.N  c.N=  c.(N-1)  up.(N-1) The guard already implies the statement, hence W1 is already implemented after the mapping Since (  t.j   t.j  false), W2 is also vacuously implemented after the mapping So, the mapped BTR is trivially stabilizing

BTR4: compressing the high atomicity actions Unfortunately, some of the actions in BTR are high atomicity, where j writes the state of j-1 or j+1, e.g. Propagate down-token at j c.j=c.(j+1)   up.(j+1)  up.j   up.j; c.(j-1) = c.j  up.(j-1) To reduce the atomicity, we convergence refine these actions by omitting write accesses to both neighbors Propagate down-token at j c.j=c.(j+1)   up.(j+1)  up.j   up.j // c.(j-1) = c.j  up.(j-1)

BTR4 is a convergence refinement of BTR Token at process 2 disappears after a propagate down-token action at process 2 Token at process 2 disappears only after two steps c.0=1  up.1 c.1=1 up.2 c.2=1 c.3=1 c.0=1  up.1 c.1=1  up.2 c.2=1 c.3=1  t.0  t.2  t.1  t.0

BTR4 Propagate up-token at j c.j  c.(j-1) //  up.(j-1)   up.j  c.j = c.(j-1); up.j // c.(j+1)  c.j   up.(j+1) Bounce up-token at N c.N  c.(N-1) //  up.(N-1)  c.N = c.(N-1) // up.(N-1) Propagate down-token at j c.j=c.(j+1)   up.(j+1)  up.j   up.j // c.(j-1) = c.j  up.(j-1) Bounce down-token at 0 c.0=c.1   up.1  c.0 =  c.0 //  up.1

BTR4  W1’  W2’ = Dijkstra’s 4-state system Up-token actions c.j  c.(j-1)  c.j = c.(j-1); up.j; c.N  c.(N-1)  c.N = c.(N-1); Down-token actions c.j=c.(j+1)   up.(j+1)  up.j   up.j; c.0=c.1   up.1  c.0 =  c.0;

Concluding remarks One step in simplifying fault-tolerance implementation is to derive wrappers based not on the concrete code but on the abstract specification:  Resettable vector clocks [PODC’00]  Dependable home networking: Aladdin [IPDPS’02] We are developing tools and frameworks for mechanical, reusable synthesis of wrappers from abstract specifications The second step is to use transformers that refine fault-tolerance :  Everywhere-eventually refinements [ICDSN’01]  AP to APC compiler [McGuire, Gouda] Since most such transformers still depend on the semantics of the input, we are developing fault-tolerance preserving compilers that depend only on syntax

Compositionality enables wrapper-based design Wrapper-based design is desirable for a posteriori or dynamic addition of stabilization Compositionality implies that a wrapper W designed to add stabilization to A also adds stabilization to any convergence refinement C of A  stabilization of C is achieved oblivious of implementation details of C Moreover, any convergence refinement W’ of W also makes C stabilizing  implementation of W can be oblivious of implementation of A

Continuous maintenance of network services As one example, consider routing in sensor networks; it is required to dynamically adapt the fault-tolerance mechanisms to new environments We have built a.NET framework for dynamic composition of fault-tolerance wrappers We are using the Austin Protocol Compiler [Gouda, McGuire] that yields C- code + UDP socket based communication while preserving certain stabilization properties to get concrete wrappers e.g., A–Routing C–Routing APC W-ABP W’–ABP APC dynamically