1. DISTRIBUTED ALGORITHMS Spring 2014 Prof. Jennifer Welch Set 9: Fault Tolerant Consensus 1

2. Processor Failures in Message Passing Set 9: Fault Tolerant Consensus 2  Crash: at some point the processor stops taking steps  at the processor's final step, it might succeed in sending only a subset of the messages it is supposed to send  Byzantine: processor changes state arbitrarily and sends messages with arbitrary content

3. Consensus Problem Set 9: Fault Tolerant Consensus 3  Every processor has an input.  Termination: Eventually every nonfaulty processor must decide on a value.  decision is irrevocable!  Agreement: All decisions by nonfaulty processors must be the same.  Validity: If all inputs are the same, then the decision of a nonfaulty processor must equal the common input.

4. Examples of Consensus Set 9: Fault Tolerant Consensus 4  Binary inputs:  input vector 1,1,1,1,1 decision must be 1  input vector 0,0,0,0,0 decision must be 0  input vector 1,0,0,1,0 decision can be either 0 or 1  Multi-valued inputs:  input vector 1,2,3,2,1 decision can be 1 or 2 or 3

5. Overview of Consensus Results Set 9: Fault Tolerant Consensus 5  Synchronous system  At most f faulty processors  Tight bounds for message passing: crash failuresByzantine failures number of rounds f + 1 total number of processors f + 13f + 1 message sizepolynomial

6. Overview of Consensus Results Set 9: Fault Tolerant Consensus 6  Impossible in asynchronous case.  Even if we only want to tolerate a single crash failure.  True both for message passing and shared read- write memory.

7. Modeling Crash Failures Set 9: Fault Tolerant Consensus 7  Modify failure-free definitions of admissible execution to accommodate crash failures:  All but a set of at most f processors (the faulty ones) taken an infinite number of steps.  In synchronous case: once a faulty processor fails to take a step in a round, it takes no more steps.  In a faulty processor's last step, an arbitrary subset of the processor's outgoing messages make it into the channels.

8. Modeling Byzantine Failures Set 9: Fault Tolerant Consensus 8  Modify failure-free definitions of admissible execution to accommodate Byzantine failures:  A set of at most f processors (the faulty ones) can send messages with arbitrary content and change state arbitrarily (i.e., not according to their transition functions).

9. Consensus Algorithm for Crash Failures Set 9: Fault Tolerant Consensus 9 Code for each processor: v := my input at each round 1 through f+1: if I have not yet sent v then send v to all wait to receive messages for this round v := minimum among all received values and current value of v if this is round f+1 then decide on v

10. Execution of Algorithm Set 9: Fault Tolerant Consensus 10  round 1: Relation to Formal Model  send my input in channels initially  receive round 1 msgs deliver events  compute value for v compute events  round 2:  send v (if this is a new value) due to previous compute events  receive round 2 msgs deliver events  compute value for v compute events  …  round f + 1:  send v (if this is a new value) due to previous compute events  receive round f + 1 msgs deliver events  compute value for v compute events  decide v part of compute events

11. Correctness of Crash Consensus Algorithm Set 9: Fault Tolerant Consensus 11 Termination: By the code, finish in round f+1. Validity: Holds since processors do not introduce spurious messages: if all inputs are the same, then that is the only value ever in circulation.

12. Correctness of Crash Consensus Algorithm Set 9: Fault Tolerant Consensus 12 Agreement:  Suppose in contradiction p j decides on a smaller value, x, than does p i.  Then x was hidden from p i by a chain of faulty processors:  There are f + 1 faulty processors in this chain, a contradiction. q1q1 q2q2 qfqf q f+1 pjpj pipi round 1 round 2 round f round f+1 …

13. Performance of Crash Consensus Algorithm Set 9: Fault Tolerant Consensus 13  Number of processors n > f  f + 1 rounds  at most n 2 |V| messages, each of size log|V| bits, where V is the input set.

14. Lower Bound on Rounds Set 9: Fault Tolerant Consensus 14 Assumptions:  n > f + 1  every processor is supposed to send a message to every other processor in every round  Input set is {0,1}

15. Failure-Sparse Executions Set 9: Fault Tolerant Consensus 15  Bad behavior for the crash algorithm was when there was one crash per round.  This is bad in general.  A failure-sparse execution has at most one crash per round.  We will deal exclusively with failure-sparse executions in this proof.

16. Valence of a Configuration Set 9: Fault Tolerant Consensus 16  The valence of a configuration C is the set of all values decided by a nonfaulty processor in some configuration reachable from C by an admissible (failure-sparse) execution.  Bivalent: set contains 0 and 1.  Univalent: set contains only one value  0-valent or 1-valent

17. Valence of a Configuration Set 9: Fault Tolerant Consensus 17 C EFGD 0 0 0 1 1 1 1 0 0 1  decisions 0/1 : bivalent 1 : 1-valent 0 : 0-valent 0/1 0 1

18. Statement of Round Lower Bound Set 9: Fault Tolerant Consensus 18 Theorem (5.3): Any crash-resilient consensus algorithm requires at least f + 1 rounds in the worst case. Proof Strategy: show  bivalent initial config. … round 1 round 2 round f - 2 round f - 1 show we can keep things bivalent through round f - 1 round f show we can keep a n.f. proc. from deciding in round f

19. Existence of Bivalent Initial Config. Set 9: Fault Tolerant Consensus 19  Suppose in contradiction all initial configurations are univalent. inputsvalency 000…000 000…01? 000…11? … 001…11? 011…11? 111…111 by validity condition 0 1

20. Existence of Bivalent Initial Config. Set 9: Fault Tolerant Consensus 20  Let  I 0 be a 0-valent initial config  I 1 be a 1-valent initial config  s.t. they differ only in p i 's input I0I0  p i fails initially, no other failures. By termination, eventually rest decide. all but p i decide 0 I1I1  This execution looks the same as the one above to all the processors except p i. all but p i decide 0

21. Keeping Things Bivalent Set 9: Fault Tolerant Consensus 21  Let  ' be a (failure-sparse) k - 1 round execution ending in a bivalent config.  for k - 1 < f - 1  Show there is a one-round (f-s) extension  of  ' ending in a bivalent config.  so  has k < f rounds  Suppose in contradiction every one-round (f-s) extension of  ' is univalent.

22. Keeping Things Bivalent Set 9: Fault Tolerant Consensus 22 '' failure-free round k 1-val p i crashes 0-val p i fails to send to  p i fails to send to q 1,…,q m p i fails to send to q 1,…,q j+1 p i fails to send to q 1,…,q j rounds 1 to k-1 1-val 0-val bi- val … … now focus in on these two extensions

23. Keeping Things Bivalent Set 9: Fault Tolerant Consensus 23 '' 1-val 0-val p i fails to send to q 1,…,q j p i fails to send to q 1,…,q j+1 rounds 1 to k-1 round k  n.f. decide 1 n.f. decide 1 q j+1 fails in rd. k+1; no other failures  only q j+1 can tell difference Since k-1 < f-1 and α’ is failure-sparse, less than f-1 procs fail in α’. Even with p i failing in round k, less than f procs have failed. So we can have q j+1 fail in round k+1 without exceeding our budget of f failures.

24. Cannot Decide in Round f Set 9: Fault Tolerant Consensus 24  We've shown there is an f - 1 round (failure-sparse) execution, call it , ending in a bivalent configuration.  Extending this execution to f rounds might not preserve bivalence.  However, we can keep a processor from explicitly deciding in round f, thus requiring at least one more round (f+1).

25. Cannot Decide in Round f Set 9: Fault Tolerant Consensus 25 Case 1: There is a 1-round (f-s) extension of  ending in a bivalent config. Then we are done. Case 2: All 1-round (f-s) extensions of  end in univalent configs.

26. Cannot Decide in Round f Set 9: Fault Tolerant Consensus 26  0-val p i fails to send to nf p j rounds 1 to f-1 1-val round f failure free bi- val. p i fails to send to nf p j, sends to another nf p k p i might send to p k p i sends to p j and p k look same to p k look same to p j p k either undecided or decided 1 p j either undecided or decided 0