1. Aran Bergman Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 1 Principles of Reliable Distributed Systems Recitation 5: Byzantine Synchronous Consensus Spring 2009 Alex Shraer

2. Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring 2008 2 Byzantine Synchronous Consensus נדיח את מרינה נדיח את גיא מרינה גיא

3. Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring 2008 3 Model Round-based synchronous –Send messages to any set of processes; –Receive messages from this round; –Do local processing (possibly decide, halt) Static set P = {p 1, …, p n } of processes t-out-of-n Byzantine (arbitrary) failures Authentication Messages between correct processes cannot be lost

4. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 4 Validity and Byzantine Failures Validity – Decision is input of one process Why is that a problem when Byzantine failures can occur? –What is the input of a Byzantine process? Why would we be ok with deciding on this input –A Byzantine leader can lie about its input Strong unanimity - If the input of all correct processes is v then no correct process decides a value other than v

5. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 5 Weak Unanimity Weak Unanimity: If the input of all the correct processes is v and no process fails then no correct process decides a value other than v We will next see an algorithm for t<n with authentication and Weak Unanimity

6. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 6 Algorithm (for any t<n) The proposed algorithm is not symmetric (not all processes use the same rules) –One process, p 1, is defined as the leader –Leader’s input – v 1 There is a “default” value, known a-priori v default  {possible decision values}

7. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 7 Algorithm Process p 1 : SendBuffer = {v 1 }; All other procs: SendBuffer = { } In every round 1 ≤ k ≤ t+1 do –For every message m in SendBuffer Send pi to all the processes that did not sign m –Clear SendBuffer –Receive round k messages –For every received message m, if m has k different valid signatures beginning with p 1 ’s Valid = Valid  {v}, where v is the value received in m SendBuffer = SendBuffer  {m} if Valid contains exactly one value, decide it else decide v default Proof of Termination – trivial In the proof, we will call such messages legitimate

8. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 8 Proof of Weak Unanimity Weak Unanimity: If the input of all the correct processes is v and no process fails then no correct process decides a value other than v If p 1 (the leader) is correct –All correct processes get v 1 in round 1 and insert into Valid –No other value is inserted into Valid only messages beginning with p 1 ’s signature are considered processes cannot forge leader’s signature –All correct processes decide on v 1 If p 1 is not correct –Weak Unanimity requires nothing

9. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 9 Why don’t we get Strong Unanimity from this algorithm? Strong unanimity - If the input of all correct processes is v then no correct process decides a value other than v If p 1 is Byzantine, it can send the same value to all processes, but this value can be different than that of correct processes

10. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 10 Why do we need t+1 rounds? Suppose that a correct process receives a value at the end of the last round and no other correct process has this value… –Can this happen if there are t rounds? –Can this happen if there are t+1 rounds?

11. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 11 Agreement Lemma: For every two correct processes p i and p j, if v  Value i at the end of round t+1, then v  Value j at the end of round t+1 –i.e., Valid sets of correct processes are the same Then, agreement follows –if the sets are empty or contain more than one value, every correct process decides v default –Otherwise all correct processes decide on the single value in Valid

12. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 12 Proving the Lemma Lemma: For every two correct processes p i and p j, if v  Value i at the end of round t+1, then v  Value j at the end of round t+1 –i.e., Valid sets of correct processes are the same Consider a correct process p i Suppose that v  Value i at the end of round t+1 –When was v added to Value i ? Denote this round by k –There are two cases: k ≤ t and k = t+1 –We need to prove that by the end of round t+1, v  Value j for every correct client p j –Note: v was a legitimate value when p i received it in round k

13. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 Proof of Lemma Case 1: k ≤ t –Since links from p i to other correct processes cannot lose messages, all correct processes receive v by round k+1 and add it to Valid if its not already there Case 2: k = t+1 The first t processes that signed the message must be faulty –Otherwise, p i would receive v in an earlier round from a correct process the last process p that signed the message is correct –v is a legitimate message received in round t+1, thus all t+1 signatures on v are different. But there are only t faulty processes p received v in round t From Case 1 we know that all correct processes receive v by round t+1 and add it to Valid if its not already there

14. Q1 from HW2 – part (b) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 14 p1p1 p2p2 p3p3 m1m1 m2m2 deliver(m 1 ) deliver(m 2 ) bcast(m 1 ) bcast(m 2 ) Prove that in the absence of failures a broadcast algorithm that guarantees FIFO + Total Order - also guarantees Causal Order First, the broadcast must be RELIABLE. Otherwise, the statement is not true. Counter example: FIFO is trivially preserved here since each process bcasts only one message TOTAL order is trivially preserved, since only one process delivers 2 messages Causal is not preserved! Is this execution reliable?

15. Q1 HW2 – part (c) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 15 Does this claim hold when there are failures in the system? The claim doesn’t hold: p 3 never delivers m 1 - this is allowed by the Validity and agreement of reliable broadcast since p 1 faulty. It is also allowed by Agreement of reliable broadcast because p 1 and p 2 are faulty and therefore p 3 can deliver different messages Need to explain why 3 properties of Reliable Broadcast preserved Need to explain why FIFO and TOTAL order are preserved Need to explain why Causal order is violated p1 p2 p3 m1m1 m2m2 bcast(m 2 ) bcast(m 1 ) deliver(m 2 )

16. Q1 from HW2 – part (a) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 16 Suppose that m →m’. This means that bcast(m) → bcast(m’). This means that there exists a chain of bcast-deliver events Induction on the number deliver events in the chain from m* to m’. If there are 0 deliver events, the claim holds by FIFO Assume for k, and lets show for k+1: Observe the last sub-chain of the form: Since all processes are correct, by Validity of the Reliable Broadcast, p 2 delivers m’. From Integrity, this is after deliver(m 1 ) From TOTAL, all processes deliver m 1 and m’ in the same order as p 2. m* is delivered before m 1 at all processes (induction assumption) => m* is delivered before m’. If m  m*, from FIFO m is delivered before m* bcast(m) deliver(m*)bcast(m’’) deliver(m’’) bcast(m’) deliver(m 1 ) bcast(m’) bcast(m 1 ) p1p1 p2p2 bcast(m*)

17. Q3 from HW2 1. Initially: 2. TS[j] ← 0 for all 1≤j≤ n /* array of integers */ 3. pending ← empty /* set of messages */ 4. abcast(msg): 5. TS[i] ← TS[i] + 5 6. bcast( msg, 〈 TS[i], i 〉 ) 7. upon recv( msg, 〈 ts, j 〉 ): 8. add( pending, ( msg, 〈 ts, j 〉 ) ) 9. TS[j] ← ts 10. TS[i] ← max( TS[i], ts ) 11. forever do 12. let ( msg, 〈 ts, j 〉 ) be the entry in pending with the smallest 〈 t, j 〉 13. if 〈 ts, j 〉 ≤ 〈 TS[k], k 〉 for all 1≤k≤ n then 14. remove( pending, ( msg, 〈 ts, j 〉 ) ) 15. adeliver( msg ) האלגוריתם לעיל נבדל מאלגוריתם LTS שנלמד בכיתה בשורות 5 ו 10 Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 17

18. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 18 0 0 0 5 5 5 5 10 5 15 15 10 5515 m2m1 p1 p2 p3 99 -- - - Delivery according to the new algorithm Q3 from HW2 – part (a)

19. Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 19 0 0 0 1 1 1 3 Q3 from HW2 – part (c) The original LTS would deliver m1 at time 7 When would it deliver m2?