Aran Bergman Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Recitation 5: Byzantine Synchronous Consensus Spring 2009 Alex Shraer

Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Byzantine Synchronous Consensus נדיח את מרינה נדיח את גיא מרינה גיא

Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Model Round-based synchronous –Send messages to any set of processes; –Receive messages from this round; –Do local processing (possibly decide, halt) Static set P = {p 1, …, p n } of processes t-out-of-n Byzantine (arbitrary) failures Authentication Messages between correct processes cannot be lost

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Validity and Byzantine Failures Validity – Decision is input of one process Why is that a problem when Byzantine failures can occur? –What is the input of a Byzantine process? Why would we be ok with deciding on this input –A Byzantine leader can lie about its input Strong unanimity - If the input of all correct processes is v then no correct process decides a value other than v

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Weak Unanimity Weak Unanimity: If the input of all the correct processes is v and no process fails then no correct process decides a value other than v We will next see an algorithm for t<n with authentication and Weak Unanimity

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Algorithm (for any t<n) The proposed algorithm is not symmetric (not all processes use the same rules) –One process, p 1, is defined as the leader –Leader’s input – v 1 There is a “default” value, known a-priori v default  {possible decision values}

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Algorithm Process p 1 : SendBuffer = {v 1 }; All other procs: SendBuffer = { } In every round 1 ≤ k ≤ t+1 do –For every message m in SendBuffer Send pi to all the processes that did not sign m –Clear SendBuffer –Receive round k messages –For every received message m, if m has k different valid signatures beginning with p 1 ’s Valid = Valid  {v}, where v is the value received in m SendBuffer = SendBuffer  {m} if Valid contains exactly one value, decide it else decide v default Proof of Termination – trivial In the proof, we will call such messages legitimate

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Proof of Weak Unanimity Weak Unanimity: If the input of all the correct processes is v and no process fails then no correct process decides a value other than v If p 1 (the leader) is correct –All correct processes get v 1 in round 1 and insert into Valid –No other value is inserted into Valid only messages beginning with p 1 ’s signature are considered processes cannot forge leader’s signature –All correct processes decide on v 1 If p 1 is not correct –Weak Unanimity requires nothing

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Why don’t we get Strong Unanimity from this algorithm? Strong unanimity - If the input of all correct processes is v then no correct process decides a value other than v If p 1 is Byzantine, it can send the same value to all processes, but this value can be different than that of correct processes

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Why do we need t+1 rounds? Suppose that a correct process receives a value at the end of the last round and no other correct process has this value… –Can this happen if there are t rounds? –Can this happen if there are t+1 rounds?

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Agreement Lemma: For every two correct processes p i and p j, if v  Value i at the end of round t+1, then v  Value j at the end of round t+1 –i.e., Valid sets of correct processes are the same Then, agreement follows –if the sets are empty or contain more than one value, every correct process decides v default –Otherwise all correct processes decide on the single value in Valid

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Proving the Lemma Lemma: For every two correct processes p i and p j, if v  Value i at the end of round t+1, then v  Value j at the end of round t+1 –i.e., Valid sets of correct processes are the same Consider a correct process p i Suppose that v  Value i at the end of round t+1 –When was v added to Value i ? Denote this round by k –There are two cases: k ≤ t and k = t+1 –We need to prove that by the end of round t+1, v  Value j for every correct client p j –Note: v was a legitimate value when p i received it in round k

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring 2006 Proof of Lemma Case 1: k ≤ t –Since links from p i to other correct processes cannot lose messages, all correct processes receive v by round k+1 and add it to Valid if its not already there Case 2: k = t+1 The first t processes that signed the message must be faulty –Otherwise, p i would receive v in an earlier round from a correct process the last process p that signed the message is correct –v is a legitimate message received in round t+1, thus all t+1 signatures on v are different. But there are only t faulty processes p received v in round t From Case 1 we know that all correct processes receive v by round t+1 and add it to Valid if its not already there

Q1 from HW2 – part (b) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring p1p1 p2p2 p3p3 m1m1 m2m2 deliver(m 1 ) deliver(m 2 ) bcast(m 1 ) bcast(m 2 ) Prove that in the absence of failures a broadcast algorithm that guarantees FIFO + Total Order - also guarantees Causal Order First, the broadcast must be RELIABLE. Otherwise, the statement is not true. Counter example: FIFO is trivially preserved here since each process bcasts only one message TOTAL order is trivially preserved, since only one process delivers 2 messages Causal is not preserved! Is this execution reliable?

Q1 HW2 – part (c) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Does this claim hold when there are failures in the system? The claim doesn’t hold: p 3 never delivers m 1 - this is allowed by the Validity and agreement of reliable broadcast since p 1 faulty. It is also allowed by Agreement of reliable broadcast because p 1 and p 2 are faulty and therefore p 3 can deliver different messages Need to explain why 3 properties of Reliable Broadcast preserved Need to explain why FIFO and TOTAL order are preserved Need to explain why Causal order is violated p1 p2 p3 m1m1 m2m2 bcast(m 2 ) bcast(m 1 ) deliver(m 2 )

Q1 from HW2 – part (a) Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Suppose that m →m’. This means that bcast(m) → bcast(m’). This means that there exists a chain of bcast-deliver events Induction on the number deliver events in the chain from m* to m’. If there are 0 deliver events, the claim holds by FIFO Assume for k, and lets show for k+1: Observe the last sub-chain of the form: Since all processes are correct, by Validity of the Reliable Broadcast, p 2 delivers m’. From Integrity, this is after deliver(m 1 ) From TOTAL, all processes deliver m 1 and m’ in the same order as p 2. m* is delivered before m 1 at all processes (induction assumption) => m* is delivered before m’. If m  m*, from FIFO m is delivered before m* bcast(m) deliver(m*)bcast(m’’) deliver(m’’) bcast(m’) deliver(m 1 ) bcast(m’) bcast(m 1 ) p1p1 p2p2 bcast(m*)

Q3 from HW2 1. Initially: 2. TS[j] ← 0 for all 1≤j≤ n /* array of integers */ 3. pending ← empty /* set of messages */ 4. abcast(msg): 5. TS[i] ← TS[i] bcast( msg, 〈 TS[i], i 〉 ) 7. upon recv( msg, 〈 ts, j 〉 ): 8. add( pending, ( msg, 〈 ts, j 〉 ) ) 9. TS[j] ← ts 10. TS[i] ← max( TS[i], ts ) 11. forever do 12. let ( msg, 〈 ts, j 〉 ) be the entry in pending with the smallest 〈 t, j 〉 13. if 〈 ts, j 〉 ≤ 〈 TS[k], k 〉 for all 1≤k≤ n then 14. remove( pending, ( msg, 〈 ts, j 〉 ) ) 15. adeliver( msg ) האלגוריתם לעיל נבדל מאלגוריתם LTS שנלמד בכיתה בשורות 5 ו 10 Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring m2m1 p1 p2 p Delivery according to the new algorithm Q3 from HW2 – part (a)

Aran Bergman/Eddie Bortnikov, Principles of Reliable Distributed Systems, Technion EE, Spring Q3 from HW2 – part (c) The original LTS would deliver m1 at time 7 When would it deliver m2?