1. 1 Resilience by Distributed Consensus : Byzantine Generals Problem Adapted from various sources by: T. K. Prasad, Professor Kno.e.sis : Ohio Center of Excellence in Knowledge-enabled Computing Department of Computer Science and Engineering Wright State University, Dayton, OH-45435

2. Resiliency In computer networking: “Resiliency is the ability to provide and maintain an acceptable level of service in the face of faults and challenges to normal operation.” 2

3. Motivation  Coping with failures in computer systems Failed component stops working Failed component sends conflicting information to different parts of a system. (Byzantine Fault)  Agreement in the presence of faults. Managing redundancy in P2P Networks (“coherence”).  Failures can be non-malicious (due to faults) or malicious (as a result of being attacked and compromised) 3

4. Motivation  Build reliable systems in presence of faulty components.  Common approach: Send request (or input) to some “f-tolerant” server Have multiple (potentially faulty) components compute the same function Perform majority vote on outputs to get the “right” result C1 C2 C3 majority(v1,v2,v3) f faulty, f+1 good components ==> 2f+1 total

5. Assumptions for F-tolerant Servers  For majority voting (for consensus) to work: 1) All non-faulty processors must use same input to compute same output. 2) If input is non-faulty, then all non-faulty processors compute same correct output. C1 C2 C3 A B

6. What is a Byzantine Failure?  Three primary differences from Fail- Stop Failure 1) Component can produce arbitrary output Fail-stop: produces correct output or none 2) Cannot always detect that output is faulty Fail-stop: can always detect that component has stopped 3) Components may work together maliciously (collusion)

7. 7 The Byzantine Generals Problem: Distributed Consensus Let us assume we have five generals…

8. 8 The Byzantine Generals Let us assume one is malicious…

9. 9 The Byzantine Generals Each local general decides on an attack plan… 0 1 2 3 4

10. 10 The Byzantine Generals … and accurately relays their plan …

11. 11 The Byzantine Generals …except the random malicious one…

12. 12 The Byzantine Generals Each general collects his or her votes… 0 1 2 3 4

13. 13 The Byzantine Generals Assume each general takes the majority vote…

14. 14 The Byzantine Generals The generals now move based upon their ‘ agreed ’ orders…

15. 15 The Byzantine Generals Since less than half of the military attacked, the military attack failed… 0 2 1 3 4

16. 16 The Byzantine Generals What ’ s more troubling is that: the remaining loyal nodes do n o t know which node(s) among them are disloyal. #0#1#2#3#4 #1 #3 #4

17. Reduction of General Problem Insight: We can restrict ourselves to the problem of one general sending its order to others. Byzantine Generals Problem (BGP): A commanding general (commander) must send an order to his n-1 lieutenants. Interactive Consistency Conditions: IC1: All loyal lieutenants obey the same order. IC2: If the commanding general is loyal, then every loyal lieutenant obeys the order he sends. Note: If General is loyal, IC2 => IC1. Original problem: each general sends his value v(i) by using the above solution, with other generals acting as lieutenants.

18. 18 The Byzantine Generals Let us replan the attack…

19. 19 The Byzantine General Problem Let the generals decide for one to be the leader and others to simply be lieutenants. I ’ ll be the general!

20. 20 The Byzantine General Problem Now the general plans the attack…

21. 21 The Byzantine General Problem The general sends out his or her order to all lieutenants…

22. 22 The Byzantine General Problem Each site records the message they received…

23. 23 The Byzantine General Problem Each site now sends the attack plan they ’ ve received to the other sites…

24. 24 The Byzantine General Problem Again, each site records all messages received…

25. 25 The Byzantine General Problem This process may continue for any number of rounds*, but we ’ ll stop here for now…

26. 26 The Byzantine General Problem Each site finds the majority value of its final round…

27. 27 The Byzantine General Problem Result: All loyal nodes agree on the same result!

28. 28 The Byzantine General Problem  What assumptions were made? A1: Every message sent was delivered correctly. …as we didn ’ t see:

29. 29 The Byzantine General Problem  What assumptions were made? A1: Every message sent was delivered correctly. A2: The receiver of the message knows who sent it. …as we didn ’ t see:

30. 30 The Byzantine General Problem  What assumptions were made? A1: Every message sent was delivered correctly. A2: The receiver of the message knows who sent it. A3 ’ : All sites sent a message. …as we didn ’ t see: ???

31. 31 The Byzantine General Problem  What assumptions were made? A1: Every message sent was delivered correctly. A2: The receiver of the message knows who sent it. A3: The absence of a message can be detected. …so we might see: No message for me… :( …a pre-defined default value may be used…

32. 32 The Byzantine General Problem How many disloyal troops can we have and still reach consensus?

33. 33 The Byzantine General Problem Pre-determined general creates an attack plan…

34. 34 The Byzantine General Problem Round 1: Send out messages; Record

35. 35 The Byzantine General Problem Round 2: Send out messages; Record

36. 36 The Byzantine General Problem Round 3: Send out messages; Record

37. 37 The Byzantine General Problem Seems like 1 disloyal troop with 2 loyal troops works…

38. 38 The Byzantine General Problem …but does it?

39. 39 The Byzantine General Problem Round 1: Send out messages; Record

40. 40 The Byzantine General Problem Round 2: Send out messages; Record

41. 41 The Byzantine General Problem Round 3: Send out messages; Record

42. 42 The Byzantine General Problem Round 4: Send out messages; Record

43. 43 The Byzantine General Problem Round 5: Send out messages; Record

44. 44 The Byzantine General Problem Round 6: Send out messages; Record

45. General Impossibility In general, no solutions with fewer than 3m+1 generals can cope with m traitors. Proof by contradiction. Assume there is a solution for 3m Albanians with m traitors. Reduce to 3-General problem. - Solution to 3m problem => Solution to 3-General problem!!

46. 46 The Byzantine General Problem  Lamport shows (by proof): For a system of n+1 nodes, there cannot exist more than n/3 faulty nodes. Alternatively:  There must be more than 3m troops in any army with up to m traitors (that is, at least 2m + 1 loyal troops).

47. 47 The Byzantine General Problem  General Proof Outline: Pair two loyal troops with each disloyal troop:

48. 48 The Byzantine General Problem  General Proof Outline: There must exist one more loyal troop to sway the balance of the majority:

49. 49 The Byzantine General Problem  General Proof Outline: But the proof only holds if the algorithm runs for m (or more) total rounds! 1 2 3 4 5 6

50. Impossibility Result Illustrated  With only 3 generals, no solution can work with even 1 traitor (given oral messages) commander attack retreat L1L2 What should L1 do? Is commander or L2 the traitor???

51. Option 1: Loyal Commander commander attack retreat L1L2 attack What must L1 do? By IC2: L1 must obey commander --> L1 must attack.

52. Option 2: Loyal L2 commander attack retreat L1L2 retreat What must L1 do? By IC1: L1 and L2 must obey same order --> L1 must retreat. Problem: L1 cannot distinguish between the two scenarios.

53. Solution I – Oral Messages If there are 3m+1 generals, solution allows up to m traitors. Oral messages – the sending of content is entirely under the control of sender. Assumptions on oral messages: A1 – Each message that is sent is delivered correctly. A2 – The receiver of a message knows who sent it. A3 – The absence of a message can be detected. Assures: Traitors cannot interfere with communication as third party. Traitors cannot send fake messages Traitors cannot interfere by being silent. Default order to “ retreat ” for silent traitor.

54. Oral Messages (Cont) Algorithm OM(0) Commander sends his value to every lieutenant. Each lieutenant (L) uses the value received from commander, or RETREAT if no value is received. Algorithm OM(m), m>0 Commander sends his value to every Lieutenant (v i ). Each Lieutenant acts as commander for OM(m-1) and sends v i to the other n-2 lieutenants (or RETREAT). For each i, and each j<>i, let v j be the value lieutenant i receives from lieutenant j in step (2) using OM(m-1). Lieutenant i uses the value majority (v 1, …, v n-1 ).  Why j<>i? “ Trust myself more than what others said I said. ”

55. Algorithm OM(M): Commander sends out command. Each lieutenant acts as commander in OM(m-1). Sends out command to other lieutenants. Use majority to compute value based on commands received by other lieutenants in OM(m-1) Interactive Consistency goals: IC1: All loyal lieutenants obey the same command. IC2: If the commanding general is loyal, then every loyal lieutenant obeys the command he sends.

56. Expensive Communication OM(m) invokes n-1 OM(m-1) OM(m-1) invokes n-2 OM(m-2) OM(m-2) invokes n-3 OM(m-3) … OM(m-k) will be called (n-1)…(n-k) times O(n m ) – Expensive!

57. Example: Bad Lieutenant Scenario: m=1, n=4, traitor = L3 C L1 L3L2 A A A OM(1): OM(0):??? C L1 L3L2 A A R R Decision??L1 = m (A, A, R); L2 = m (A, A, R); Both attack! A A

58. Example: Bad Commander Scenario: m=1, n=4, traitor = C C L1 L3L2 A R A OM(1): OM(0):??? L1 L3L2 A R A A Decision?? L1=m(A, R, A); L2=m(R, A, A); L3=m(A, R, A); Attack! R A

59. Bigger Example: Bad Lieutenants Scenario: m=2, n=3m+1=7, traitors=L5, L6 C A A A L2 L6L3 L5L4L1 A A A L2 L6L3 L5L4L1 AAAARR Decision??? Messages? m(A,A,A,A,R,R) ==> All loyal lieutenants attack!

60. Bigger Example: Bad Commander+ Scenario: m=2, n=7, traitors=C, L6 C L2 L6L3 L5L4L1 R A R A A x A,R,A,R,A ARR A A Decision??? L2 L6L3 L5L4L1 Messages?

61. Decision with Bad Commander+ L1: m(A,R,A,R,A,A) ==> Attack L2: m(A,R,A,R,A,R) ==> Retreat L3: m(A,R,A,R,A,A) ==> Attack L4: m(A,R,A,R,A,R) ==> Retreat L5: m(A,R,A,R,A,A) ==> Attack Problem: All loyal lieutenants do NOT choose same action. Two rounds insufficient!

62. Next Step of Algorithm Verify that lieutenants tell each other the same thing Requires rounds = m+1 OM(0): Msg from Lieut i of form: “L0 said v0, L1 said v1, etc.” What messages does L1 receive in this example? OM(2): A OM(1): 2R, 3A, 4R, 5A, 6A (doesn’t know 6 is traitor) OM(0): 2{ 3A,4R, 5A,6R} 3{2R, 4R, 5A, 6A} 4{2R, 3A, 5A, 6R} 5{2R, 3A, 4R, 6A} 6{ total confusion } All see same messages in OM(0) from L1, L2, …, L5 m(A,R,A,R,A,-) ==> All attack

63. Next Step of Algorithm What messages does L2 receive in this example? OM(2): R OM(1): 1A, 3A, 4R, 5A, 6R (doesn’t know 6 is traitor) OM(0): 1{ 3A,4R, 5A,6A} 3{1A, 4R, 5A, 6A} 4{1A, 3A, 5A, 6R} 5{1A, 3A, 4R, 6A} 6{ total confusion } All see same messages in OM(0) from L1, L2, …, L5 m(A,R,A,R,A,-) ==> All attack

64. 64 The Byzantine General Problem  Tough stuff. But if we add one more assumption, we can make the problem a lot easier: A4: Messages are signed.  a) A loyal general has a signature that cannot be forged.  b) A signed message cannot be altered without detection.  c) Anyone can verify the signature.

65. 65 The Byzantine General Problem Returning to the problem that didn ’ t work with unsigned messages…

66. 66 The Byzantine General Problem Previously, our general sent two orders out…

67. 67 The Byzantine General Problem But when the algorithm runs for a second round… Conflicting Orders!

68. 68 The Byzantine General Problem  The authors find that, by using signed messages: Any number of disloyal generals may exist in a system. All loyal generals will agree on a common result after m rounds.

69. 69 The Two Generals’ Problem  Reviewing our assumptions: A1: Every message sent was delivered correctly.  The “ Two Generals ’ Problem ” showed that two generals cannot ever reach consensus with the possibility of lost messages. Developed by Akkoyunlu et al. in 1975.

70. 70 Useful?  In a system with a bound on adversarial nodes, you must perform at least m rounds to reach consensus. Unsigned Messages: m ≤ (n-1)/3 Signed Messages: m ≤ n  Requires PKI or some similar system.

71. 71 Discussion: The Byzantine Generals In the beginning of this presentation, we began with a problem of every general giving an initial value and no coordinated leader.

72. 72 Discussion: The Byzantine Generals How do we reach the end such that all loyal generals agree on the same outcome?

73. 73 Discussion: The Byzantine Generals  Trivial Solution (Lamport et al., 1982): Run Byzantine Generals a total of n times, where the chosen general is a different site each of the n times. Take the majority vote of the total of n rounds.  Is there a more optimal solution?

74. 74 Discussion: The Byzantine Generals The Byzantine generals problem is presented in [1] in terms of only two options: “ attack ” or “ retreat ”. What if we needed an agreed upon int?

75. 75 Discussion: The Byzantine Generals  The Byzantine Generals problem requires m rounds to protect against m disloyal troops. We could reduce the number of rounds if we could somehow determine how much ‘ disloyalty ’ exists in the system. Could we?

76. 76 Discussion: The Byzantine Generals  With a PKI (signed messages): Allows m = n; therefore, n rounds must be made AND requires the overhead of a PKI.  Without a PKI (unsigned messages): Forces m < n/3; therefore, only n/3 rounds and no PKI.  Therefore, are there a significant number of systems where a PKI-free system would be desirable?

77. Signed Messages Problem: Traitors can lie about what others said; how can we remove that ability? New assumption: Signed messages (Cryptography) A4) a. Loyal general’s signature cannot be forged and contents cannot be altered. b. Anyone can verify authenticity of signature.

78. Signed Messages Simplifies problem: When lieutenant i passes on signed message from j, receiver knows that i did not lie about what j said Lieutenants cannot do any harm alone (cannot forge loyal general’s orders) Only have to check for traitor commander With cryptographic primitives, can implement Byzantine Agreement with m+2 nodes, using SM(m)

79. Signed Messages Algorithm: SM(m) 1. Commander signs v and sends to all as (v:0) 2. Each lieut i: A) If receive (v:0) and no other order 1) Vi = v 2) send (V:0:i) to all B) If receive (v:0:j:...:k) and v not in Vi 1) Add v to Vi 2) if (k<m) send (v:0:j:...:k:i) to all not in j...k 3. When no more msgs, obey order of choice(Vi)

80. SM(1) Example: Bad Commander Scenario: m=1, n=m+2=3, bad commander C L1 L2 A:0R:0 What next? L1 L2 A:0:L1 R:0:L2 V1={A,R} V2={R,A} Both L1 and L2 can trust orders are from C. Both apply same decision to {A,R}.

81. SM(2): Bad Commander+ Scenario: m=2, n=m+2=4, bad commander and L3 C L1 L3L2 A:0 x Goal? L1 and L2 must make same decision. L1 L3L2 A:0:L1 A:0:L2 A:0:L3 R:0:L3 L1 L2 R:0:L3:L1 V1 = V2 = {A,R} ==> Same decision

86. 86 Thanks!