1. 1 Chapter 12 Consensus ( Fault Tolerance)

2. 2 Reliable Systems Distributed processing creates faster systems by exploiting parallelism but also improve reliability by replicating a computation in several processors Distributed processing creates faster systems by exploiting parallelism but also improve reliability by replicating a computation in several processors A reliable system can be: A reliable system can be: Fail-safe if one or more failures do not cause damage to the system or to its users Fail-safe if one or more failures do not cause damage to the system or to its usersand/or Fault-tolerant if it continues to fulfill its requirements even if there are one or more failures Fault-tolerant if it continues to fulfill its requirements even if there are one or more failures

3. 3 Typical Architectures for a Reliable System

4. 4 The Problem Statement A group of Byzantine armies is surrounding an enemy city. The balance of force is such that if all armies attack together, they can capture the city; otherwise, they must all retreat to avoid defeat. The generals of the armies have reliable messengers who successfully deliver any message sent from one general to another. However, some of the generals may be traitors endeavoring to bring about the defeat of the Byzantine armies. A group of Byzantine armies is surrounding an enemy city. The balance of force is such that if all armies attack together, they can capture the city; otherwise, they must all retreat to avoid defeat. The generals of the armies have reliable messengers who successfully deliver any message sent from one general to another. However, some of the generals may be traitors endeavoring to bring about the defeat of the Byzantine armies. Devise an algorithm so that all loyal generals come to a consensus on a plan. Devise an algorithm so that all loyal generals come to a consensus on a plan. The final decision should be almost the same as a majority vote of their initial choices; if the vote is tied. The final decision should be almost the same as a majority vote of their initial choices; if the vote is tied.

5. 5 The Problem Statement (Cont.) In distributed systems, the generals are the nodes and the messengers model communication channels In distributed systems, the generals are the nodes and the messengers model communication channels Generals may fail (being traitors), but the messengers are assumed to be reliable Generals may fail (being traitors), but the messengers are assumed to be reliable Models for node failures: Models for node failures: Crash failures: A traitor (failure node) simply stops sending messages at any arbitrary point during the execution of the algorithm Crash failures: A traitor (failure node) simply stops sending messages at any arbitrary point during the execution of the algorithm Byzantine failures: A traitor can send arbitrary messages, not just the messages required by the algorithm Byzantine failures: A traitor can send arbitrary messages, not just the messages required by the algorithm

6. 6 Consensus – One-round Algorithm The values for planType are A for attack and R for retreat The values for planType are A for attack and R for retreat Each general chooses a plan, sends its plan to the other generals and receives their plans Each general chooses a plan, sends its plan to the other generals and receives their plans The final plan is the majority vote among all plans, both the general’s own plan and the plans received from the others The final plan is the majority vote among all plans, both the general’s own plan and the plans received from the others

7. 7 Messages Sent in a One-round Algorithm Generals Zoe and Leo are loyal, Basil is a traitor Generals Zoe and Leo are loyal, Basil is a traitor Basil and Zoe chooses to attack, Leo chooses to retreat Basil and Zoe chooses to attack, Leo chooses to retreat Messages are exchanged, but Basil has crashed after sending an attack message to Leo. No message is received by Zoe from Basil Messages are exchanged, but Basil has crashed after sending an attack message to Leo. No message is received by Zoe from Basil Zoe decides to retreat (may have chosen attack), Leo decides to attack by majority voting Zoe decides to retreat (may have chosen attack), Leo decides to attack by majority voting Basil has crashed, Zoe retreats – ties are resolved in favour of retreat – common sense, Leo decides to attack; no consensus Basil has crashed, Zoe retreats – ties are resolved in favour of retreat – common sense, Leo decides to attack; no consensus If a general crashes, it can cause the remaining loyal generals to fail to come to a consensus; no consensus If a general crashes, it can cause the remaining loyal generals to fail to come to a consensus; no consensus

8. 8 The Byzantine Generals Algorithm In the one-round algorithm, the fact that certain generals been loyal is not considered. Leo should somehow be able to attribute more weight to the plan received from loyal Zoe than the traitor Basil In the one-round algorithm, the fact that certain generals been loyal is not considered. Leo should somehow be able to attribute more weight to the plan received from loyal Zoe than the traitor Basil In a distributed system an individual node can not know the identities of the traitors directly; rather, it must ensure that the plan of the traitors can not cause the loyal generals to fail to reach consensus In a distributed system an individual node can not know the identities of the traitors directly; rather, it must ensure that the plan of the traitors can not cause the loyal generals to fail to reach consensus

9. 9 Algorithm in Brief The Byzantine Generals algorithm sends messages twice: The Byzantine Generals algorithm sends messages twice: In the first round each general sends its own plan In the first round each general sends its own plan In the second round each general sends what is received from other generals In the second round each general sends what is received from other generals Loyal generals relay exactly what they received, so that if there are enough loyal generals, they can reach to a consensus Loyal generals relay exactly what they received, so that if there are enough loyal generals, they can reach to a consensus

10. 10 First round sends plans and receives plans. At the end each general has the plan of each general First round sends plans and receives plans. At the end each general has the plan of each general In the second round, these plans are send to the other generals (except himself) and received back again In the second round, these plans are send to the other generals (except himself) and received back again Byzantine Generals Algorithm

11. 11 Two Loyal, One Traitor – Crash Failure Same scenario in the one-round algorithm, where Basil (traitor) crashes after sending the first round message to Leo, but before sending to Zoe Same scenario in the one-round algorithm, where Basil (traitor) crashes after sending the first round message to Leo, but before sending to Zoe 2’nd column is the first round plans (Zoe: gets Leo’s plan and nothing from crashed Basil, Leo: has all plans) 2’nd column is the first round plans (Zoe: gets Leo’s plan and nothing from crashed Basil, Leo: has all plans) 3’rd and 4’th are the second round plans (Zoe: No plan from Basil - crashed, Basil’s A from Leo, Leo does not send its plan –R- again; Leo: No plan from Basil, No plan from Zoe – sent in the first round) 3’rd and 4’th are the second round plans (Zoe: No plan from Basil - crashed, Basil’s A from Leo, Leo does not send its plan –R- again; Leo: No plan from Basil, No plan from Zoe – sent in the first round) Majority voting : Basil: crashed; Zoe: Attack; Leo: Attack Majority voting : Basil: crashed; Zoe: Attack; Leo: Attack Two of the generals reached to a consensus Two of the generals reached to a consensus

12. 12 Another Scenario Basil, the traitor, sends all its first round messages and reports to Leo before crashing Basil, the traitor, sends all its first round messages and reports to Leo before crashing Second Round; Leo: Basil sends Zoe’s Attack plan, Zoe sends Basil’s Attack plan; Zoe: No plan from Basil, Attack from Leo Second Round; Leo: Basil sends Zoe’s Attack plan, Zoe sends Basil’s Attack plan; Zoe: No plan from Basil, Attack from Leo Majorty voting: Both decide to attack Majorty voting: Both decide to attack

13. 13 Byzantine Failures with Three Generals (One-round Algorithm) Basil, the traitor, sends a retreat message to Zoe and attack to Leo Basil, the traitor, sends a retreat message to Zoe and attack to Leo One round algorithm fails – no consensus – like the crash failure case One round algorithm fails – no consensus – like the crash failure case

14. 14 In the first round, Basil sends an A message to both Zoe and Leo In the first round, Basil sends an A message to both Zoe and Leo In the second round, he correctly reports to Zoe that Leo’s plan is R, but erroneously reports to Leo that Zoe’s plan is R In the second round, he correctly reports to Zoe that Leo’s plan is R, but erroneously reports to Leo that Zoe’s plan is R Leo decides to retreat (ties are broken in favour of retreat), Zoe decides to attack – no consensus again Leo decides to retreat (ties are broken in favour of retreat), Zoe decides to attack – no consensus again The algorithm is not correct for three generals of whom one is a traitor The algorithm is not correct for three generals of whom one is a traitor Byzantine Failures with Three Generals (Two-round Algorithm)

15. 15 Byzantine Failures with Four Generals John, Basil, Leo are loyal generals; Zoe is the traitor John, Basil, Leo are loyal generals; Zoe is the traitor Zoe sends first-round messages of R to Basil and Leo and A to John. These messages are relayed correctly by loyal generals and Basil has the table shown on the left Zoe sends first-round messages of R to Basil and Leo and A to John. These messages are relayed correctly by loyal generals and Basil has the table shown on the left The final decision will be a 2-1 vote in favor of R for Zoe’s plan The final decision will be a 2-1 vote in favor of R for Zoe’s plan So, if the loyal generals choose the same plan initially, the final decision would be this plan, regardless of the actions of the traitor So, if the loyal generals choose the same plan initially, the final decision would be this plan, regardless of the actions of the traitor

16. 16 Consensus Crash Failures: consensus is reached in t+1 rounds where t is number of traitors Crash Failures: consensus is reached in t+1 rounds where t is number of traitors Byzantine Failures: If more then two-thirds of the generals are loyal, there is a solution regardless the messages issued by traitorous generals. If one-third or more of the generals are traitors then there is no solution. In the case of one traitor, there is a solution for four generals and none for three. That is, the total number of generals must be at least 3t+1, where t is the number of traitors Byzantine Failures: If more then two-thirds of the generals are loyal, there is a solution regardless the messages issued by traitorous generals. If one-third or more of the generals are traitors then there is no solution. In the case of one traitor, there is a solution for four generals and none for three. That is, the total number of generals must be at least 3t+1, where t is the number of traitors