1. CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
Set 11: Asynchronous Consensus
CSCE 668 DISTRIBUTED ALGORITHMS AND SYSTEMS
CSCE 668
Spring 2014
Prof. Jennifer Welch

2. Impossibility of Asynchronous Consensus
Show impossible in read/write shared memory with n processors and n - 1 faults
prove directly: not hard since so many faults
implies there is no 2-proc algorithm for 1 fault
Show impossible in r/w shared memory with n processors and 1 fault. Two approaches:
Reduction: use a hypothetical n-proc algorithm for 1 fault as a subroutine to design a 2-proc algorithm for 1 fault
Direct proof: Use similar ideas to n-1 failures case
Set 11: Asynchronous Consensus
CSCE 668

3. Impossibility of Asynchronous Consensus
Show impossible in message passing with n processors and 1 fault. Two approaches:
Reduction: Use a hypothetical message passing algorithm for n procs and 1 fault as a subroutine to design a shared memory algorithm for n procs and 1 fault. This would contradict previous result.
Direct approach: Use similar ideas to shared memory case, augmented to handle messages. (Historically, this was the first version that was proven.)
Set 11: Asynchronous Consensus
CSCE 668

4. Modeling Asynchronous Systems with Crash Failures
Let f be the maximum number of faulty processors.
For both SM and MP: All but f of the processors must take an infinite number of steps in an admissible execution.
For MP: Also require that all messages sent to a nonfaulty processor must eventually be delivered, except for those sent by a faulty processor in its last step, which might or might not be delivered.
Set 11: Asynchronous Consensus
CSCE 668

5. Wait-Free Algorithms
An algorithm for n processors is wait-free if it can tolerate n - 1 failures.
Intuition is that a nonfaulty processor does not wait for other processors to do something: it cannot, because it might be the only processor left alive.
First result is to show that there is no wait-free consensus algorithm in the asynchronous r/w shared memory model.
Set 11: Asynchronous Consensus
CSCE 668

6. Impossibility of Wait-Free Consensus
Suppose in contradiction there is an n-processor algorithm for n - 1 faults in the asynchronous read/write shared memory model.
Proof is similar to that showing f + 1 rounds are necessary in the synchronous message passing model. …
bivalent
initial
config
bivalent
config
bivalent
config
bivalent
config
bivalent
config
Set 11: Asynchronous Consensus
CSCE 668

7. Modified Notion of Bivalence
In the synchronous round lower bound proof, valency referred to which decisions are reachable in failure- sparse admissible executions.
For this proof, we are concerned with which decisions are reachable in any execution, as long as it is admissible (for the asynchronous shared memory model with up to n - 1 failures).
Set 11: Asynchronous Consensus
CSCE 668

8. Univalent Similarity because wait-free Thus v = w
Lemma (5.15): If C1 and C2 are both univalent and they are similar w.r.t. pi (shared memory state is same, pi’s local state is same), then they have the same valency. Proof:
because wait-free
pi-only  C1
v-valent
pi decides v 
Thus v = w C2
w-valent
pi decides v
Set 11: Asynchronous Consensus
CSCE 668

9. Bivalent Initial Configuration
Lemma (5.16): There exists a bivalent initial configuration. Proof is similar to what we did for the synchronous f + 1 round lower bound proof.
Set 11: Asynchronous Consensus
CSCE 668

10. Critical Processors
Def: If C is bivalent and i(C) (result of pi taking one step) is univalent, then pi is critical in C. Lemma (5.17): If C is bivalent, then at least one processor is not critical in C, i.e., there is a bivalent extension. Proof: Suppose in contradiction all processors are critical. C
bival.
j(C)
1-val.
i(C)
0-val. pi pj
Rest of proof is case
analysis of what pi and
pj do in their two steps
Set 11: Asynchronous Consensus
CSCE 668

11. Can't be different valencies!
Critical Processors
Case 1: pi and pj access different registers. C
bival.
i(C)
0-val. pi pj pi
Can't be different valencies!
j(C)
1-val. pj
Case 2: pi and pj read same register. Same proof.
Set 11: Asynchronous Consensus
CSCE 668

12. contradicts Univalent
Critical Processors
Case 3: pi writes to a register R and pj reads from R.
pj reads from R
j(C)
1-val C
bival.
pi writes
to R
contradicts Univalent
pi writes
to R
Similarity Lemma
i(C)
0-val
i(j(C))
1-val
similar w.r.t. pi
Set 11: Asynchronous Consensus
CSCE 668

13. Critical Processors
Case 4: What if pi and pj both write to the same shared variable?
Can "assume away" the problem by assuming we only have single-writer shared variables.
Or, can do a similar proof for this case.
Set 11: Asynchronous Consensus
CSCE 668

14. Finishing the Impossibility Proof
Create an admissible execution C0,i1,C1,i2,C2,… in which all configurations are bivalent.
contradicts termination requirement
Start with bivalent initial configuration.
Suppose we have bivalent Ck.
To get bivalent Ck+1:
Let pi_k+1 be a processor that is not critical in Ck.
Let Ck+1 be ik+1(Ck).
Set 11: Asynchronous Consensus
CSCE 668

15. Impossibility of 1-Resilient Consensus: Reduction Idea
Even if the ratio of nonfaulty processors becomes overwhelming, consensus still cannot be solved in asynchronous SM (with read/write registers).
Assume there exists an algorithm A for n processors and 1 failure.
Use A as a subroutine to design an algorithm A' for 2 processors and 1 failure.
We just showed such an A' cannot exist.
Thus A cannot exist.
Set 11: Asynchronous Consensus
CSCE 668

16. Impossibility of 1-Resilient Consensus: Direct Proof Idea
Suppose in contradiction there is such an algorithm.
Strategy: Construct an admissible execution (at most 1 fault) that never terminates:
show there is a bivalent initial configuration
show how to go from one bivalent configuration to another, forever (so can never terminate)
Technically more involved because in constructing this execution, we cannot kill more than one processor.
Set 11: Asynchronous Consensus
CSCE 668

17. Impossibility of Consensus in Message Passing: Reduction
Strategy:
Assume there exists an n-processor 1-resilient consensus algorithm A for the asynchronous message passing model.
Use A as a subroutine to design an n-processor 1-resilient consensus algorithm A' for asynchronous shared memory (with read/write variables).
Previous result shows A' cannot exist.
Thus A cannot exist.
Set 11: Asynchronous Consensus
CSCE 668

18. Impossibility of Consensus in MP
Idea of A':
Simulate message channels with read/write registers.
Then run algorithm A on top of these simulated channels.
To simulate channel from pi to pj:
Use one register to hold the sequence of messages sent over the channel
pi "sends" a message m by writing the old value of the register with m appended
pj "receives" a message by reading the register and checking for new values at the end
Set 11: Asynchronous Consensus
CSCE 668

19. Randomized Consensus
To get around the negative results for asynchronous consensus, we can:
weaken the termination condition: nonfaulty processors must decide with some nonzero probability
keep the same agreement and validity conditions
This version of consensus is solvable, in both shared memory and message passing!
Set 11: Asynchronous Consensus
CSCE 668

20. Motivation for Adversary
Even without randomization, in an asynchronous system there are many executions of an algorithm, even when the inputs are fixed, depending on when processors take steps, when they fail, and when messages are delivered.
To be able to calculate probabilities for a randomized algorithm, we need to separate out variation due to causes other than the random choices
Group executions of interest so that members of each group differ only in the random choices
Perform probabilistic calculations separately for each group and then combine somehow
Set 11: Asynchronous Consensus
CSCE 668

21. Adversary
Concept used to account for all variability other than the random choices is that of "adversary”:
Message delays, relative proc speeds, which procs fail and when,…
Adversary is a function that takes an execution prefix and returns the next event to occur.
Adversary must obey admissibility conditions of the revelant model
Other conditions might be put on the adversary:
what information it can observe, how much computational power it has,…
Set 11: Asynchronous Consensus
CSCE 668

22. Probabilistic Definitions
An execution of a specific algorithm, exec(A,C0,R), is uniquely determined by
an adversary A
an initial configuration C0
a collection of random numbers R
Given a predicate P on executions and a fixed adversary A and initial config C0, Pr[P] is the probability of
{R : exec(A,C0,R) satisfies P}
Let T be a random variable (e.g., running time). For a fixed A and C0, the expected value of T is ∑
x Pr[T = x]
x is a value
of T
Set 11: Asynchronous Consensus
CSCE 668

23. Probabilistic Definitions
We define the expected value of a complexity measure to be the maximum over all admissible adversaries A and initial configurations C0, of the expected value for that particular A and C0.
So this is a "worst-case" average: worst possible adversary (pattern of asynchrony and failures) and initial configuration, averaging over the random choices.
Set 11: Asynchronous Consensus
CSCE 668

24. A Randomized Consensus Algorithm
Works in message passing model
Tolerates f crash failures
more complicated version handles Byzantine failures
Works in asynchronous case
circumvents asynchronous impossibility result (by weakening termination condition)
Requires n > 2f
this is optimal
Set 11: Asynchronous Consensus
CSCE 668

25. Consensus Algorithm Code for processor pi:
Initially r = 1 and prefer = pi 's input
while true do
votes := get-core(<VOTE,prefer,r>)
let v be majority of phase r votes
if all phase r votes are v then decide v
outcomes := get-core(<OUTCOME,v,r>)
if all phase r outcome values are w
then prefer := w
else prefer := common-coin()
r := r + 1
ensures a high level
of consistency b/w
what different procs
get
uses randomization
to imitate tossing
a coin
Set 11: Asynchronous Consensus
CSCE 668

26. Properties of Get-Core
Executed by n processors, at most f of which can crash.
Input parameter is a value supplied by the calling processor.
Return parameter is an n-array, one entry per processor
Every nonfaulty processor's call to get-core returns.
There exists a set C of more than n/2 processors such that every array returned by a call to get- core contains the input parameter supplied by every processor in C.
Set 11: Asynchronous Consensus
CSCE 668

27. Properties of Common-Coin
Subroutine implements an f-resilient common coin with bias .
Executed by n processors, at most f of which can crash.
No input parameter
Return parameter is a 0 or 1.
Every nonfaulty processor's call to common-coin returns.
Probability that a return value is 0 is at least .
Probability that a return value is 1 is at least .
Set 11: Asynchronous Consensus
CSCE 668

28. Correctness of Consensus Algorithm
For now, don't worry about how to implement get- core and common-coin.
Assuming we have subroutines with the desired properties, we'll show
validity
agreement
probabilistic termination (and expected running time)
Set 11: Asynchronous Consensus
CSCE 668

29. Unanimity Lemma
Lemma (14.6): If all procs. that reach phase r prefer v, then all nonfaulty procs decide v by phase r.
Proof:
Since all prefer v, all call get-core with v
Thus get-core returns a majority of votes for v
Thus all nonfaulty procs. decide v
Set 11: Asynchronous Consensus
CSCE 668

30. Validity If all processors have input v, then all prefer v in phase 1.
By unanimity lemma, all nonfaulty processors decide v by phase 1.
Set 11: Asynchronous Consensus
CSCE 668

31. Agreement
Claim: If pi decides v in phase r, then all nonfaulty procs. decide v by phase r + 1.
Proof: Suppose r is earliest phase in which any proc. decides.
pi decides v in phase r
all its phase r votes are v
pi 's call to get-core(<VOTE,prefer,r>) returns more than n/2 non-nil entries and all are <VOTE,v,r>
all entries for procs. in C are <VOTE,v,r>
continued on next slide…
Set 11: Asynchronous Consensus
CSCE 668

32. Agreement (cont’d)
Thus every pj receives more than n/2 <VOTE,v,r> entries
pj does not decide a value other than v in phase r
Also if pj calls get-core a second time in phase r, it uses input <OUTCOME,v,r>
Every pk gets only <OUTCOME,v,r> as a result of its second call to get-core in phase r
pk sets preference to v at end of phase r
in round r + 1, all prefer v and Unanimity Lemma implies they all decide v in that round.
Set 11: Asynchronous Consensus
CSCE 668

33. Termination
Lemma (4.10): Probability that all nonfaulty procs decide by any particular phase is at least .
Proof:
Case 1: All nonfaulty procs set preference in that phase using common-coin.
Prob. that all get the same value is at least 2 ( for 0 and  for 1), by property of common-coin
Then apply Unanimity Lemma (14.6)
Set 11: Asynchronous Consensus
CSCE 668

34. Termination
Case 2: Some processor does not set its preference using common-coin.
All procs. that don't use common-coin to set their preference for that round have the same preference, v (convince yourself)
Probability that the common-coin subroutine returns v for all procs. that use it is at least .
Then apply the Unanimity Lemma (14.6).
Set 11: Asynchronous Consensus
CSCE 668

35. Expected Number of Phases
What is the expected number of phases until all nonfaulty processors have decided?
Probability of all deciding in any given phase is at least .
Probability of terminating after i phases is (1–)i- 1.
Geometric random variable whose expected value is 1/.
Set 11: Asynchronous Consensus
CSCE 668

36. Implementing Get-Core
Difficulty in achieving consistency of messages is due to combination of asynchrony and crash possibility:
a processor can only wait to receive n - f messages
the first n - f messages that pi gets might not be from the same set of processors as pj 's first n - f messages
Overcome this by exchanging messages three times
Set 11: Asynchronous Consensus
CSCE 668

37. Get-Core First exchange ("round"): send argument value to all
wait for n - f first round msgs
Second exchange ("round"):
send values received in first round to all
wait for n - f second round msgs
merge data from second round msgs
Third exchange ("round"):
send values received in second round to all
wait for n - f third round msgs
merge data from third round msgs
return result
Set 11: Asynchronous Consensus
CSCE 668

38. Analysis of Get-Core
Lemmas 14.4 and 14.5 show that it satisfies the desired properties (termination and consistency).
Time is O(1) (using standard way of measuring time in an asynchronous system)
Set 11: Asynchronous Consensus
CSCE 668

39. Implementing Common-Coin
A simple algorithm:
Each processor independently outputs 0 with probability 1/2 and 1 with probability 1/2.
Bias  = 1/2n
Advantage: simple, no communication
Disadvantage: Expected number of phases until termination is 2n
Set 11: Asynchronous Consensus
CSCE 668

40. A Common Coin with Constant Bias
0 with probability 1/n 1 with probability 1 – 1/n coins := get-core(<FLIP,c>) if there exists j s.t. coins[j] = 0 then return 0 else return 1
c :=
Set 11: Asynchronous Consensus
CSCE 668

41. Correctness of Common Coin
Lemma (14.12): Common-coin implements a (n/2 – 1)-resilient coin with bias 1/4. Proof: Fix any admissible adversary that is weak (cannot see the contents of messages) and any initial configuration. All probabilities are calculated with respect to them.
Set 11: Asynchronous Consensus
CSCE 668

42. Probability of Flipping 1
Probability that all nonfaulty processors get 1 for the common coin is at least the probability that they all set c to 1.
This probability is at least (1 – 1/n)n
When n = 2, this function is 1/4
This function increases up to its limit of 1/e.
Thus the probability that all nonfaulty processors get 1 is at least 1/4.
Set 11: Asynchronous Consensus
CSCE 668

43. Probability of Flipping 0
Let C be the set of core processors (whose existence is guaranteed by properties of get-core).
If any processor in C sets c to 0, then all the nonfaulty processors will observe this 0 after executing get-core, and thus return 0.
Probability at least one processor in C sets c to 0 is 1 – (1 – 1/n)|C|.
This expression is at least 1/4 (by arithmetic).
Set 11: Asynchronous Consensus
CSCE 668

44. Summary of Randomized Consensus Algorithm
Using the given implementations for get- core and common-coin, we get an asynchronous randomized consensus algorithm for f crash failures with
n > 2f
O(1) expected time complexity
expected number of phases is 4
time per phase is O(1)
Set 11: Asynchronous Consensus
CSCE 668