Transis Dynamic Voting for Consistent Primary Components PODC 1997 talk slides Esti Yeger Lotem, Idit Keidar and Danny Dolev The Hebrew University

Transis Primary Components Allows one subset of the processes to function when failures occur: –Database applications –Group communication systems (e.g., ISIS) Often based on majority (quorum) –Difficult to adapt to dynamic changes in the set of participants –Problematic in unreliable networks

Transis Dynamic Voting Defines quorums adaptively: majority of the previous quorum, e.g., {a,b,c,d,e} {a,b,c} {a,b} Naturally adapts to dynamic changes in the set of participants In unreliable networks, proven to lead to better performance: simulations, empirical tests, stochastic analysis

Transis Dynamic Linear Voting Breaks ties between groups of equal size Uses a linear order, L, on all potential processes in the system Sub_Quorum(S, T) if: –T contains a majority of S, or –T contains half the members of S including the member, p, of S with the highest L( p )

Transis Our Dynamic Voting Protocol for primary components Consistent Allows processes to join and leave on the fly Efficient –Low communication overhead Simple to implement Robust –Processes and links may fail

Transis The Challenge: Coping with failures that occur in the course of the protocol {a, b, c} attempt to form a quorum a and b succeed c detaches, unaware of the attempt

Transis The Challenge (Cont’d) {a, b} form a quorum –majority of {a, b, c} Concurrently {c, d, e} form a quorum –majority of {a, b, c, d, e}  Inconsistency!

Transis Other Protocols Inconsistent Two-phase Commit –Limits the availability ISIS - Cold start when primary is lost Phoenix - Three phase consensus protocol –High communication overhead

Transis Our Solution: A Second Level of Knowledge If a and b succeed in forming {a, b, c} then c is aware of the attempt. For c, {a, b, c} is ambiguous: {a, b, c} may or may not have been formed. –Processes record ambiguous attempts In our example, c records both: {a, b, c, d, e} and {a, b, c} –Requires a majority of both  c will refuse to form {c, d, e}

Transis A Session of the Protocol General Scheme Invoked Upon Membership Changes 1Exchange information 2If Sub_Quorum of the last primary and of all ambiguous attempts, Attempt: –Record the attempt as ambiguous 3If all attempted, Form: –Become the primary in the system Delete all ambiguous attempts

Transis Storing all the Ambiguous Attempts is not Feasible When failures cascade, the number of ambiguous attempts may be exponential: {a,b,c,d,e}, {a,b,c}, {a,b,d}, {a,b,e}, {a,b,c,f,g}, {a,b,c,f}, {a,b,d,g}, … Ambiguous attempts  constraints We use a “garbage collection” mechanism to store only a linear number of attempts

Transis Our “Garbage Collection” Mechanism Resolution rules: –If the attempt was formed by some member, adopt it as your primary. –If the attempt was not formed by any of the members, delete it. Learning rules: – p learns the status of q w.r.t. attempt A1 during a later attempt A2 Linear

Transis Why is this linear? (In the number of processes) If p and q participate in two attempts, A1 and A2 then: p learned whether q formed A1 before attempting to form A2 Once p learns about A1 from all its members, p can resolve A1  In each recorded attempt there is a member that does not appear in later attempts

Transis Criticism of Dynamic Voting Quorums can become very small (even one process) –Failure of a single process may cause the rest of the system to block Desirable solution: –Set a threshold, Min_Quorum, on quorum size –(N - Min_Quorum) process are always a quorum –Min_Quorum reflects the tradeoff between “static” and “dynamic”

Transis The Challenge What happens if N (the number of processes) changes on the fly? –N - Min_Quorum changes –The “truth value” of Sub_Quorum changes (no longer a predicate) Asynchronous distributed system –Different processes may know of different values of N

Transis Adding New Processes Carefully - two steps W - The set of participating processes –may “vote” for quorums A - Candidates to be added to W –do not “vote” for quorums –“vote against” large quorums New processes are added: – to A in the attempt step, – to W in the form step

Transis The Min_Quorum Requirement Every quorum must contain more than Min_Quorum members of W –at least Min_Quorum “vote for” Every group that contains all but Min_Quorum members of W and A is a quorum (regardless of past quorums) –At most Min_Quorum “vote against”

Transis Conclusions Consistently maintaining a primary component using dynamic voting More available than other protocols Simple and efficient No need for cold start New mechanism: always allowing large groups to be quorums where processes can join on the fly