1 Chapter 11: Coordination and Agreement From Coulouris, Dollimore and Kindberg Distributed Systems: Concepts and Design Edition 3, © Addison-Wesley 2001 Presentation based on slides by Coulouris et al; modified by Jens B Jorgensen, University of Aarhus

2 Failure model zAssumptions: Independent processes, reliable channels, no Byzantine errors. zFailure detector: yMay be unreliable (yield “Unsupected” or “Suspected”). yMay be reliable (yield “Unsuspected” or “Failed”).

3 Distributed mutual exclusion zSituation: A number of processes want to access some shared resource. zProblem: Prevent interference, maintain consistency; critical section. zExamples: yShared files. yCar park monitoring.

4 Distributed mutual exclusion – basics zApplication-level protocol: yenter() yresourceAccess() yexit() zCorrectness criteria: ySafety yLiveness y->-fairness zPerformance measures: yBandwith. yClient delay. yThroughput. ySynchronization delay.

5 Distributed mutual exclusion – central server algorithm

6 Distributed mutual exclusion – ring-based algorithm

7 Distributed mutual exclusion – time-stamp based algorithm On initialization state := RELEASED; To enter the section state := WANTED; Multicast request to all processes;request processing deferred here T := request’s timestamp; Wait until (number of replies received = (N – 1)); state := HELD; On receipt of a request at p j (i ≠ j) if (state = HELD or (state = WANTED and (T, p j ) < (T i, p i ))) then queue request from p i without replying; else reply immediately to p i ; end if To exit the critical section state := RELEASED; reply to any queued requests;

8 Distributed mutual exclusion – time-stamp based algorithm example p 3 34 Reply p 1 p 2 Reply

9 Elections zSituation: A unique process to play a particular role among a set of processes must be chosen. zProblem: All processes must agree on the choice. zExamples: yCentral server algorithm for mutual exclusion. yCoordinator process in Berkeley algorithm for internal clock synchronization.

10 Elections – basics zProtocol: yA process may call an election. yAny process is either participant or non-participant in an election. yThe elected process should be chosen as the one with largest identifier. zCorrectness criteria: ySafety. yLiveness. zPerformance measures: yBandwidth. yTurn-around time.

11 Elections – ring-based algorithm zN processes arranged in a ring; a coordinator must be elected; no failures occur. zInitially, each process is non-participant. zSome process pi sends an election message, elctn(i). zWhen a process pr receives a message elctn(i): yIf r<i then forward elctn(i); participant(pr) := true; endif; yIf (r>i and not(participant(pr))) then xforward (elctn(r)); participant(pr) := true; endif; yIf (r>i and participant(pr)) then skip (* do not forward *); endif; yIf r=i then participant(pr) := false; send(elctd(r)); endif; zWhen a process pr receives an elected message, elctd(c): yparticipant(pr) := false; elected(r) := c; y if r=c then skip (* do not forward *) else forward (elctd(c)); endif;

12 Elections – ring-based algorithm example Note: The election was started by process 17. The highest process identifier encountered so far is 24. Participant processes are shown darkened

13 Multicast communication zThe aim is for each of a group of processes to receive copies of the messages sent to the group. zA process issues only one multicast operation to send a message to a group of processes instead of issuing multiple send operations. zMulticast operations may provide: yDelivery guarantees. yEfficiency. yConvenience to programmers.

14 Multicast communication – basics zFailure model: Reliable channels, processes may crash. zProcesses are member of groups (open or closed). zOperations: ymulticast(g,m). ydeliver(m).

15 Multicast communication – basic multicast zB-multicast based on a reliable one-to-one send operation: yTo B-multicast(g,m), for each p in g, send(p,m). yOn receive(m) at p, B-deliver(m) at p. zMay be implemented using threads to perform the send operations concurrently. zMay suffer from the ack-implosion problem.

16 Multicast communication – reliable multicast zIntegrity: A correct process p delivers a message m at most once. Furthermore p in group(m) and m was supplied to a multicast operation by sender(m). zValidity: If a correct process multicasts message m, then it will eventually deliver m (assumption: closed groups). zAgreement: If a correct process delivers message m, then all other correct processes in group(m) will eventually deliver(m).

17 Multicast communication – reliable multicast algorithm

18 Multicast communication – orderings; bulletin board example Bulletin board: os.interesting Item FromSubject 23A.HanlonMach 24G.JosephMicrokernels 25A.HanlonRe: Microkernels 26T.L’HeureuxRPC performance 27M.WalkerRe: Mach end

19 Multicast communication – ordering relations zFIFO ordering: If a correct process issues multicast(g,m) and then multicast(g,m’), then every correct process that delivers m’ will deliver m before m’. zCausal ordering: If multicast(g,m) -> multicast(g,m’), then any correct process that delivers m’ will deliver m before m’. zTotal ordering: If a correct process delivers m before it delivers m’, then any other correct process that delivers m’ will deliver m before m’. zHybrid ordering relations exist.

20 Multicast communication – ordering examples Notice the consistent ordering of totally ordered messages T 1 and T 2, the FIFO-related messages F 1 and F 2 and the causally related messages C 1 and C 3 – and the otherwise arbitrary delivery ordering of messages.

21 Multicast communication – bulletin board example revisited Bulletin board: os.interesting Item FromSubject 23A.HanlonMach 24G.JosephMicrokernels 25A.HanlonRe: Microkernels 26T.L’HeureuxRPC performance 27M.WalkerRe: Mach end

22 Summary zFailures and failure detection. zDistributed mutual exclusion: yCentral-server algorithm. yRing-based algorithm. yTime-stamp based algorithm. zElections: yRing-based algorithm. zMulticast communication yBasic multicast. yReliable multicast. yOrdering semantics.