1. Coordination and Agreement
This powerpoint presentation has been adapted from:
web.njit.edu/~gblank/.../Coordination%20and%20Agreement.ppt

2. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

3. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

4. Coordination and Agreement
Introduction
How processes coordinate their actions?
Main assumptions in coordination:
Each pair of processes is connected by reliable channels
Processes independent from each other
Processes fail only by crashing

5. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

6. Coordination and Agreement
Distributed Mutual Exclusion
Distributed processes require a mechanism that can coordinate their activities because they share a resource or collection of resources
Mutual exclusion is required to
prevent interference
ensure consistency when accessing the resources
Mutual exclusion: the process of ensuring that no more than one process is at critical section at one time
Process 2
Process 1
Process 3
Process n …
Shared
resource

7. Coordination and Agreement
Distributed Mutual Exclusion
Algorithms for mutual exclusion
Requirements for mutual exclusion are:
Safety - At most one process may execute in the critical section (CS) at a time.
Liveness - Requests to enter and exit the critical section eventually succeed.
Ordering - If one request to enter the CS happened-before another, then entry to the CS is granted in that order.

8. Coordination and Agreement
Distributed Mutual Exclusion
Algorithms for mutual exclusion
The criteria:
the bandwidth consumed, which is proportional to the number of messages sent in each entry and exit operation;
the client delay incurred by a process at each entry and exit operation;
the algorithm’s effect upon the throughput of the system.
Some examples of algorithms:
The central server algorithm
Ring-Based Algorithm
Multicast and Logical Clocks
Maekawa’s Voting Algorithm

9. Coordination and Agreement
Distributed Mutual Exclusion
The central server algorithm
Employs a server that grants permission to enter the critical section.

10. Coordination and Agreement
Distributed Mutual Exclusion
Ring-Based Algorithm
Arrange the processes in a logical ring

11. Coordination and Agreement
Distributed Mutual Exclusion
Multicast and Logical Clocks
processes that require entry to a critical section multicast a request message, and can enter it only when all the other processes have replied to this message

12. Coordination and Agreement
Distributed Mutual Exclusion
Maekawa’s Voting Algorithm
Processes need to obtain permission to enter from subsets of their peers, as long as the subsets used by any two processes overlap
Not all of the peers
Must collect sufficient votes to enter to the critical section

13. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

14. Coordination and Agreement
Election
An algorithm for choosing a unique process to play a particular role
a process pi can be
a participant : is engaged in some run of the election algorithm
a non-participant : is not currently engaged in any election
Some examples of election algorithms
A ring-based election algorithm
The bully algorithm

15. Coordination and Agreement
Election
A ring-based election algorithm
To elect a single process known as the coordinator (the process with the largest identifier)
The algorithm:
Every process is marked as a non-participant in an election.
Any process can begin an election by marking itself as a participant, placing its identifier in an election message and sending it to its clockwise neighbour
When a process receives an election message, it compares the identifier in the message with its own.
If the arrived identifier is greater, then it forwards the message to its neighbour.
If the arrived identifier is smaller and the receiver is not a participant, then it substitutes its own identifier in the message and forwards it; but it does not forward the message if it is already a participant.
On forwarding an election message in any case, the process marks itself as a participant.

16. Coordination and Agreement
Election

17. Coordination and Agreement
Election
The Bully algorithm
Allows processes to crash during an election, although it assumes that message delivery between processes is reliable
It assumes that the system is synchronous (uses timeouts to detect a process failure)
It assumes that each process knows which processes have higher identifiers, and that it can communicate with all such processes
3 types of message used in this algorithm
an election message - is sent to announce an election;
an answer message - is sent in response to an election message
a coordinator message - is sent to announce the identity of the elected process

18. Coordination and Agreement
Election

19. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

20. Coordination and Agreement
Coordination and Agreement in Group Communication
Each of a group of processes must receive copies of the messages sent to the group
Group communication requires:
Coordination
Agreement: on the set of messages that is received and on the delivery ordering
This chapter will discuss multicast communication of processes whose membership is known (static groups)

21. Coordination and Agreement
Coordination and Agreement in Group Communication
System: contains a collection of processes, which can communicate reliably over one-to-one channels
Processes: members of groups, may fail only by crashing
Closed group
Open group

22. Coordination and Agreement
Coordination and Agreement in Group Communication
Primitives:
multicast(g, m): sends the message m to all members of group g
deliver(m) : delivers the message m to the calling process
sender(m) : unique identifier of the process that sent the message m
group(m): unique identifier of the group to which the message m was sent

23. Coordination and Agreement
Coordination and Agreement in Group Communication
Algorithms for coordination and agreement in group communication
Basic Multicast
Reliable Multicast
Ordered Multicast

24. Coordination and Agreement
Coordination and Agreement in Group Communication
Basic Multicast
Guarantee that a correct process will eventually deliver the message as long as the multicaster does not crash
Primitives: B_multicast, B_deliver
Implementation: Use a reliable one-to-one communication
Unreliable: Acknowledgments may be dropped

25. Coordination and Agreement
Coordination and Agreement in Group Communication
Reliable Multicast
Integrity: A correct process P delivers the message m at most once
Properties to satisfy:
Validity: If a correct process multicasts a message m, then it will eventually deliver m
Agreement: If a correct process delivers the message m, then all other correct processes in group(m) will eventually deliver m
Primitives: R_multicast, R_deliver

26. Coordination and Agreement
Coordination and Agreement in Group Communication
Implementation
Initialization
Correct algorithm, but inefficient
(each message is sent |g| times to each process)
msgReceived := {};
R-multicast(g, m) by p
B-multicast(g, m); // p g
B-deliver(m) by q with g = group(m)
If (m  msgReceived)
Then msgReceived := msgReceived  {m};
If (q  p) Then B-multicast(g, m);
R-deliver(m);

27. Coordination and Agreement
Coordination and Agreement in Group Communication
Ordered Multicast
Categories of ordering:
FIFO ordering
Causal ordering
Total Ordering

28. Coordination and Agreement
Coordination and Agreement in Group Communication
Ordered Multicast
FIFO ordering
If a correct process issues multicast(g, m1) and then multicast(g, m2), then every correct process that delivers m2 will deliver m1 before m2
Process 1
Process 2
Process 3 m2 m3
Time

29. Coordination and Agreement
Coordination and Agreement in Group Communication
Ordered Multicast
Total Ordering
If a correct process delivers message m2 before it delivers m1, then any correct process that delivers m1 will deliver m2 before m1
Process 1
Process 2
Process 3 m1 m2
Time

30. Coordination and Agreement
Coordination and Agreement in Group Communication
Ordered Multicast
FIFO ordering
Causal ordering
Total Ordering

31. Coordination and Agreement
Coordination and Agreement in Group Communication
Ordered Multicast
Causal ordering
If multicast(g, m1)  multicast(g, m3), then any correct process that delivers m3 will deliver m1 before m3
Process 1
Process 2
Process 3 m1 m2 m3
Time

32. Contents Introduction Distributed mutual exclusion Elections
Coordination and agreement
Consensus and related problems

33. Coordination and Agreement
Consensus and Related Problems
Make agreement in a distributed manner
Mutual exclusion: who can enter the critical region
Totally ordered multicast: the order of message delivery
Byzantine generals: attack or retreat?
Consensus (agreement ) problem
Agree on a value after one or more of the processes has proposed what the value should be

34. Coordination and Agreement
Consensus and Related Problems
System model and problem definition
The system model comprises a collection of processes pi ( i = 1, 2, , , , N ) communicating by message passing
To reach consensus, every process pi begins in the undecided state and proposes a single value vi , drawn from a set D ( i = 1, 2, , , , N )
The processes communicate with one another, exchanging values.
Each process then sets the value of a decision variable, di . In doing so it enters the decided state, in which it may no longer change di ( i = 1, 2, , , , N )

35. Coordination and Agreement
Consensus and Related Problems
System model and problem definition

36. Coordination and Agreement
Consensus and Related Problems
Requirements for consensus:
Termination—eventually every correct process sets its decision variable
Agreement—the decision value of all correct processes is the same
Integrity—if the correct processes all proposed the same value, then any correct process in the decided state has that value.
Consensus problems:
The Byzantine generals problem
Interactive consistency problem

37. Coordination and Agreement
Consensus and Related Problems
The Byzantine generals problem
The Byzantine Empire was fraught by frequent infighting among rival military leaders for control of the empire.
Several generals had to cooperate to achieve an objective, a treacherous general could weaken or even eliminate a rival by retreating and encouraging another general to retreat while encouraging the rival to attack. Without expected support, the rival was likely to be defeated.
The Byzantine Generals problem concerns decision making in anticipation of an attack.

38. Coordination and Agreement
Consensus and Related Problems
Here is the Byzantine Generals problem:
Three or more generals must agree to attack or retreat
One general, the commander, issues the order
Other generals, the lieutenants, must decide to attack or retreat
One or more generals may be treacherous
A treacherous general tells one general to attack and another to retreat
Difference from consensus is that a single process supplies the value to agree on

39. Coordination and Agreement
Consensus and Related Problems
Interactive consistency problem
A problem related to the Byzantine Generals problem is interactive consistency. In this, all correct processes agree on a vector of values, one for each process. This is called the decision vector

40. Coordination and Agreement
Consensus and Related Problems
Consensus (C), Byzantine Generals (BG), and Interactive Consistency (IC) are all problems concerned with making decisions in the context of arbitrary or crash failures.
Solutions can be generated for one problem in terms of another.
IC can be derived from BG by running BG N times, once for each process with that process acting as commander.
C can be derived from IC by running IC to produce a vector of values at each process, then applying a function to the vector’s values to derive a single value.
BG can be derived from C by
Commander sends proposed value to itself and each remaining process
All processes run C with received values
They derive BG from the vector of C values

41. End of the Chapter…