Timed Distributed System Models  A. Mok 2014 CS 386C

System Attributes 1) Synchrony 1) Synchronous 2) Asynchronous 3) Mixed 2) Time source 1) Global clock / Local clocks 2) Real-time / Logical Time 3) Failure semantics 4) Communication framework 1) Message passing / Shared memory 2) Latency / Buffering 3) Addressing and routing

The Two Generals Problem Two honest Generals, X and Y want to coordinate an attack by passing messages The message passing system has performance failure semantics At midnight, X sends a message to inform Y that he wants to attack at a certain time, say at dawn Y replies with a “Agreed” message Y’s reply may be late (after dawn), lost or even early (before midnight – assuming arbitrary failure semantics). What can the two generals do to assure a simultaneous attack?

Two Generals with Unsynchronized Clocks At midnight, X sends a message to inform Y that he wants to attack at dawn. Y replies with a “Agreed” message Y’s reply may be late (after dawn) or lost If Y’s reply is late, X may not attack at dawn Even if X gets a reply before dawn, Y does not know if X has gotten his reply before dawn Even worse, if X and Y’s clocks are unsynchronized, they cannot attack at the same time at dawn So let us at least give them synchronized clocks

Two Generals with Synchronized Clocks At midnight, X sends a message to inform Y that he wants to attack at dawn. Y replies with a “Agreed” message Y’s reply may be late (after dawn) or lost or can it even be early (before midnight)? If Y’s reply is late, X may not attack at dawn Even if X gets Y’s reply before dawn, Y does not know if X has gotten his reply before dawn So Y waits for a confirmation from X. Should Y attack after getting a confirmation from X before dawn? But then Y will not know if X knows that Y has received the confirmation … We are back to square one

Two Generals with Synchronized Clocks and Message Counter (1) To keep track of the messages, let us introduce a message counter: [count] initialized to 1 At midnight, X sends a message [1]”Attack at Dawn?” to inform Y that he wants to attack at dawn. Y replies with a [2]”Agreed” message Y’s reply may be late (after dawn), lost or even early (before midnight). If Y’s reply is late, X may not attack at dawn. Even if X gets a reply before dawn, Y does not know if X has gotten his reply before dawn. But if X gets Y’s reply, X knows that Y must have gotten the [1]”Attack at Dawn?”. X replies to Y’s [2]”Agreed” message with [3]”Confirmed” What happens now?

Two Generals with Synchronized Clocks and Message Counter (2) At midnight, X sends a message [1]”Attack at Dawn?” to inform Y that he wants to attack at dawn. Y replies with a [2]“Agreed” message. If X gets Y’s reply, X knows that Y must have gotten the [1]“Attack at Dawn?”. X then replies to Y’s [2]”Agreed” message with [3]“Confirmed” message. If Y gets [3]“Confirmed” from X, Y knows X has gotten his [2]“Agreed” message. So Y knows that X has agreed to attack. But Y does not know that X knows that Y agrees to attack. Does this matter? It does matter if Y will not attack unless Y can confirm to X X’s [3]“Confirmed” message. This message exchange pattern could go on forever

What Do the Generals Know? If X gets message [2], both X and Y know that both messages [1] and [2] have been sent, and [1] has been received. But only X knows that [2] has been received. If Y gets message [3], both X and Y know that messages [1], [2] and [3] have been sent, and [1] and [2] have been received. But only Y knows that [3] has been sent and received. Messages exchanged so far [1] “Attack at Dawn?” Both know message [1] has been sent and received [2] “Agreed” Both know message [2] has been sent and received [3] “Confirmed” Both know message [3] has been sent but only Y knows it has been received What is the least knowledge that both X and Y must know before they can attack with full confidence of the other general’s support?

Minimum Sufficient Mutual Knowledge A message is mutual knowledge if both X and Y know that both parties know the contents of the message What is the minimum mutual knowledge that is sufficient for both X and Y to decide that they can attack with full confidence of the other general’s support? Both X and Y must know that they are acting on the same set of messages M and that they will make the same decision based on the contents of M Suppose X and Y have a pre-agreement that both will act on messages [1] and [2] after they are sure that messages [1] and [2] are mutual knowledge. Then X and Y can both attack at dawn with full confidence! But they do not have such a pre-agreement. So the message exchange will go on.

Attaining Mutual Knowledge If enough messages have been received, both parties can determine from the message count that for some n, both parties must know that at least the first n messages have been received by both parties, i.e., the first n messages is mutual knowledge. Mutual knowledge attained after message 4 has been sent and received: X→Y: [1] “Attack at Dawn?” Y→X: [2] “Agreed” X→Y: [3] “Confirmed” Y→X: [4] “Reconfirmed” After Y receives message [3], Y knows that X must have received message [2] which implies that Y knows that X has sent and therefore must know message [1]. Likewise, after X receives message [2], X knows Y has sent message [2], and so X knows that Y must know message [1]. Message [1] is now mutual knowledge.

Probabilistic Guarantee In the absence of any pre-agreement, how many messages must be sent to attain high confidence that both generals will attack simultaneously if every message can be lost with a given probability p? Eventually, both generals might attain sufficient mutual confidence to attack but then they might not in any given finite time. This is determined by the probabilistic assumption about message loss and the number of duplicate messages that can be sent before the deadline for taking joint action (before dawn when the attack is supposed to occur). An optimization is to resend a confirmation only if an expected confirmation does not arrive. This way, absence of message would be evidence of consensus. Use sufficient long time-outs to ensure high confidence.

Timed Asynchronous System Model Assumptions 1) Network topology Every process is known to every other process Communication is by messages. Automated routing assumed 2) Synchrony Service times have known upper bounds Local clocks have bounded drift with known rates 3) Failure semantics Processes have crash or performance failures Message delivery has omission or performance failures 4) Message buffering Finite message buffers. Buffer overflow does not block sender FIFO message delivery not assumed