1. Data Link Protocols Reliable FIFO communication using less reliable channels By Ken Schmidt

2. Protocols discussed Stennings Protocol –Sent with unbounded integer tags –Tolerates loss –Duplication –reordering Alternating Bit Protocol –Sent with bounded tags –Tolerates loss –Duplication –Does NOT tolerate reordering Bounded Tag Protocol –That tolerates reordering Tolerating Crashes

3. Message Levels Two levels User –High-level –Alphabet M –SEND, RECEIVE, etc upper case Underlying channel –Low-level –Alphabet M’ –send, receive, etc lower case

4. How are these protocols used? OSI Model Application Layer Presentation Layer Session Layer Transport Layer Network Layer Data Link Layer Physical Layer TCP-speed up, slow down IP addr: 192.168.1.1 MAC addr: 00:60:08:C4:99:AA Cat 5 cable FTP SEND send RECEIVE receive

5. Data Link Protocol Universal Reliable FIFO Channel RECEIVE 1-2 P1P1 P2P2 SEND 1-2 C 2,1 C 1,2 send 1,2 send 2,1 receive 1,2 receive 2,1 Unreliable FIFO Channel (The Internet)

6. Loss Definitions SLL: Strong loss limitation – fair to a particular message (infinitely many receive events all caused by the same send event WLL: Weak loss limitation – fair to messages as a group ( infinitely many sends cause receives of infinitely many messages)

7. Three types of unreliability in the channel Loss of a message Duplication of messages Reordering of messages

8. Stenning’s Protocol Tolerates all three types of channel unreliability P 1 puts high-level message from U 1 in buffer 1, tags with unbounded successive integers starting with 1 Repeatedly sends first message in buffer 1, along with it’s tag, to P 2 P 2 accepts the first message tagged with 1 that it receives P 2 accepts subsequent messages each one of which has a tag that is one greater than the previously accepted one P 2 places messages in buffer 2 for reliable, in order, delivery to U 2 P 2 acks a high-level message repeatedly by sending its tag back to P 1 When P 1 receives an ack for its current tag, it moves to the next high-level message 694

9. Lemma 22.2: Let  be any execution of Stenning 1.  |P 1 &  |P 2 are fair 2.  |A 1,2 &  |A 2,1 satisfy liveness Then  |ext(F)  fairtraces(F) Show that any high level message that is submitted to P 1 eventually gets to P 2 By contradiction: suppose that a message is submitted but not delivered, k is the tag, so the tag 2 remains <= k-1 forever The message reached the front of buffer 1 (to be sent), if k=1, then it is the 1 st message, if k>=1 then previous message was received by P 2 and delivered to U 2 Tag 2 eventually reaches and stays at k-1 By fairness of P 2, P 2 keeps sending k-1 messages forever and by weak loss limitation (WLL), a copy of k-1 is received by P 1, the message with tag = k-1 gets removed from buffer 1 and m reaches the front Once at the front, it stays there forever because we supposed that it never reached P 2, so P 1 keeps sending forever (m,k) WLL implies that eventually a copy of (m,k) is received and accepted by P 2, a contradiction This is the main correctness result for Stenning protocol

10. Alternating Bit Protocol Similar to Stenning but with tags bounded to 0,1 (Stenning integer tags replaced with the low order bit) Can loose or duplicate messages,can NOT reorder them P 1 places high level message in buffer 1 tag alternating 0-1 P 1 repeatedly sends the 1 st message in its buffer & tag to P 2 P 2 accepts first message tagged with 1 it receives and each subsequent message if its tag is different from the tag of the previously accepted message P 2 places message in buffer 2 and delivers it to U 2 P 2 acks the message repeatedly by sending its tag back to P 1 When P 1 receives and ack for its current tag, it begins processing the next message Satisfies lossy FIFO channel of 14.1.1 462 (limited loss, finite duplication, no reordering)

11. Premise of proofs for ABP is that ABP maps exactly to Stenning for all buffers and tags except that ABP tags are Stenning tags mod 2 And where ABP specifies a lossy FIFO channel and Stenning uses a lossy reordering FIFO channel to obtain reliable FIFO delivery Similarity between Stenning and ABP for proofs

12. Bounded Tag Protocols Tolerating Reordering What goes wrong with ABP Channels that reorder low level messages can get fooled into accepting an old high level message that happens to arrive tagged with the same bit as the one currently expected Can cause duplicate delivery of a high level message, violating requirements for reliable communication – ABP does not work with channels that can reorder low level messages P 1 P 2 (m,1) (m’,0) accept m accept m’

13. Impossibility of a protocol that both Reorders and Duplicates Theorem 22.8: There is no bounded tag protocol that guarantees reliable FIFO delivery using channels that can reorder and duplicate messages By contradiction, suppose there is such an implementation, construct an execution with incorrect behavior Run the system until it is not possible for Process P 1 to send any additional low-level messages with new values (a finite extension of  1 ) Let  2 be a fair consistent extension of  1 with one additional SEND event for a total of n+1 SEND events

14. By correctness, all messages submitted by U 1 in  2 must eventually get delivered to U 2, so that there are n+1 RECEIVE events in  2 Let  3 be a prefix to  2 up to an including the last RECEIVE event Construct a finitely consistent execution  4 where –  4 is an extension of  1 –  4 is indistinguishable from  1 to P 1 –  4 is indistinguishable from  3 to P 2 Construct  4 by preventing all events involving P 1 immediately after  1 while allowing all events of P 2 to proceed exactly as in  3 The additional events of P 2 might include receive events, send events, and internal events, as well as the required RECEIVE event Show that P 2 can be permitted to receive the same low level message after  1 as it dos in  3 even though P 1 does not send any additional low level messages after  1 This is possible because all low level messages sent by P 1 in  3 after  1 contain values that P 1 has already sent in  1 Any low level message received by P 2 after  1 could be a duplication of some low level message sent in  1 In  4 there are exactly n SEND events and n+1 RECEIVE events Extend  4 to a fair consistent execution without introducing any new SEND events, the resulting execution has more RECEIVE events than SEND events contradicting correctness.

15. Bounded Tag Protocol Tolerating Loss and Reordering Layer 1 Layer 2 Layered structure of the Probe protocol 707 I 1,2 I 2,1

16. Probe algorithm Layer 1 uses the given channels to implement intermediate channels I 1,2 and I 2,1 that do not reorder messages, but can lose or duplicate them (satisfies the cause function) Cause function is required not to reorder messages, but to satisfy the WLL, infinite duplication is allowed Layer 2 uses the resulting FIFO channels to implement reliable FIFO communication

17. Layer 1 Layer 2 SEND 1-2 RECEIVE 1-2 send 1,2 receive 2,1 receive 1,2 send 2,1 I 1,2 I 2,1 Layer 2, ABP can be used I 1,2 and I 2,1 are not only lossy FIFO channels, but also allow infinite duplication

18. In Each Layer 1 implementation: Process P 1 sends a low level message to P 2 only in response to an explicit probe message from P 2 The low level message P 1 sends always contains the value of the most recent high level message that it has received form U 1 which it keeps track of in latest e.g.:P 1 does not remember all messages submitted by U 1, just the most recent one (I channel is permitted to lose some high level messages) To ensure that P 1 only sends messages in response to probes, P 1 keeps a variable unanswered which it increments when it receives a probe and decrements whenever it sends a low level message Process P 2 continually sends probes to P 1, keeping track in pending the # of probes it has ever sent P 2 counts in count(m) # of copies of each high level message received since the last time it delivered a high level message to U 2 Whenever it delivers a msg to U 2, P 2 sets old to pending When count(m) exceeds old, P 2 can output m

19. P 1 P 2 pending ++ (= 1) probe unanswered ++ (= 1) pending ++ (= 2) probe unanswered ++ (= 2) SEND(m) latest = m unanswered - - (= 1) note: init old=0 send (m = latest) count ++ (= 1) now, count > old so: RECEIVE reset count =0 old = pending = 2 unanswered - - (= 0) send (m = latest) count ++ (= 1) since no duplication, if count<old received, must have been sentno RECEIVE Layer 1(p710) can loose and reorder, cannot duplicate Layer 2(ABP) can loose and duplicate, cannot reorder unanswered = probes received - # of sends pending = total no of probes ever sent count = # msg received since last RECEIVE old = # of receives before a RECEIVE can occur Now for the next SEND, the # of sends must exceed old before one is RECEIVEd For instance, the next SEND would require 3 probes and 3 sends before a RECEIVE could occur, since there are no channel duplicated messages, at least one of these 3 must have been sent by P 1 as a new message since the last RECEIVE. & Layer 2 no reordering

20. Lemma 22.9: Layer 1 of the probe protocol, with the given nonduplicating channels implements the intermediate channel I 1,2 I 1,2 does not reorder messages: when P 2 performs any RECEIVE after the first one, it checks that count(m)>old where m is the high level msg being delivered Old variable plus the fact that P 1 only sends msg in response to probe and channels do not duplicate msg, imply at most old low level msg in transit from P 1 to P 2 at the point of the preceding RECEIVE event At least one of the msg containing m must have been sent by P 1 since the preceding RECEIVE event Implies that m must have been the value of latest 1 at some point after the preceding RECEIVE event, implying no reordering occurs I 1,2 guarantees the WLL condition, if there are infinitely many SEND events, then infinitely many of them must have corresponding RECEIVE events Suppose infinitely many SEND events, P 2 keeps sending probes P 1 keeps responding to probe messages, liveness for the channel and the finiteness of the high level message alphabet M imply that P 2 performs infinitely many RECEIVE events Any msg that is delivered to U 2 after the first one must have the value of latest at some point after the previous RECEIVE event, implying that RECEIVE events correspond to infinitely many different SEND events

21. Complete probe needs SLL Theorem 22.10: The probe protocol, using any nonduplicating SLL channels implements the reliable FIFO channel F For every fair execution ,  |ext(F)  fairtraces(F) Complexity Can require more and more low level messages to deliver later and later high level messages Layer 1, once k low level messages have been lost, it requires at least k +1 low level messages to deliver each subsequent high level message even if no further losses occur

22. Nonexistence of Efficient Protocols Tolerating Loss and Reordering Definition of k-extension: if  is a complete execution (# of SEND and RECEIVE events are equal) –there is an extension  ’ after  which contains exactly one SEND and one RECEIVE –no old messages are received in  ’ –# of receive events in  ’ after  <= k (k-message bounded) Theorem 22.11: There is no message bounded protocol that implements a reliable FIFO communication using channels that can lose and reorder messages –By contradiction, assume there is such a channel that is k bounded –Similar to Theorem 22.8, produce a finitely consistent execution  1 –  1 is an extension of  –  1 is indistinguishable from  in P 1 –  1 is indistinguishable from  ’ in P 2 –Do this by preventing all events involving P 1 immediately after  while allowing all events of P 2 to proceed exactly as in  ’ –Because messages already in transit after  are received at P 2 –Contradiction: there are more RECEIVE events than SEND events

23. Tolerating Crashes 1.Crash with later recovery where process just resumes where it left off, no different from a process that pauses 2.Crash where information in the state is lost (from volatile memory) and the process later recovers Run a recovery protocol to restore info from stable memory to restore the volatile memory (recover step) Input action CRASH and output RECOVER, if CRASH occurs, enables RECOVER and disables everything else until RECOVER completes RECEIVE 1-2 P1P1 P2P2 SEND 1-2 C 2,1 C 1,2 send 1,2 send 2,1 receive 1,2 receive 2,1 CRASH 1 CRASH 2 RECOVER 1 RECOVER 2

24. A Simple Impossibility Result RECOVER sets the entire state back to some arbitrary start state, all state info is lost Easy to see that is impossible to solve the reliable FIFO communication problem even with reliable FIFO channels –After a crash of P 2, the protocol is unable to tell whether or not a high level message has just been delivered to U 2

25. A Harder Impossibility Result Weaken the problem statement by requiring less at the external interface: no duplication but allows reordering, and only require that a message be delivered if its SEND event has no following RECOVER event (infinitely many CRASH and RECOVER events, no messages are required to be delivered at all, if finitely many events, any message sent after the last RECOVER must be delivered) Do not permit duplication or reordering, all channels can do is lose messages Theorem 22.14: There is no protocol using lossy low level channels that guarantees communication with no duplication and with no losses after all crashes and recoveries have ceased It is possible, using crashes, to create a situation in which both processes have the same states that they had at the end of , but in which one of the channels has in transit the entire sequence of low level messages sent along that channel in 

26. Notation ī is the opposite process to i Claim 22.15 Let  be any crash free finitely consistent execution, either  contains no steps or the last step in  is a step of P i, then there is a finitely consistent execution of  ’ where –The state of P i is the state ( , i) –The state of P ī is the state ( , ī) –The sequence out( , i) is in transit from P i to P ī –Execution  ’ may contain CRASH and RECOVER events, but there are no unmatched CRASH events, i.e.: each CRASH has a following corresponding RECOVER Use claim 22.15 to complete Theorem 22.14 –Let  be any crash free finitely consistent execution containing exactly one SEND event and its corresponding RECEIVE event and assume without loss of generality that  ends with the RECEIVE event –Construct  1 whose final process states are the same as those in  but that has a SEND as its last external event (no following SEND, RECEIVE, CRASH, or RECOVER events and no unmatched CRASH events) 721

27. –Claim 22.15 yields a finitely consistent execution  ’ that ends with the process state equal to state ( , 1) and state ( , 2) and with out( , 2) in transit from P 2 to P 1 and no unmatched CRASH events –Construct  1 by extending  ’ by crashing and recovering P 1 then running it on its own just as in  (with the input sequence in( , 1) is in transit in the incoming channel) –This allows P 1 to reach state( , 1) again (is a SEND step but no other user interface step in the portion of  1 after  ’, yielding the claim properties for  1 ) –Contradiction: let  2 be an extension of  1 to a fair consistent execution that contains no further SEND, CRASH, or RECOVER events and in which every low level message received after  1 is send after  1 (all old low level messages are lost) –By correctness, there is at least one RECEIVE event in the suffix, to correspond with the last SEND in  1 (the portion of  2 after  1 could also be attached after  yielding a fair consistent execution because the two processes are in the same states after  and  1 and because all old low level messages are lost –This violates correctness, since  already has an equal number of SEND and RECEIVE events (one of each) and the suffix contains at least one more RECEIVE but no more SENDs)

28. A Practical Protocol It is important in practice to have reliable FIFO message delivery which will work in spite of crashes FivePacketHandshake Protocol is used in setting up network connections, used in TCP, ISO TP-4, NCP and other transport protocols (packet = low level message)

29. P 1 maintains a buffer of high level message submitted by U 1 and works to get the message to P 2 one at a time For each high level message that P 1 tries to send, there is an initial two way exchange of low level packets P 1 first sends a new UID v to P 2 in a (“needuid”,v) packet P 2 pairs this UID v with another new UID u that it generates and sends the pair (u, v) back to P 1 in an (“accept”, u, v) P 1 can now recognize that this packet is recent because it contains P 1 ’s latest UID v P 1 then chooses u as the UID for the high level message it is trying to send P 1 sends the latest high level message m to P 2 paired with the new UID u in a (“send”, m, u) packet P 2 can recognize that the packet is recent because it contains P 2 ’s latest UID u After accepting a message, P 2 acks the packet with a (“ack”, u) P 1 sends a (“cleanup”, u) to P 2 to tell P 2 to discard the current UID P 1 P 2 “needuid”, v “accept”, u, v “send”, m, u “ack”, u “cleanup”, u

30. Theorem 22.16: FivePacketHandshake guarantees message delivery using channels that lose, reorder and duplicate messages with SLL and finite restrictions on duplications Safety: protocol does not reorder or duplicate messages is easy to see that the use of UIDs u and v prevent this from happening Liveness: There is a situation where P 1 has status = needuid while P 2 has status = accept but UID v = value v that is different from P 1 s current value of UID v (any current needuid packet from P 1 is ignored by P 2 Show that v is eventually dislodged –Fairness for P 2 implies that P 2 sends infinitely many (“accept”, u, v) packets –Channel liveness implies that infinitely many of these arrive at P 1 which cause a (“cleanup, u) to be sent –Channel liveness implies that eventually one of these (“cleanup, u) packets must arrive at P 2 dislodging v –After v is dislodged, it is possible for P 2 to acquire another value of UID v that is also not current value at P 1, which can happen if P 1 receives an old needuid packet. –This old value v is dislodged as above, which can happen only finitely many times (because channels only allow finite duplication), so eventually a current needuid packet arrives at P 2

31. Eventual quiescence –In case there are only finitely many SEND, CRASH, and RECOVER events, eventually both processes reach and remain in states that are the same as their initial states, except for the used sets –No memory needs to be reserved for the use of this protocol when it is not actively processing messages from U 1 to U 2 –Allows the same pair of processes in a network to simulate the FivePacketHandshake protocol in parallel for a very large number of distinct pairs (U 1, U 2 ) of users Finite UID sets –The number of available UIDs is very large but not infinite –UIDs can be chosen to be successively increasing integers mod n for some very large n –Finite version works correctly (and can be proven via simulation relation) provided that UIDs wrap around and old values of u have already been discarded –This is possible because of known limits on the message delivery time, local processing time and rate of submission of high level messages, or of an explicit policy of discarding old packets