Self-Stabilizing End-to-End Communication in Bounded Capacity, Omitting, Duplicating and non-FIFO Dynamic Networks Shlomi Dolev 1, Ariel Hanemann 1, Elad M. Schiller 2, and Shantanu Sharma 1 1 Department of Computer Science, Ben-Gurion University of the Negev, Israel 2 Department of Computer Science and Engineering, Chalmers University of Technology, Sweden 14th International Symposium on Stabilization, Safety, and Security of Distributed Systems (SSS 2012) Toronto, Canada (1-4 October 2012)

Outline Introduction Problem Specification The Algorithm Key Features Conclusion 2 of 27

End-to-end communication is an essential task in communicating systems Intromission of errors in message transmission is common Dynamic networks have a greater chance of having errors, like, omission, duplication, reordering, than conventional networks Network has known capacity constant, which is an upper bound on number of packets in the network Introduction S R Omission of packets S R Duplication by the sender S R Duplication by the channel S R Reorder S R capacity of the channel S R capacity packets capacity of the channel

Self-Stabilization – A self-stabilizing system is a system that can automatically recover following the occurrence of transient faults – e.g.-token based mutual exclusion algorithm Introduction 4 of 27

Outline Introduction Problem Specification The Algorithm Key Features Conclusion

Problem Specification Design an algorithm for a bounded capacity non-FIFO network which – Guarantee exactly one copy of packet delivery – Tolerate corruption, omission, duplication of messages by the channel and the sender – Deliver messages in the same order they were sent – Applied to topological independent network – Employed to any starting configuration 5 of 27

Problem Specification Why new end-to-end communication protocol? – TCP is not self-stabilizing 6 of 27

Outline Introduction Problem Specification The Algorithm Key Features Conclusion

The Algorithm-Motivation Dolev et al* has presented self-stabilizing data link algorithm for reliable FIFO packet delivery over bounded non-FIFO and non-duplicating channel The sender sends a message infinitely often until it gets 3capacity + 2 identical acknowledgements The receiver delivers a message when it gets identicalcapacity + 1 messages Shlomi Dolev, Swan Dubois, Maria Potop-Butucaru, and Sébastien Tixeuil. Stabilizing data-link over non-fifo channels with optimal fault-resilience. Inf. Process. Lett., 111(18):912–920, * 7 of 27

The Algorithm-Motivation SR Alternating bit ab = 0 Now, the receiver delivers the message m only once and changes last_delivered = 0 Last delivered alternating bit last_delivered = 1 m, 0 ack(m, 0) 1 st message 2 ed message 3 rd message capacity+1 message m m m m 1 2 capacity capacity+1 3capacity+1 3capacity+2 Acknowledgments Messages capacity messages The channel may contain capacity (or zero) messages in the channel from sender to receiver Hence, the sender sends capacity +1 messages to the receiver. capacity messages The channel may contain capacity (or zero) messages in the channel from receiver to sender Hence, the sender waits 3capacity+2 acknowledgement messages capacity acks for message in the channel from sender to receiver capacity+1 acks for the message sent by sender capacity acks for message in the channel from receiver to sender 1 ack for bad initialization m, 0 ack(m, 0) Last delivered alternating bit last_delivered = 0 But, the sender sends message until it gets 3capacity+2 acknowledgements. However, the receiver acknowledges all incoming message but do not store (due to same last_delivered=0) m, 0 ack(m, 0) m, 0 ack(m, 0) Now, the sender fetches new message, changes alternating bit to 1 and repeat the whole process 8 of 27

The Algorithm-Motivation Disadvantages Dolev et al*: – Explicit synchronization of the sender and the receiver – One packet delivery requires lots of packet flow overburden network – Does not consider duplication by the channel Shlomi Dolev, Swan Dubois, Maria Potop-Butucaru, and Sébastien Tixeuil. Stabilizing data-link over non-fifo channels with optimal fault-resilience. Inf. Process. Lett., 111(18):912–920, * 9 of 27

S 2 E 2 C Algorithm S 2 E 2 C (Self-Stabilizing End-to-End Communication) Algorithm – Overcomes all the cons of Dolev et al (presented previously) – Sends n messages (packets) infinitely often instead of one message at a time – Consider the duplication by the channel – Message to packet transformation 10 of 27

SR Alternating Index AltIndex s = 1 LastDeliveredIndex r = 0 Now, the receiver selects majority of the message s and deliver it Packet_set 1,0,m 1,1,m (2,2,m) (1,2,m) 1,n-1,m (1, 0, m) (1, 1, m) (0, 2, m) (1, 2, m) 2,2,d 3 1,capacity,m (2, 3, m) (1, 3,m) (1,3,m) (2,3,m) (1, capacity, m) (1, capacity+1, m) 1,capacity+1,m (1, n-1, m) m Correct messages S 2 E 2 C Algorithm (1, capacity, m) 2,3,m 1,2,m 1,3,m After the delivery of the correct messages, the receiver acknowledges the sender. Alternating Index Label Data (2, 2, m)(1, 0, m) Transmission of messages from the sender to the receiver. Capacity With only one message 11 of 27

The Algorithm-The Idea Distinct Labels with each outgoing packets – prevent duplication and guarantee in-order delivery of messages as they were sent Alternating index 0, 1 or 2 – avoids explicit synchronization between the sender and the receiver The receiver accumulates packets of n distinct labels with identical alternating index before the delivery of messages The sender waits for capacity+1 distinct labels acknowledgements before the next fetch 12 of 27

SR Alternating Index AltIndex s = 1 LastDeliveredIndex r = 0 Now, the receiver selects n distinct labels which have identical alternating index except LastDeliveredIndex. ACK_set 0 1 Packet_set 1,0,d 1 1,1,d 2 (2,2,d 3 ) (1,2,d 3 ) 1,n-1,d n-1 (1, 0, d 1 ) (1, 1, d 2 ) (0, 2, d 3 ) (1, 2, d 3 ) 2,2,d 3 1,capacity,d c (2, 3, d 4 ) (1, 3, d 4 ) (1,3,d 4 ) (2,3,d 4 ) (1, capacity, d c ) (1, capacity+1, d c+1 ) 1,capacity+1,d c+1 (1, n-1, d n-1 ) 1 st message 2 ed message 3 rd message pl th message Messages with errors 1 st message 2 ed message 3 rd message pl th message Correct messages 0, 1 1, 1capacity, 1 S 2 E 2 C Algorithm capacity (1, capacity, d c ) 2,3,d 4 1,2,d 3 1,3,d 4 After the delivery of the correct messages, the receiver acknowledges the sender. Alternating Index Label Data (2, 2, d 3 )(1, 0, d 1 ) Transmission of messages from the sender to the receiver. Transmission of acknowledgements from the receiver to the sender. Capacity 13 of 27

1 st message 2 ed message 3 rd message pl th message ml The sender fetches pl messages of ml length The Algorithm at Sender 14 of 27

1 st message 2 ed message 3 rd message pl th message ml 1 st bit 2 ed bit 3 rd bit ml th bit Error Correcting Encoding Error Correcting Bits n > ml AltIndex lbl 1 st Packet 2 ed Packet 3 rd Packet n th Pacet The Algorithm at Sender The sender appends redundant bits to each pl message which results in message length n (n>ml) Consider all the bits of each pl messages ai ml n-1 n n 15 of 27

The Algorithm at Sender The transmitting packet has the form of ai, lbl, dat ai is alternating index lbl is labels dat is data which is formed by i th bit of all messages The sender waits for capacity + 1 distinct label acknowledgements before next fetch The sender changes its alternating index x to x+1 mod 3, at every fetch 16 of 27

The Algorithm at Receiver The receiver collects packets of n distinct labels which have alternating index different from last delivered index – capacity distinct labels may be from the channel – n – capacity distinct label packets must be from the sender The receiver forms messages form all incoming packets i th message is formed by the i th bit of every packet ai ml n-1 n n ai

The Algorithm at Receiver After delivery of the message – Resets data structure – changes LastDeliveredIndex to the value of delivered message alternating index The receiver acknowledges the sender withcapacity + 1 distinct labels and the value of LastDeliveredIndex 18 of 27

Outline Introduction Problem Specification The Algorithm Key Features Conclusion

Key Features Handles capacity corruptions in the messages Does one time delivery of message with handling of omission and reorder, i.e., simulate reliable FIFO message delivery over the non-FIFO network No explicit synchronization between the sender and the receiver No need of denotative reset operation at the sender and at the receiver Topological independence 19 of 27

Key Features Handles capacity corruptions in the messages The intromission of error correcting bits is capable to prevent capacity errors or loss of capacity packets capacity corruption is equivalent to loss of distinct capacity labels packets ai ml n-1 n n ai 20 of 27

Key Features Handles capacity corruptions in the messages Does one time delivery of message with handling of omission and reorder, i.e., simulate reliable FIFO message delivery over the non-FIFO network No explicit synchronization between the sender and the receiver No need of denotative reset operation at the sender and at the receiver Topological independence

Key Features Does one time delivery of message with handling of omission and reorder – The omission of packets is prevented by error correcting bits – The duplication of messages is done by either channel or the sender The channel can duplicate up to capacity packets The sender duplicates each packets infinity often – Distinct labels ensures one time order delivery of all the messages 21 of 27

Key Features Handles capacity corruptions in the messages Does one time delivery of message with handling of omission and reorder, i.e., simulate reliable FIFO message delivery over the non-FIFO network No explicit synchronization between the sender and the receiver No need of denotative reset operation at the sender and at the receiver Topological independence

Key Features No explicit synchronization between the sender and the receiver – Synchronization of alternating index is necessary due to arbitrary initialization – If the sender and the receiver are having same alternating index, then the receiver does not deliver incoming messages. But, it acknowledges the sender by capacity + 1 distinct labels – On reception of capacity + 1 distinct labels, the sender changes alternating index and fetches new messages 22 of 27

Key Features No explicit synchronization between the sender and the receiver – Why alternating bit 0 and 1 is not useful in implicit synchronization? SR Alternating Index AltIndex s = 0 LastDeliveredIndex r = 0 ACK_set 0,0 1,0 capacity-1,0 Packet_set Alternating Index AltIndex s = 1 1,0,* 1,1,* 1,2,* capacity,0 (0, 0, *) 1,n -2,* (1,n-1,*) 1,n-1,* The receiver changes LastDeliveredIndex r to 1 and resets Packet_set LastDeliveredIndex r = 1 0,0,* 0,1,* 0,2,* 0,n -2,* Now, the sender sends packets with AltIndex s 0 (0, 1, *) (0, 2, *) (0, n-2, *) The sender gets acknowledgement from the channel capacity,0 Now, the sender resets and changes AltIndex s 0,1 1,1 capacity-1,1 The receiver acknowledges the sender 1,1 Capacity-1,1 This situation is same as initial starting configuration, where the sender and the receiver are having alternting bit 0. Now, we are having alternating bit 1 at both the end Such situation is successfully handled by alternating index 0, 1, 2 23 of 27

Key Features No explicit synchronization between the sender and the receiver – Use of alternating index 0, 1 and 2 SR Alternating Index AltIndex s = 1 LastDeliveredIndex r = 1 ACK_set 0,1 1,1 capacity-1,1 Packet_set Alternating Index AltIndex s = 2 0,0,* 0,1,* 0,2,* capacity,1 (1, 0, *) 0,n -2,* (0,n-1,*) 0,n-1,* The receiver changes LastDeliveredIndex r to 0 and resets Packet_set LastDeliveredIndex r = 0 1,0,* 1,1,* 1,2,* 1,n -2,* 1,n-1,* Now, the sender sends packets with AltIndex 1 (1, 1, *) (1, 2, *) (1, n-2, *) The sender gets acknowedgement from the channel Capacity, 1 Now, the sender resets and changes AltIndex to 2 The receiver gets packet form the channel (1,n-1,*) The receiver changes LastDeliveredIndex r to 1 and resets Packet_set Now, the sender and the receiver are synchronized 0, 1 24 of 27

Key Features Handles capacity corruptions in the messages Does one time delivery of message with handling of omission and reorder, i.e., simulate reliable FIFO message delivery over the non-FIFO network No explicit synchronization between the sender and the receiver No need of denotative reset operation at the sender and at the receiver Topological independence

Key Features No need of denotative reset operation at the sender and at the receiver – In arbitrary initialization, the sender and the receiver may have some garbage in its data structure then both do reset – The reset process for garbage items is only required at arbitrary stages 25 of 27

Key Features Handles capacity corruptions in the messages Does one time delivery of message with handling of omission and reorder, i.e., simulate reliable FIFO message delivery over the non-FIFO network No explicit synchronization between the sender and the receiver No need of denotative reset operation at the sender and at the receiver Topological independence 26 of 27

Outline Introduction Problem Specification The Algorithm Key Features Conclusion

A self-stabilizing end-to-end communication algorithm Topological independent, fits for dynamic networks Implicit sender/receiver synchronization Simulate perfect FIFO behavior – in-order delivery, avoid omission, duplication by the sender, duplication by the channel - over non-FIFO networks Interested audience may refer the paper with the same title for details and proofs. 27 of 27

Shlomi Dolev 1, Ariel Hanemann 1, Elad M. Schiller 2, and Shantanu Sharma 1 1 Department of Computer Science, Ben-Gurion University of the Negev, Israel 2 Department of Computer Science and Engineering, Chalmers University of Technology, Sweden Presentation is available at