1. Linearizing Peer-to-Peer Systems with Oracles by Rizal Mohd Nor Mikhail Nesterenko Sébastien Tixeuil SSS 2013 Nov 13-16, 2013

2. SSS 2013 Introduction Fundamental limits on construction of structured peer-to-peer systems in asynchronous [1]  inability to connect a disconnected network  discard peer identifiers that are not present in the system Studying these limits  identify peer-to-peer system specific oracles and isolate the source of impossibility  show minimality of oracles and proving their necessity for solution existence  provide an oracle-based algorithm Why Linearization (topological sort)  foundational for most popular peer-to-peer systems as construction starts with topologically sorting the peers in the system  similar to consensus as one can observe how it pertain to all peer-to-peer systems 2 Rizal Mohd Nor, Mikhail Nesterenko, and Christian Scheideler. Corona: A stabilizing deterministic message-passing skip list. In 13 th. International Symposium on Stabilization, Safety and security of Distributed Systems (SSS) pages 356-370, October 2011 [1]

3. SSS 2013 Outline  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 3

4. SSS 2013 Linearization Problem  Strict (SL) and eventual (EL) linearization variants  SL requires each process to output its neighbors exactly once and allows only correct output  EL’s computation contains a suffix where the output of each process is correct  Existing (EID) and non-existing (NID) identifiers within the linearization problem  EID prohibits the existence of non-existing identifiers  NID allows non-existing identifiers 4

5. SSS 2013 Outline  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 5

6. SSS 2013 Solution Oracles  weak connectivity oracle (WC) has a single action that:  selects a pair of processes p and q such that they are disconnected in the channel connectivity graph CC, then  connecting graph by adding q to the incoming channel of p creating a link (p,q)  participant detector (PD) oracle removes a non-existent identifier stored in p.  neighbor output oracle (NO) just output identifiers stored in left and right variables of p  consequent process detector (CD)  outputs the stored identifier only if it is consequent with p  the guard of CD mentions all the identifiers of the system 6

7. SSS 2013 Outline  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 7

8. SSS 2013 Necessary Conditions: Weak Connectivity Oracle 8 Theorem 1. Every solution to the linearization problem requires a weak connectivity oracle. Theorem 1. Every solution to the linearization problem requires a weak connectivity oracle.  For a linearization algorithm A, p1 has to eventually output p2 p1p2p3p4p5 p1 will eventually add p2, if the computation contains the actions of a weak connectivity oracle. Without WC, it will remain disconnected.

9. SSS 2013 Subset Splittable  subset splittable  prevents a subset of processes from learning the state of the rest of the system  denoted as SS (subset spilttable) and NSS (non-subset spilttable)  neighborhood output oracle (NO) is subset spilttable (SS)  subset splittable since its guards only mention local variables  participant detector oracle (PD) is subset spilttable (SS) 9 abnidde Consider 2 sets of non-intersecting processes, S1 and S2 S1 S2

10. SSS 2013 Subset Splittable  weak connectivity oracle (WC) is subset spilttable (SS) 10 abcde Consider 2 sets of non-intersecting processes, S1 and S2, where c and d are disconnected S1 S2

11. SSS 2013 Non Subset Splittable  consequent process detector (CD) is not subset spilttable (NSS) 11 Consider 2 sets of non-intersecting processes, S1 and S2 are totally ordered acbde S1 S2 If a process has a consequent process, it is enabled. If b is added to set S1, c would be enabled. This disables CD enabled in the previous state.

12. SSS 2013 Necessary Conditions: Non-subset Splittable Oracle 12 Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle. Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle. Consider 2 sets of non-intersecting processes, S1 and S2 abcde S1 S2

13. SSS 2013 Necessary Conditions: Non-subset Splittable Oracle 13 Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle. Theorem 2. A solution to the strict linearization problem requires a non-subset splittable oracle. abcde Since strict linearization requires each process to output only once, in this case, b or c cannot output without knowing the state of the rest of the system

14. SSS 2013 Necessary Conditions: NID requires PD oracles 14 Theorem 3. A proper solution to the linearization problem that allows non-existing identifiers requires a participant detector oracle. Theorem 3. A proper solution to the linearization problem that allows non-existing identifiers requires a participant detector oracle. p1p2p3p4p5 Without PD, a process p1 may be connected to a non- existing identifier and result in a graph to be disconnected from the higher-id processes forever  PD is required to remove the right id of process p1 and left id of process p2.

15. SSS 2013 Outline  oracles  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 15

16. SSS 2013 Necessary Oracles The necessary oracles to solve the 4 variants of the linearization problem. 16 Eventual Linearization (EL) Strict Linearization (SL) Existing IDs (EID) WCWC+NSS Non-existing IDs (NID) WC+PDWC+PD+NSS Eventual Linearization (EL) Strict Linearization (SL) Existing IDs (EID) L+WC+NOL+WC+CD Non-existing IDs (NID) L+WC+NO+PDL+WC+CD+PD Linearization algorithm L with a combination of oracles to solve the linearization problem

17. SSS 2013 Outline  oracles  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 17

18. SSS 2013 EL+EID Execution Example using L+WC Starting from an arbitrary state where processes are weakly connected except process a abcde 18

19. SSS 2013 EL+EID Execution Example using L+WC abcde 19

20. SSS 2013 EL+EID Execution Example using L+WC abcde a 20 Node a WC oracle is enabled and node a sends its ID to node e

21. SSS 2013 EL+EID Execution Example using L+WC  starting from this state, processes are weakly connected and WC is never enabled  the only actions enabled are the actions of the linearization algorithm L  actions  receive ID from right → set closer right neighbor or forward to right neighbor  receive ID from left → set closer left neighbor or forward to left neighbor  true (timeout) → send ID to right and left neighbor abcde a 21 Node e receive a's ID, forward to c

22. SSS 2013 Node c receive a's ID, forward to b EL+EID Execution Example using L+WC  the only actions enabled are the actions of the linearization algorithm L  actions  receive ID from right → set closer right neighbor or forward to right neighbor  receive ID from left → set closer left neighbor or forward to left neighbor  true (timeout) → send ID to right and left neighbor abcde a 22

23. SSS 2013 Node b receive a's ID, set its new left neighbor EL+EID Execution Example using L+WC  the only actions enabled are the actions of the linearization algorithm L  actions  receive ID from right → set closer right neighbor or forward to right neighbor  receive ID from left → set closer left neighbor or forward to left neighbor  true (timeout) → send ID to right and left neighbor abcde 23

24. SSS 2013 Node b sends it's ID, to Node a EL+EID Execution Example using L+WC  the only actions enabled are the actions of the linearization algorithm L  actions  receive ID from right → set closer right neighbor or forward to right neighbor  receive ID from left → set closer left neighbor or forward to left neighbor  true (timeout) → send ID to right and left neighbor abcde b 24

25. SSS 2013 Node a receive b's ID, set its new right neighbor EL+EID Execution Example using L+WC  the only actions enabled are the actions of the linearization algorithm L  actions  receive ID from right → set closer right neighbor or forward to right neighbor  receive ID from left → set closer left neighbor or forward to left neighbor  true (timeout) → send ID to right and left neighbor abcde 25

26. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions c 1. abcde 26

27. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions 1. 2. abcde c abcde d 27

28. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions 1. 2. 3. abcde c abcde d abcde d 28

29. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions 1. 2. 3. 4. abcde c abcde d abcde d abcd e 29

30. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions e 1. 2. 3. 4. 5. abcde c abcde d abcde d abcd e abcd e 30

31. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions e 1. 2. 3. 4. 5. abcde c abcde d abcde d abcd e abcd e 31 6. e abcd e

32. SSS 2013 EL+EID Execution Example using L+WC processes take independent actions e 1. 2. 3. 4. 5. abcde c abcde d abcde d abcd e abcd e 32 6. e abcd e 7. e abcd e

33. SSS 2013 Outline  oracles  linearization problem  solution oracles  necessary conditions  linearization solutions  execution example  oracle implementation 33

34. SSS 2013 Oracle Implementation Example: WC  WC repairs the network disconnections, an encapsulation of bootstrap service commonly found in peer-to-peer systems  Example:  One bootstrap process b is always present in the system  the responsibility of this process is to maintain the greatest and smallest identifier of the system  if process p is disconnected, it does not have a left or right neighbor, it assumes that its own identifier is the greatest or, respectively, smallest  process p then sends its identifier to b  process b then either confirms this assumption or sends p, its current smallest or greatest identifier  weak connectivity is restored 34 b q p A disconnected peer will contact the bootstrap service to get smallest or greatest identifier q

35. SSS 2013 Oracle Implementation Example: PD  PD encapsulates the limits between relative process speeds and maximum message propagation delay and can be implemented using a heartbeat protocol  Example:  if process p contains an identifier q, p sends q a heartbeat message requesting a reply  if p does not receive this reply after the time above the maximum network delay, p considers q non-existent and discards it 35 p q heartbeat No reply from q Discard left link

36. SSS 2013 Future Research  Narrowing the gap between necessary and sufficient  Strict linearization solution relies on CD, which is a specific kind of the necessary non-subset splittable detector  Narrowing the gap between necessary and sufficient conditions for the solution to the strict linearizability problem remains to be addressed in future research.  Thank you. Any Questions. 36