1. Pastry Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK Some slides are borrowed from the original presentation by the authors

2. Outline Background Pastry Pastry proximity routing PAST

3. Background Peer-to-peer systems distribution decentralized control self-organization symmetry (communication, node roles)

4. Common issues Organize, maintain overlay network Resource allocation/load balancing Resource location Network proximity routing Pastry provides a generic p2p substrate

5. Architecture TCP/IP Pastry Network storage Event notification Internet P2p substrate (self-organizing overlay network) P2p application layer ?

6. Structured p2p overlays The primitive route(M, X) routes message M to the live node with node Id closest to key X Node ids and keys are from a large, sparse id space

7. Distributed Hash Tables (DHT) k6,v6 k1,v1 k5,v5 k2,v2 k4,v4 k3,v3 nodes Operations: insert(k,v) lookup(k) P2P overlay network p2p overlay maps keys to nodes completely decentralized and self-organizing robust, scalable

8. Outline Background Pastry Pastry proximity routing PAST SCRIBE Conclusions

9. Pastry Generic p2p substrate Self-organizing overlay network Lookup/insert object in < log 16 N routing steps (expected) O(log N) state per-node Network proximity routing

10. Pastry: Object distribution objId Consistent hashing [Karger et al. ‘97] 128 bit circular id space nodeIds (uniform random) objIds (uniform random) Invariant: node with numerically closest nodeId maintains object nodeIds O 2 128 -1

11. Pastry: Object insertion/lookup X Route(X) Msg with key X is routed to live node with nodeId closest to X Problem: complete routing table not feasible O 2 128 -1

12. Pastry: Routing Tradeoff O(log N) routing table size O(log N) message forwarding steps

13. Routing table of # 65a1fcx log 16 N rows Row 0 Row 1 Row 2 Row 3

14. Pastry: Routing Properties log 16 N steps O(log N) state d46a1c Route(d46a1c) d462ba d4213f d13da3 65a1fc d467c4 d471f1

15. Pastry: Leaf sets Each node maintains IP addresses of the nodes with the L/2 numerically closest larger and smaller nodeIds, respectively. routing efficiency/robustness fault detection (keep-alive) application-specific local coordination

16. Pastry: Routing procedure if (destination is within range of our leaf set) forward to numerically closest member else let l = length of shared prefix let d = value of l-th digit in D’s address if ( R l d exists) (R l d = entry at column d row l) forward to R l d else forward to a known node that (a) shares at least as long a prefix (b) is numerically closer than this node

17. Pastry: Performance Integrity of overlay/ message delivery: guaranteed unless L/2 simultaneous failures of nodes with adjacent nodeIds Number of routing hops: No failures: < log 16 N expected, 128/b + 1 max During failure recovery: –O(N) worst case, average case much better

18. Self-organization How are the routing tables and leaf sets initialized and maintained? Node addition Node departure (failure)

19. Pastry: Node addition d46a1c Route(d46a1c) d462ba d4213f d13da3 65a1fc d467c4 d471f1 New node: d46a1c

20. Node departure (failure) Leaf set members exchange heartbeat Leaf set repair (eager): request set from farthest live node in set Routing table repair (lazy): get table from peers in the same row, then higher rows

21. Pastry: Experimental results Prototype implemented in Java emulated network deployed currently at ~25 sites worldwide

22. Pastry: Average # of hops L=16, 100k random queries

23. Outline Background Pastry Pastry proximity routing

24. Pastry: Proximity routing Proximity metric = time delay estimated by ping A node can probe distance to any other node Each routing table entry uses a node close to the local node (in the proximity space), among all nodes with the appropriate node Id prefix.

25. Pastry: Routes in proximity space d46a1c Route(d46a1c) d462ba d4213f d13da3 65a1fc d467c4 d471f1 NodeId space d467c4 65a1fc d13da3 d4213f d462ba Proximity space

26. Pastry: Distance traveled L=16, 100k random queries, Euclidean proximity space

27. Pastry: Locality properties Expected distance traveled by a message in the proximity space is within a small constant of the minimum Among k nodes with node Ids closest to the key, message likely to reach the node closest to the source node first

28. d467c4 65a1fc d13da3 d4213f d462ba Proximity space Pastry: Node addition New node: d46a1c d46a1c Route(d46a1c) d462ba d4213f d13da3 65a1fc d467c4 d471f1 NodeId space

29. Pastry delay vs IP delay GATech top.,.5M hosts, 60K nodes, 20K random messages

30. Pastry: API route(M, X): route message M to node with nodeId numerically closest to X deliver(M): deliver message M to application forwarding(M, X): message M is being forwarded towards key X newLeaf(L): report change in leaf set L to application

31. Pastry: Security Secure nodeId assignment Secure node join protocols Randomized routing Byzantine fault-tolerant leaf set membership protocol

32. Pastry: Summary Generic p2p overlay network Scalable, fault resilient, self-organizing, secure O(log N) routing steps (expected) O(log N) routing table size Network proximity routing

33. PAST: File Retrieval fileId file located in log 16 N steps (expected) usually locates replica nearest client C Lookup k replicas C