1. PIC: Practical Internet Coordinates for Distance Estimation Manuel Costa joint work with Miguel Castro, Ant Rowstron, Peter Key Microsoft Research Cambridge

2. Why estimate distances?

3. Distance estimation can be used to optimize large scale distributed systems: –Server selection –Locality aware peer-to-peer overlay networks –Application level multicast Problems with on-demand measurement: –Slow –High overhead

4. PIC Maps the Internet into a geometric space Allows very low cost distance estimation Fully decentralized Tolerates malicious nodes

5. Outline Estimating distances with coordinates Securing the coordinate computation process Application to peer-to-peer overlays Conclusion

6. Internet as a geometric space Map each node to a position in the geometric space Compute distances based on coordinates Any node can compute the distance between any other two nodes Proposed by GNP (Global Network Positioning) y x (x 2,y 2 ) (x 3,y 3 ) (x 1,y 1 )

7. GNP – computing coordinates Measure distance to fixed landmarks Assign coordinates by solving a multi-dimensional global minimization problem There is no exact solution: –Internet is not euclidean –Measurements have errors y x (x 1,y 1 ) (x 2,y 2 ) (x 3,y 3 ) (x 4,y 4 ) d1d1 d2d2 d3d3

8. PIC – computing coordinates Any node in the system can act as a landmark Strategies for choosing landmarks include: –Random nodes –Close nodes –Hybrid y x (x 1,y 1 ) (x 2,y 2 ) (x 3,y 3 ) (x 4,y 4 ) (x 5,y 5 ) d1d1 d2d2 d3d3

9. PIC – any node can act as landmark

10. PIC – advantages Self-organizing - no provisioning of servers needed Scalable - load distributed among all the peers Resilient - avoids centralized points of failure

11. Experimental evaluation 40 000 node network on 3 topologies: Georgia Tech, Mercator, Corpnet Compare predicted distance to real distance for 100 000 node pairs Euclidean space with 8 dimensions, 16 landmarks

12. Accuracy: Georgia Tech

13. Accuracy over short distances

14. Accuracy: CorpNet

15. Accuracy: Mercator

16. PIC – security Problem: Malicious/compromised nodes can provide incorrect coordinates or fake distances Solution –Incorrect coordinates and distances are likely to violate triangle inequality –Remove landmarks that violate triangle inequality

17. PIC – security Remove landmarks with highest sum of deviations from these bounds When testing landmark i, check:

18. Security evaluation Fraction f of colluding attackers –Know everything When a node joins, attackers collude to provide a set of fake coordinates and distances that maximize the distance to the correct position This is a very powerful attack

19. Accuracy under attack

20. Application to peer-to-peer overlays Structured overlays: –Nodes have nodeIds –Message sent to a key is delivered to node with closest nodeId

21. Structured overlays: Mapping keys to nodes large id space (128-bit integers) nodeIds picked randomly from space keys picked randomly from space key is managed by its root node: live node with id closest to the key root node for key id space nodeId key

22. Pastry: Node routing state 0*1*2*3* 20*21*22*23* 200*201*202*203* 2030*2031*2032*2033* 203231 topology aware routing table nodeIds and keys in some base 2 b (e.g., 4) prefix constraints on nodeIds for each slot pick closest node satisfying slot constraints leaf set nodeId

23. Pastry: routing prefix matching: each hop resolves an extra key digit 323310 323211 322021 313221 103231 nodeId key route(m,323310)

24. Proximity neighbour selection Select close nodes for use in routing Important to achieve low delay routes PIC can replace network distance probes

25. Pastry: prefix-based routing Prefix matching: each hop resolves an extra key digit Proximity neighbour selection: use closest known node that matches an extra digit 323211 322021 313221 103231 route(m,323310) 323310

26. Proximity test variants Full probing –RTT measured by taking the minimum of three probes PIC –RTT estimated with coordinates Filtered probing –Use coordinates to filter bad candidates, always probe before replacing a neighbour

27. Trace-driven evaluation Dynamic node arrival and failure generated from UW Gnutella study –60 hour trace –Average session time 2.3 hours –number of active nodes varies from 1300- >2700 Georgia Tech topology

28. Distance probes

29. Relative delay penalty

30. Related Work GNP: maps Internet into geometric space using centralized landmarks Lighthouses: uses decentralized random landmarks Mithos: uses closest nodes as landmarks Virtual landmarks: partitions nodes into sets, maps coordinates between sets Vivaldi: computes coordinates continuously by passively monitoring RPC delays

31. Conclusion PIC enables practical distance estimation in large distributed systems –Accurate –Self-organizing –Scalable –Secure Future Work –Deployment and evaluation on the Internet –Different distance metrics (e.g. bandwidth)

32. Questions ?