1. 1 2/4/2008 Network Applications: P2P Applications

2. 2 Recap: Objectives of P2P r Bypass DNS to locate resources! m The lookup problem r Share the storage and bandwidth of individual users o The bandwidth sharing problem Internet

3. Recap: The Lookup Problem Internet N1N1 N2N2 N3N3 N6N6 N5N5 N4N4 Publisher Key=“title” Value=MP3 data… Client Lookup(“title”) ? find where a particular document is stored

4. 4 Outline  Recap  P2P  the lookup problem  Napster (central query server)  Gnutella (decentralized, flooding)  Freenet

5. 5 Recap: Freenet Query r Search by routing r API: file = query(id); r Upon receiving a query for file id m check whether the queried file is stored locally m check TTL to limit the search scope each query is associated a TTL that is decremented each time the query message is forwarded when TTL=1, the query is forwarded with a probability TTL can be initiated to a random value within some bounds to obscure distance to originator m look for the “closest” id in the table with an unvisited next_hope node if found one, forward the query to the corresponding next_hop otherwise, backtrack –ends up performing a Depth First Search (DFS)-like traversal –search direction ordered by closeness to target r When file is returned it is cached along the reverse path (any advantage?) id next_hop file … … …

6. 6 Insert r API: insert(id, file); r Two steps m first attempt a “search” for the file to be inserted m if found, report collision m if not found, insert the file by sending it along the query path thus inserted files are placed on nodes already possessing files with similar keys a node probabilistically replaces the originator with itself (why?)

7. 7 Insert Example r Assume query returned failure along the shown path (backtrack slightly complicate things); insert f10 4 n1 f4 12 n2 f12 5 n3 9 n3 f9 3 n1 f3 14 n4 f14 5 n3 14 n5 f14 13 n2 f13 3 n6 n1 n2 n3 n4 4 n1 f4 11 n5 f11 8 n6 n5 insert(10, f10)

8. 8 Insert Example 10 n1 f10 4 n1 f4 12 n2 14 n5 f14 13 n2 f13 3 n6 n1 n4 4 n1 f4 11 n5 f11 8 n6 n5 insert(10, f10) 9 n3 f9 n2 orig=n1 3 n1 f3 14 n4 f14 5 n3 n3

9. 9 Insert Example 10 n1 f10 4 n1 f4 12 n2 10 n1 f10 9 n3 f9 14 n5 f14 13 n2 f13 3 n6 n1 n2 n4 4 n1 f4 11 n5 f11 8 n6 n5 insert(10, f10) 3 n1 f3 14 n4 f14 5 n3 n3

10. 10 Insert Example r n2 replaces the originator (n1) with itself 10 n1 f10 4 n1 f4 12 n2 10 n1 f10 9 n3 f9 14 n5 f14 13 n2 f13 3 n6 n1 n2 n4 4 n1 f4 11 n5 f11 8 n6 n5 insert(10, f10) orig=n2 3 n1 f3 14 n4 f14 5 n3 n3

11. 11 Insert Example 10 n1 f10 4 n1 f4 12 n2 10 n1 f10 9 n3 f9 10 n2 10 3 n1 f3 14 n4 14 n5 f14 13 n2 f13 3 n6 n1 n2 n3 n4 4 n1 f4 11 n5 f11 8 n6 n5 insert(10, f10) orig=n2

12. 12 Insert Example 10 n1 f10 4 n1 f4 12 n2 10 n1 f10 9 n3 f9 10 n2 10 3 n1 f3 14 n4 10 n2 f10 14 n5 f14 13 n2 n1 n2 n3 n4 10 n4 f10 4 n1 f4 11 n5 n5 Insert(10, f10)

13. 13 Freenet Analysis r Authors claim the following effects: m nodes eventually specialize in locating similar keys if a node is listed in a routing table, it will get queries for related keys thus will gain “experience” answering those queries m popular data will be transparently replicated and will exist closer to requestors m as nodes process queries, connectivity increases nodes will discover other nodes in the network r Caveat: lexigraphic closeness of file names/keys may not imply content similarity

14. 14 Understanding Freenet Self- Organization: Freenet Graph r We create a Freenet reference graph m creating a vertex for each Freenet node m adding a directed link from A to B if A refers to an item stored at B id next_hop file … …

15. 15 Experiment: Freenet Graph: Init - Assume a network of 1000 nodes, with node id 0 to 999 - Each node can store 50 data items, and 200 references - Assume initially each node i has i, and knows the storage of i – 2, -1, i + 1, i + 2 (all mod 1000) - thus a regular, locally-clustered graph with avg path length ~ 1000 / 8 = 125 i-2i-1 i i+1 i+2 id next_hop file … …

16. 16 Experiment: Evolution of Freenet Graph r At each step m pick a node randomly m flip a coin to determine search or insert if search, randomly pick a key in the network if insert, pick a random key Evolution of path length and clustering; Clustering is defined as percentage of local links

17. 17 Freenet Evolves to Small-World Network r With usage, the regular, highly localized Freenet network evolved into one irregular graph r High percentage of highly connected nodes provide shortcuts/bridges m make the world a “small world” m most queries only traverse a small number of hops to find the file

18. 18 Small-World r First discovered by Milgrom m in 1967, Milgram mailed 160 letters to a set of randomly chosen people in Omaha, Nebraska m goal: pass the letters to a given person in Boston each person can only pass the letter to an intermediary known on a first-name basis pick the person who may make the best progress m result: 42 letters made it through ! m median intermediaries was 5.5---thus six degree of separation m a potential explanation: highly connected people with non- local links in mostly locally connected communities improve search performance !

19. 19 Kleinberg’s Result on Distributed Search r Question: how many long distance links to maintain so that distributed (greedy) search is effective? r Assume that the probability of a long link is some (  ) inverse-power of the number of lattice steps r Kleinberg’s Law: Distributed algorithm exists only when probability is proportional to (lattice steps) -d, where d is the dimension of the space

20. 20 Distributed Search r In other words, if double distance, increase number of neighbors by a constant -> see Chord probability is proportional to (lattice steps) -d

21. 21 Small World Saul Steinberg; View of World from 9 th Ave

22. 22 Freenet: Properties r Query using intelligent routing m decentralized architecture  robust m avoid flooding  low overhead m DFS search guided by closeness to target r Integration of query and caching makes it m adaptive to usage patterns: reorganize network reference structure m free speech: attempts to discover/supplant existing files will just spread the files ! r Provide publisher anonymity, security m each node probabilistically replaces originator with itself

23. 23 Freenet: Issues r Does not always guarantee that a file is found, even if the file is in the network r Good average-case performance, but a potentially long search path in a large network m approaching small-world…

24. 24 Summary r All of the previous p2p systems are called unstructured p2p systems r Advantages of unstructured p2p m algorithms tend to be simple m can optimize for properties such as locality r Disadvantages m hard to make performance guarantee m failure even when files exist

25. Distributed Hash Tables (DHT): History r In 2000-2001, academic researchers jumped on to the P2P bandwagon r Motivation: m frustrated by popularity of all these “half- baked” P2P apps. We can do better! (so they said) m guaranteed lookup success for data in system m provable bounds on search time m provable scalability to millions of node r Hot topic in networking ever since

26. 26 DHT: Overview DHT Distributed application get (key) node …. put(key, data) r Abstraction: a distributed “hash-table” (DHT) data structure m put(key, value) and get(key)  value m DHT imposes no structure/meaning on keys m one can build complex data structures using DHT r Implementation: m nodes in system form an interconnection network: ring, zone, tree, hypercube, butterfly network,...

27. 27 DHT Applications r File sharing and backup [CFS, Ivy, OceanStore, PAST, Pastiche …] r Web cache and replica [Squirrel, Croquet Media Player] r Censor-resistant stores [Eternity] r DB query and indexing [PIER, Place Lab, VPN Index] r Event notification [Scribe] r Naming systems [ChordDNS, Twine, INS, HIP] r Communication primitives [I3, …] r Host mobility [DTN Tetherless Architecture]

28. 28 DHT: Basic Idea

29. 29 DHT: Basic Idea (2)

30. 30 DHT: Basic Idea (3)

31. 31 DHT: Basic Idea (4)

32. 32 DHT: Basic Idea (5) Two questions: - how are the nodes connected? - how the key space is partitioned?

33. 33 Outline  Admin. and review  P2P  the lookup problem  Napster (central query server; distributed data server)  Gnutella (decentralized, flooding)  Freenet (search by routing)  Chord

34. 34 Chord r Provides lookup service: m Lookup(key)  IP address m chord does not store the data; after lookup, a node queries the IP address to store or retrieve data r m bit identifier space for both keys and nodes m key identifier = SHA-1(key), where SHA-1() is a popular hash function, Key=“Matrix3”  ID=60 m node identifier = SHA-1(IP address) IP=“198.10.10.1”  ID=123

35. 35 Chord: Storage using a Ring r A key is stored at its successor: node with next higher ID N32 N10 N123 N90 Circular 7-bit ID Space K11, K20 K125, K5, K10 K60 K101 IP=“198.10.10.1” Key=“Matrix 3” N55

36. 36 How to Search: One Extreme r Every node knows of every other node r Routing tables are large O(N) r Lookups are fast O(1)

37. 37 How to Search: the Other Extreme r Every node knows its successor in the ring r Requires O(N) lookups

38. 38 Chord Solution: “finger tables” r Node K knows the node that is maintaining K + 2 i, where K is mapped id of current node m increase distance exponentially (small world?) N80 K+64 K+32 K+16 K+8 K+4 K+2 K+1

39. 39 Joining the Ring r use a contact node to obtain info r transfer keys from successor node to new node r updating fingers of existing nodes

40. DHT: Chord Join r Assume an identifier space [0..8] r Node n1 joins 0 1 2 3 4 5 6 7 i id+2 i succ 0 2 1 1 3 1 2 5 1 Succ. Table

41. DHT: Chord Join r Node n2 joins 0 1 2 3 4 5 6 7 i id+2 i succ 0 2 2 1 3 1 2 5 1 Succ. Table i id+2 i succ 0 3 1 1 4 1 2 6 1 Succ. Table

42. DHT: Chord Join r Nodes n0, n6 join 0 1 2 3 4 5 6 7 i id+2 i succ 0 2 2 1 3 6 2 5 6 Succ. Table i id+2 i succ 0 3 6 1 4 6 2 6 6 Succ. Table i id+2 i succ 0 1 1 1 2 2 2 4 0 Succ. Table i id+2 i succ 0 7 0 1 0 0 2 2 2 Succ. Table

43. DHT: Chord Join r Nodes: n1, n2, n0, n6 r Items: f7, f1 0 1 2 3 4 5 6 7 i id+2 i succ 0 2 2 1 3 6 2 5 6 Succ. Table i id+2 i succ 0 3 6 1 4 6 2 6 6 Succ. Table i id+2 i succ 0 1 1 1 2 2 2 4 0 Succ. Table 7 Items 1 i id+2 i succ 0 7 0 1 0 0 2 2 2 Succ. Table

44. 7 DHT: Chord Routing r Upon receiving a query for item id, a node: r checks whether stores the item locally r if not, forwards the query to the largest node in its successor table that does not exceed id 0 1 2 3 4 5 6 7 i id+2 i succ 0 2 2 1 3 6 2 5 6 Succ. Table i id+2 i succ 0 3 6 1 4 6 2 6 6 Succ. Table i id+2 i succ 0 1 1 1 2 2 2 4 0 Succ. Table Items 1 i id+2 i succ 0 7 0 1 0 0 2 2 2 Succ. Table query(7) Two questions: - how are the nodes connected? - How the key space is partitioned?

45. 45 Outline  Admin. and review  P2P  the lookup problem  Napster (central query server; distributed data server)  Gnutella (decentralized, flooding)  Freenet (search by routing)  Chord (search by routing on a virtual ring)  Content Addressable Network

46. 46 Content Addressable Network (CAN) r Key space is an (virtual) d-dimensional Cartesian space m associate to each item a unique coordinate in the space m partition the space amongst all of the nodes

47. 47 CAN Example: Two Dimensional Space r Space divided among nodes r Each node covers either a square or a rectangular area of ratios 1:2 or 2:1 1

48. 48 CAN Example: Two Dimensional Space r Space divided among nodes r Each node covers either a square or a rectangular area of ratios 1:2 or 2:1 1 2

49. 49 CAN Example: Two Dimensional Space r Space divided among nodes r Each node covers either a square or a rectangular area of ratios 1:2 or 2:1 1 2 3

50. 50 CAN Example: Two Dimensional Space r Space divided among nodes r Each node covers either a square or a rectangular area of ratios 1:2 or 2:1 1 2 3 4

51. 51 CAN Example: Two Dimensional Space r Space divided among nodes r Each node covers either a square or a rectangular area of ratios 1:2 or 2:1

52. 52 CAN Insert: Example (1) node I::insert(K,V) I

53. 53 CAN Insert: Example (2) node I::insert(K,V) (1) a = h x (K) b = h y (K) I x = a y = b Example: Key=“Matrix3” h(Key)=60

54. 54 CAN Insert: Example (3) node I::insert(K,V) (1) a = h x (K) b = h y (K) (2) route(K,V) -> (a,b) I x = a y = b

55. 55 CAN Insert: Routing r A node maintains state only for its immediate neighboring nodes r Forward to neighbor which is closest to the target point m a type of greedy, local routing scheme

56. 56 CAN Insert: Example (4) node I::insert(K,V) (1) a = h x (K) b = h y (K) (2) route(K,V) -> (a,b) (3) (K,V) is stored at (a,b) I x = a y = b

57. 57 CAN Retrieve: Example node J::retrieve(K) (1) a = h x (K) b = h y (K) (2) route “retrieve(K)” to (a,b) J x = a y = b

58. 58 CAN Insert: Join (1) 1) Discover some node “J” already in CAN J 2) pick a random point (p,q) in space new node

59. 59 CAN Insert: Join (2) 3) J routes to (p,q), discovers node N J new node N

60. 60 CAN Insert: Join (3) 4) split N’s zone in half… new node owns one half J new node N Inserting a new node affects only a single other node and its immediate neighbors

61. 61 Chord/CAN Summary r Each node “owns” some portion of the key-space m in CAN, it is a multi-dimensional “zone” m in Chord, it is the key-id-space between two nodes in 1-D ring r Files and nodes are assigned random locations in key-space m provides some load balancing probabilistically equal division of keys to nodes r Routing/search is local (distributed) and greedy m node X does not know of a path to a key Z m but if it appears that its node Y is the closest to Z among all of the nodes known to X m so route to Y

62. 62 There are other DHT algorithms r Tapestry (Zhao et al) –Keys interpreted as a sequence of digits –Incremental suffix routing »Source to target route is accomplished by correcting one digit at a time »For instance: (to route from 0312  1643) »0312  2173  3243  2643  1643 –Each node has a routing table r Skip Graphs (Aspnes and Shah)

63. 63 Skip List

64. 64 Summary: DHT r Underlying metric space. r Nodes embedded in metric space r Location determined by key r Hashing to balance load r Greedy routing r Typically m O(log n) space at each node m O(log n) routing time

65. 65 Outline  Recap  P2P  the lookup problem  Napster (central query server; distributed data server)  Gnutella (decentralized, flooding)  Freenet (search by routing)  Chord (search by routing on a virtual ring)  Content Addressable Network (virtual zones)  the scalability problem

66. The Scalability Problem  Maximum throughput R = min{C 0, (C 0 +  C i )/n}  The bound is theoretically approachable  Problems in practice  efficiency  incentive, … 66 server C0C0 client 1 client 2 client 3 client n C1C1 C2C2 C3C3 CnCn

67. 67 BitTorrent: Initialization webserver user HTTP GET MYFILE.torrent http://mytracker.com:6969/ S3F5YHG6FEB FG5467HGF367 F456JI9N5FF4E … MYFILE.torrent

68. Metadata File Structure r Contains information necessary to contact the tracker and describes the files in the torrent m announce URL of tracker m file name m file length m piece length (typically 256KB) m SHA-1 hashes of pieces for verification m also creation date, comment, creator, …

69. 69 File Organization Piece 256KB Block 16KB File 42 1 3 Incomplete Piece

70. 70 Tracker Protocol tracker webserver user “register” ID1 169.237.234.1:6881 ID2 190.50.34.6:5692 ID3 34.275.89.143:4545 … ID50 231.456.31.95:6882 list of peers Peer 40 Peer 2 Peer 1 …

71. Tracker Protocol r Communicates with clients via HTTP/HTTPS r Client GET request m info_hash: uniquely identifies the file m peer_id: chosen by and uniquely identifies the client m client IP and port m numwant: how many peers to return (defaults to 50) m stats: bytes uploaded, downloaded, left r Tracker GET response m interval: how often to contact the tracker m list of peers, containing peer id, IP and port m stats: complete, incomplete r Tracker-less mode; based on the Kademlia DHT

72. 72 “On the Wire” Protocol (Over TCP) Local PeerRemote Peer ID/Infohash Handshake BitField 1001 Questions to think about for next class: - How to efficiently utilize the peer bandwidth? - How to deal with the incentive issue?

73. Backup Slides 73

74. 74 Recall: Linda API r out(): m writes tuples to shared space m example: out("abc", 1.5, 12). m result: insert (“abc”, 1.5, 12) into space r read(): m retrieves tuple copy matching arg list (blocking) m example: read(“abc”, ? A, ? B) m result: finds (“abc”, 1.5, 12) and sets local variables A = 1.5, B = 12. Tuple (“abc”, 1.5, 12) is still resident in space. r in(): m retrieves and deletes matching tuple from space (blocking) m example: same as above except (“abc”, 1.5, 12) is deleted r Eval() m evaluates a tuple on the server

75. 75 Intentional Naming System (INS) r A domain name system based on the Linda model m names are intentional, based on attributes (Linda tuple) apps know WHAT they want, not WHERE m tuples are stored by a p2p system, where each node in the p2p system is called an INS resolver (INR) m a tuple expires if not refreshed (soft state)

76. 76 INS Intentional Naming [building = AKW [floor = 3]] [entity = printer] Intentional name Intentional Name Resolvers (INR) form a p2p network Lookup lw3.cs.yale.edu INR

77. 77 CAN Complexity r Guaranteed to find an item if in the network r Scalability m for a uniform (regularly) partitioned space with n nodes and d dimensions per node, number of neighbors is 2d routing path length is dn 1/d average routing path is (dn 1/d )/3 hops (due to Manhattan distance routing, expected hops in each dimension is dimension length * 1/3) m a fixed d can scale the network without increasing per-node state r Load balancing m hashing achieves some load balancing m overloaded node replicates popular entries at neighbors r Robustness m no single point of failure m can route around trouble