A Scalable Peer-to-peer Lookup Service for Internet Applications Chord A Scalable Peer-to-peer Lookup Service for Internet Applications Dimitris Mavrommatis Computer Science Department University of Crete May 10, 2016

What is Chord? In short: a peer-to-peer lookup service. Solves problem of locating a data item in a collection of distributed nodes, considering frequent node arrivals and departures. Uses a variant of consistent hashing. Supports just one operation: given a key, it maps the key onto a node.

Comparative Performance Memory Lookup Latency Messages for Lookup Napster O(1) O(N)Server O(1) Gnutella O(N) Chord O(log(N))

Chord Characteristics Simplicity, provable correctness, and provable performance Each Chord node needs routing information about only a few other nodes Resolves lookups via messages to other nodes (iteratively or recursively) Maintains routing information as nodes join and leave the system

Addressed Difficult Problems Load balance: distributed hash function, spreading keys evenly over nodes Decentralization: chord is fully distributed, no node more important than other, improves robustness Scalability: logarithmic growth of lookup costs with number of nodes in network, even very large systems are feasible Availability: chord automatically adjusts its internal tables to ensure that the node responsible for a key can always be found Flexible naming: no constraints on the structure of the keys – key-space is flat, flexibility in how to map names to Chord keys

Consistent Hashing Hash function assigns each node and key an m-bit identifier using a base hash function such as SHA-1 ID(node) = hash(IP, Port) ID(key) = hash(key) Hash is truncated to m bits; where m must be chosen based on total entries to avoid collisions (very unlikely). Called peer id or key id (number between 0 and 2 𝑚 −1). Can then map peers and keys to one of 2 𝑚 logical points on a circle. Properties of consistent hashing: Function balances load: all nodes receive roughly the same number of keys – good? When an Nth node joins (or leaves) the network, only an O(1/N) fraction of the keys are moved to a different location

Construction of the Chord ring identifier node X key 6 4 2 6 5 1 3 7 1 successor(1) = 1 identifier circle successor(6) = 0 6 2 successor(2) = 3 2

Node Joins and Departures 6 6 4 2 6 5 1 3 7 1 successor(6) = 7 successor(1) = 3 2 1

Acceleration of Lookups Lookups are accelerated by maintaining additional routing information Each node maintains a routing table with (at most) m entries (where N=2m) called the finger table ith entry in the table at node n contains the identity of the first node, s, that succeeds n by at least 2i-1 on the identifier circle (clarification on next slide) s = successor(n + 2i-1) (all arithmetic mod 2) s is called the ith finger of node n, denoted by n.finger(i).node

Finger Tables Finger table: finger[i] = successor (n + 2 ) i-1 N8 + 2 N14 N8 + 4 N14 N8 + 8 N21 N8 +16 N32 N8 +32 N42

Finger Tables 4 2 6 5 1 3 7 finger table start int. succ. keys 6 1 2 4 [1,2) [2,4) [4,0) 1 3 4 2 6 5 1 3 7 finger table start int. succ. keys 1 2 3 5 [2,3) [3,5) [5,1) finger table start int. succ. keys 2 4 5 7 [4,5) [5,7) [7,3)

Finger Tables - Characteristics Important characteristics of this scheme: Each node stores information about only a small number of nodes (m) Each nodes knows more about nodes closely following it than about nodes further away A finger table generally does not contain enough information to directly determine the successor of an arbitrary key k Repetitive queries to nodes that immediately precede the given key will lead to the key’s successor eventually

Scalable node localization Search in finger table for the nodes which most immediatly precedes id Invoke find_successor from that node => Number of messages O(log N)!

Node Joins – with Finger Tables keys start int. succ. 6 1 2 4 [1,2) [2,4) [4,0) 1 3 6 4 2 6 5 1 3 7 finger table start int. succ. keys 1 2 3 5 [2,3) [3,5) [5,1) 6 finger table start int. succ. keys 7 2 [7,0) [0,2) [2,6) 3 finger table keys start int. succ. 2 4 5 7 [4,5) [5,7) [7,3) 6 6

Node Departures – with Finger Tables keys start int. succ. 1 2 4 [1,2) [2,4) [4,0) 1 3 3 6 4 2 6 5 1 3 7 finger table keys start int. succ. 1 2 3 5 [2,3) [3,5) [5,1) 3 6 finger table keys start int. succ. 6 7 2 [7,0) [0,2) [2,6) 3 finger table keys start int. succ. 2 4 5 7 [4,5) [5,7) [7,3) 6

Concurrent Operations and Failures Basic “stabilization” protocol is used to keep nodes’ successor pointers up to date, which is sufficient to guarantee correctness of lookups Those successor pointers can then be used to verify the finger table entries Every node runs stabilize periodically to find newly joined nodes

Stabilization after Join ns n joins predecessor = nil n acquires ns as successor via some n’ n notifies ns being the new predecessor ns acquires n as its predecessor np runs stabilize np asks ns for its predecessor (now n) np acquires n as its successor np notifies n n will acquire np as its predecessor all predecessor and successor pointers are now correct fingers still need to be fixed, but old fingers will still work pred(ns) = n n nil succ(np) = ns pred(ns) = np succ(np) = n np

Failure Recovery Key step in failure recovery is maintaining correct successor pointers To help achieve this, each node maintains a successor-list of its r nearest successors on the ring If node n notices that its successor has failed, it replaces it with the first live entry in the list stabilize will correct finger table entries and successor-list entries pointing to failed node Performance is sensitive to the frequency of node joins and leaves versus the frequency at which the stabilization protocol is invoked

Experimental Results Latency grows slowly with the total number of nodes Path length for lookups is about ½ log2N Chord is robust in the face of multiple node failures

Simulation Results The fraction of look ups the fail as a function of the fraction of nodes that fail. The fraction of lookups that fail as a function of the rate(over time) at which nodes fail and join.

Simulation Results The mean and 1st and 99th percentiles of the number of keys stored per node in a 104 node network. The probability density function (PDF) of the number of keys per node. The total number of keys is 5 × 10

What to keep? Chord protocol solves searches in decentralized manner Each node maintains routing information of about O(log(𝑁)) other nodes. Updates to the routing information require 𝑂(𝑙𝑜𝑔 2 (𝑁)) messages. Simple implementation

Thanks for the attention! Questions?

References [1] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan, “Chord: A scalable peer-to-peer lookup service for internet applications,” SIGCOMM Comput. Commun. Used reading material from University of California, Berkeley and Max Planck institute