Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications

Abstract Fundamental problem: location of a particular node that stores a particular data item Chord solves this problem –Data location solved by associating a key with a data item and storing the data/key pair at the node to which the key maps –Adapts efficiently as nodes join and leave system –Experiments show that chord is scalable

Chord vs. Other Systems DNS Freenet & Ohaha Globe Plaxton et. al. (used in OceanStore) CAN

Introduction to Chord Protocol for lookup in a dynamic P2P system Given a key, it maps the key onto a particular node Uses consistent hashing to assign keys to nodes (allows for balanced load) –Distributed routing information (log n entries) –Message based updates to maintain routing information (log^2 n)

System Model - Problems Load balance: spreading the load over all of the nodes in a dynamic manner Decentralization: no node is more important than any other Scalability: cost of look up grows as log n Availability: tables are automatically updated to show new nodes, node with responsible for any key can always be found Flexible Naming: no structure on the keys (naming)

The Chord Protocol Consistent Hashing Scalable Key Location Node Joins Stabilization Failures and Replication

Consistent Hashing Assigns each node and key an m-bit identifier –Uses the SHA-1 as a base hash function –Node ID: hash the node’s IP address –Key ID: hashing the key Intent – nodes can enter and leave with minimal disruption

Consistent Hashing con’t Identifiers are ordered in a circle (modulo 2^m) Key K is assined to the first node whose identifier is equal to or follows the ID of K in the identifier space This particular node is the successor node In the circle, the successor of K would be the first node clockwise from K

Consistent Hashing con’t Theorem #1 – For any set of N nodes and K keys 1.Each node is responsible for at most (1 + E)K/N keys E = O(log N) 1.When a (N+1) node joins/leaves the network, responsibility for K/N keys changes hands

Scalable Key Location aka routing information Only needs to know its successor –Queries can be passed around the circle till it reaches the node that is maps to –Resolution scheme is inefficient…why?

Scalable Key Location Each node maintains routing information in a finger table –M = number of bits in the key/node Ids –Each finger table has at most M entries –The i th entry contains the identity of the first node that succeeds N by at least 2 i-1 This node is called the finger of node N –Note: The first finger of N is its immediate successor on the circle So…first finger = successor

Scalable Key Location Each node only stores information about a small number of nodes Knows more information about nodes that are closer Finger table does not contain enough information to determine the successor of an arbitrary key K Theorem #2: the number of nodes that must be contacted to find a successor in an N-node network is O(log N)

Node Joins This is a dynamic network –Each node’s successor must be maintained –Every key K, K’s successor is responsible for K Theorem #3: any node joining/leaving will use O(log 2 N) messages to re-establish routing and finger tables The finger table also contains a predecessor pointer…why?

Node Joins 3 tasks needed when a node joins the network –Initialize the predecessor and fingers of the node –Update the fingers and predecessors of existing nodes –Transfer state for keys that node is now responsible for

Node Joins

Stabilization Join algorithm aggressively maintains finger tables of all nodes Stabilization is run on every node periodically to check for new nodes Theorem #4: Once a node can successfully resolve a given query, it will always be able to do so in the future. Theorem #5: At some time after the last join all successor pointers will be correct. Theorem #6: If we take a stable network with nodes, and another set of up to nodes joins the network with no finger pointers (but with correct successor pointers), then lookups will still take time with high probability.

Failures/Replication Key step in recovery is maintaining correct successor pointers –Successor list of its R nearest successors on the ring –If node N notices that its successor has failed, it uses the first entry from its successor list –Theorem #7: If we use a successor list of length r=O(log N) in a network that is initially stable, and then every node fails with probability 1/2, then with high probability find successor returns the closest living successor to the query key. –Theorem #8: If we use a successor list of length r=O(log N) in a network that is initially stable, and then every node fails with probability 1/2, then the expected time to execute find successor in the failed network is O(log N).

Simulation/Experimental Results Path Length Simultaneous Node Failures Lookups during stabilization

Path Length Experiment: 2 k nodes, 100*2 k keys, k=3-14

Simultaneous Node Failures Ability to remain consistent after high percentage of node failure Experiment: 10 4 nodes and 10 6 keys –Wait for network to stabilize then measure fraction of keys that could not be looked up correctly

Lookups During Stabilization Lookup after failure but before stabilization failure reasons –Node responsible for key may have failed –Nodes finger tables and predecessor pointers may be inconsistent

Chord Applications Following are good applications for which Chord could be a good foundation –Cooperative Mirroring: balance the demand for a particular file on a server with files that aren’t popular at all –Time Shared Storage: store someone else’s data when you’re machine is up and they will store yours when you are down –Distributed Indexes: keyword search –Large-Scale Combinatorial Search: ie-code breaking

The End! Questions Comments Discussion