1. Md Tareq Adnan adnan@cse.yorku.ca

2. Centralized Approach : Server & Clients Slow content must traverse multiple backbones and long distances Unreliable delivery may be prevented by congestion or backbone peering problems Not scalable usage limited by bandwidth available at master site

3. Distributed Approach : Servers & Clients Put servers wherever there are clients Direct clients to “nearby” servers

4. Hashing (Mapping) : servers & clients E.g., h(x) = (((a x + b) mod P) mod |B|), where P is prime, P > |U| a,b chosen uniformly at random from Z P x is a serial number associated with object Universe U of all possible objects, set B of buckets. object: web objects (client IP address, data) bucket: web server Hash function h: U  B Assigns objects to buckets

5. f(d) = d + 1 mod 5 Difficulty changing number of buckets 5 7 10 11 27 29 36 38 40 43 4321043210 bucket object f(d) = d + 1 mod 4

6. Problems in Distributed Hashing Change in buckets(servers) leads to a complete rearrangement of all the objects into the buckets We need to find a way that can minimize this arrangement

7. Consistent Hashing Idea: Map both objects and buckets to a circle. object bucket Assign object to next bucket on circle in clockwise order. new bucket

8. Consistent Hashing Identifier (both key & node) space : m=3 node key 0 1 3 8 1 2 6 6 1 2 0 4 26 5 1 3 7 identifier circle Key = object = data node = bucket = server

9. Consistent Hashing :Simple Key Lookup 9 1 2 6 0 4 26 5 1 3 7 successor(1) = 1 successor(2) = 3 successor(6) = 0 Node 0 sends a query for key 5  It may require traversing all nodes to find the appropriate mapping  Steps=O(N)  Can we do better? node key 0 1 3 1 2 6

10. Finger Table– Key Lookup For.startInt.Succ. 10 Identifier Space = m =3 Finger table Each node maintains up to m entries i th entry of node n is associated with (n+2 i-1 ) mod 2 m 0 4 26 5 1 3 7 finger table keys 0+2 0 0+2 1 0+2 2 124124 [1,2)3 3 6 [2,4) [4,0) finger table keys For.startInt.Succ. 3+2 0 3+2 1 3+2 2 457457 [4,5)6 6 0 [5,7) [7,3) 1 2 finger table keys For.startInt.Succ. 6+2 0 6+2 1 6+2 2 702702 [7,0)0 0 3 [0,2) [2,6) 5

11. Key Lookup Node 7 looks for key 5 11 0 4 26 5 1 3 7 finger table keys startInt.Succ. 1 [1,2) 3 2 [2,4) 3 4 [4,0) 4 finger table keys startInt.Succ. 4[4,5) 5[5,7) 7[7,3) 1 2 finger table keys startInt.Succ. 0[0,1)0 1[1,3)0 3[3,7) 6 4 Successor 0 Predecessor 4 3 Successor 3 Predecessor 7 Successor Predecessor 0 finger table keys startInt.Succ. 5[5,6)7 6[6,0)7 0[0,4)0 Successor Predecessor 3 7 4 7 7 successor(5) = 7 4 Every iteration of the loop halfs the distance to the target id Steps = O(logN)

12. Algorithm – Key Lookup 12 Walk clockwise to find the node which precedes id and whose successor succeeds id Start with the m th finger of node n See if it comes between node n and the id, if not, check the m-1 th finger until we find one wich does. This is the closest node preceding id among all the fingers of n Find id’s successor by finding the immediate predecessor of the id

13. Node joins startInt.Succ. 1 [1,2) 3 2 [2,4) 3 4 [4,0) 7 13 0 4 26 5 1 3 7 finger table keys startInt.Succ. 4[4,5)7 5[5,7)7 7[7,3)7 1 2 finger table keys startInt.Succ. 0[0,1)0 1[1,3)0 3[3,7)3 6 4 Successor 0 Predecessor 3 5 Successor 3 Predecessor 7 Successor Predecessor 0 finger table keys startInt.Succ. 6[6,7) 7[7,1) 1[1,5) 7 7 3 Successor Predecessor 7 3 Initializing fingers and predecessor

14. startInt.Succ. 1 [1,2) 2 [2,4) 4 [4,0) 7 14 0 4 26 5 1 3 7 finger table keys 5 startInt.Succ. 4[4,5)7 5[5,7)7 7[7,3)7 5 1 2 finger table keys startInt.Succ. 0[0,1)0 1[1,3)0 3[3,7) 6 4 Successor 0 Predecessor 3 5 Successor 3 Predecessor 6 Successor 7 Predecessor 0 5 finger table keys startInt.Succ. 6[6,7) 7[7,1) 1[1,5) 7 7 3 Successor Predecessor 7 3 5 Updating fingers of existing nodes 3 3 3 Node joins

15. startInt.Succ. 1 [1,2) 3 2 [2,4) 3 4 [4,0) 7 15 0 4 26 5 1 3 7 finger table keys 6 startInt.Succ. 4[4,5)7 5[5,7)7 7[7,3)7 5 1 2 finger table keys startInt.Succ. 0[0,1)0 1[1,3)0 3[3,7)3 6 4 Successor 0 Predecessor 3 5 Successor 3 Predecessor 6 Successor 6 Predecessor 0 5 finger table keys startInt.Succ. 6[6,7) 7[7,1) 1[1,5) 7 7 3 Successor Predecessor 7 3 5 Transferring Keys Steps = No. of nodes needed to be updated × time for updating tables = O(logN) × O(logN) = O(log 2 N) Node joins

16. Algorithm – node joins 16

17. Summary Characteristics of Algorithm Load balance distributed hash table Decentralization fully distributed Scalability cost of lookup grows logarithmic Steps for lookupO(logN) ArrivalO(log 2 N) DepartureO(log 2 N)

18. References “Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web” by David Karger, Eric Lehman, Tom Leighton, Matthew Levine, Daniel Lewin, and Rina Panigrahy “Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications” by Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan “Web Caching with Consistent Hashing” by by David Karger, Alex Sherman, Andy Berkheimer, Bill Bogstad, Rizwan Dhanidina, Ken Iwamoto, Brian Kim, Luke Matkins, Yoav Yerushalmi. http://www.wikipedia.com 18

19. Thanks