SCOPE: Scalable Consistency in Structured P2P Systems Xin Chen1, Shansi Ren, Haining Wang, and Xiaodong Zhang College of William and Mary 1. With AskJeeves, Piscataway, NJ Introduce WM, supported by AskJ

Overview: P2P Systems P2P traffic: P2P users: P2P data size: 50-70% traffic in consumer ISPs 95% upstream traffic (MediaMetrix) P2P users: 75% broadband users (Jupiter Media) 6 millions simultaneous users P2P data size: 10 Petabytes (10,000,000 GB) 11/10/2018 INFOCOM 2005

Consistency in P2P Applications Consistency: provide the most updated object to any peer. No Consistency File sharing Real-time media streaming Weak Consistency P2P-based Web caching DNS systems Partial Strong Consistency Wide-area file systems Font; consistency definition Strong Consistency Publish/subscribe systems Directory services, Online auctions Not well supported 11/10/2018 INFOCOM 2005

Problems of Existing Solutions Reliability: recoverable from node failures, e.g. eliminating single point failure. Scalability: maintaining increasingly large system size. Maintainability: low cost and low overhead. High Reliability Low Scalability High Maintenance Reliability SCOPE Highly scalable Highly reliable Low Cost Graph Path-record Low Reliability Low Scalability Low Maintenance Centralized Time-to-Live Low Reliability High Scalability Low Maintenance Decent Reliability Decent Scalability High Maintenance Scalability 11/10/2018 INFOCOM 2005

Our Objectives High Scalability High Reliability Easy management Low overhead as system scales to large size. High Reliability Recoverable for node failures. Easy management Low maintenance costs General solutions Deployable for all structured P2P systems 11/10/2018 INFOCOM 2005

Outline Background SCOPE Performance Conclusion How SCOPE record the replica locations? How SCOPE operate efficiently? Performance Conclusion font 11/10/2018 INFOCOM 2005

SCOPE: Scalable Consistency in Structured P2P Systems Design Target: High Scalability: Distributed consistency maintenance among all nodes Low maintenance overhead High Reliability: Easy to recover from frequent node failures Able to finish consistency operations with node failures Design Approach Partition the whole ID space into partitions Select a representative in every partition Construct a tree to record the replica locations Revise high level introduction of scope 11/10/2018 INFOCOM 2005

SCOPE: ID Space Partitioning 000 01 10 Partition 0: [000,011] + 00 11 111 001 010 110 10 Partition 1: [100,111] 1 + 00 01 11 Sigcomm 2004 DN SIGCOMM 2004 DNS 101 011 100 3-bit ID space: [000,111] 11/10/2018 INFOCOM 2005

Key Mapping 001 01 101 1 01 + + 000 Representative 00 10 11 111 + 10 11 001 111 Partition 0 010 110 Partition 1 1 + 00 01 10 11 101 011 Root/Representative 100 3-bit ID space: [000,111] 11/10/2018 INFOCOM 2005

Replica Partition Tree (RPT) 101 000 101 101 111 001 Y Y 001 Height O(logM) Y Y 010 Y N 110 011 101 011 Every ID is represented by a bit in RPT 100 101 11/10/2018 INFOCOM 2005

RPT Optimization: Leaf Node 101 000 101 101 111 001 Y Y 001 Y Y 010 N 110 Y 011 101 011 100 Partitioning is necessary only when #node > 1 101 11/10/2018 INFOCOM 2005

RPT Optimization: Intermediate Node 000 101 111 001 Y Y 010 110 Y 101 011 101 100 Partitioning is necessary only when #subpartition > 1 101 11/10/2018 INFOCOM 2005

New Operations -- Subscribe, Unsubscribe, and Update 3. Root Node Update the records for the lower level representatives 3 2. Intermediate Representative Update the records for its lower level representatives Inform the next upper level representative of subscriber 2 1. Subscriber Inform its immediate upper level representative Subscribe, unsubscribe, update; font 1 How to find the upper level representatives? 11/10/2018 INFOCOM 2005

Upper Level Representatives 000 1. Find the partition start address Predecessor: 111 Node address: 000 Partition Start: 000 2. Find the partition end address Partition Start: 000 Successor: 100 Partition End: 011 3. Try smaller partition if the end address larger than the successor 111 001 010 110 101 011 100 11/10/2018 INFOCOM 2005

Upper Level Representatives (cont.) 000 1. Find the partition start address Partition predecessor: 101 Partition start address: 100 Upper partition Start: 100 2. Find the partition end address Upper Partition Start: 100 Partition successor: 100 Upper partition End: 100 3. Try smaller partition if the end address larger than the successor 111 001 010 110 101 011 100 11/10/2018 INFOCOM 2005

Level Index Level Index Fast operations Easy updates which level partition should made to identify a node Each node maintain its level index Fast operations no contact with predecessor and successor Easy updates only O(1) nodes updates when a node joins/leaves 000 111 001 [1,0,0] 010 110 [1,0,3] [1,0,3] 101 011 100 11/10/2018 INFOCOM 2005

Outline Background SCOPE Performance Conclusion System Scalability Operation Effectiveness Maintenance Costs Conclusion 11/10/2018 INFOCOM 2005

Experimental environment Simulation Methods ID Space: 160-bit Hash Function: SHA-1 Partitions: 16 at each level Routing Tables: Pastry 40 levels, each with 15 entries 32 entries in each node’s leaf set Performance metrics System Scalability: load distribution, RPT height Operation Effectiveness: routing path of operations Maintenance Costs: node joining/leaving, recovery process 11/10/2018 INFOCOM 2005

System Scalability Most nodes have <3 records Very few nodes have >3 records Left: Records distribution in a 104-node network; Right: Average RPT height changes with number of nodes. 11/10/2018 INFOCOM 2005

Operation Effectiveness When the #subscriber = 1 When the #subscriber changes 200 (2%) Left: Operation path length comparison; Right: Average path length changes with # of subscribers. 11/10/2018 INFOCOM 2005

Maintenance Costs Less than 1 node’s level index changed 90% message reduction Left: Level index update costs; Right: Failure recovery costs. 11/10/2018 INFOCOM 2005

Conclusion SCOPE provides scalable consistent supports for structured P2P systems Scalable structures Effective operations Minimal maintenance overhead Compared with existing solutions Better load balance Better fault tolerance Better consistency support 11/10/2018 INFOCOM 2005