SCOPE: Scalable Consistency in Structured P2P Systems

Slides:



Advertisements
Similar presentations
Dynamic Replica Placement for Scalable Content Delivery Yan Chen, Randy H. Katz, John D. Kubiatowicz {yanchen, randy, EECS Department.
Advertisements

Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT and Berkeley presented by Daniel Figueiredo Chord: A Scalable Peer-to-peer.
Pastry Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK Some slides are borrowed from the original presentation by the authors.
Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK
Scalable Content-Addressable Network Lintao Liu
Peer-to-Peer (P2P) Distributed Storage 1Dennis Kafura – CS5204 – Operating Systems.
Multicast in Wireless Mesh Network Xuan (William) Zhang Xun Shi.
CHORD – peer to peer lookup protocol Shankar Karthik Vaithianathan & Aravind Sivaraman University of Central Florida.
Pastry Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK Some slides are borrowed from the original presentation by the authors.
LightFlood: An Optimal Flooding Scheme for File Search in Unstructured P2P Systems Song Jiang, Lei Guo, and Xiaodong Zhang College of William and Mary.
Common approach 1. Define space: assign random ID (160-bit) to each node and key 2. Define a metric topology in this space,  that is, the space of keys.
Peer-to-Peer Networks as a Distribution and Publishing Model Jorn De Boever (june 14, 2007)
SCAN: A Dynamic, Scalable, and Efficient Content Distribution Network Yan Chen, Randy H. Katz, John D. Kubiatowicz {yanchen, randy,
Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications Stoica et al. Presented by Tam Chantem March 30, 2007.
Distributed Lookup Systems
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek and Hari alakrishnan.
presented by Hasan SÖZER1 Scalable P2P Search Daniel A. Menascé George Mason University.
SCALLOP A Scalable and Load-Balanced Peer- to-Peer Lookup Protocol for High- Performance Distributed System Jerry Chou, Tai-Yi Huang & Kuang-Li Huang Embedded.
Object Naming & Content based Object Search 2/3/2003.
Chord-over-Chord Overlay Sudhindra Rao Ph.D Qualifier Exam Department of ECECS.
Wide-area cooperative storage with CFS
Roger ZimmermannCOMPSAC 2004, September 30 Spatial Data Query Support in Peer-to-Peer Systems Roger Zimmermann, Wei-Shinn Ku, and Haojun Wang Computer.
1 The Google File System Reporter: You-Wei Zhang.
Chord & CFS Presenter: Gang ZhouNov. 11th, University of Virginia.
Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications Xiaozhou Li COS 461: Computer Networks (precept 04/06/12) Princeton University.
Network Computing Laboratory Scalable File Sharing System Using Distributed Hash Table Idea Proposal April 14, 2005 Presentation by Jaesun Han.
Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT and Berkeley presented by Daniel Figueiredo Chord: A Scalable Peer-to-peer.
Benjamin AraiUniversity of California, Riverside Reliable Hierarchical Data Storage in Sensor Networks Song Lin – Benjamin.
Strong Cache Consistency Support for Domain Name System Xin Chen, Haining Wang, Sansi Ren and Xiaodong Zhang College of William and Mary, Williamsburg,
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented.
SIGCOMM 2001 Lecture slides by Dr. Yingwu Zhu Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications.
Scalable Content- Addressable Networks Prepared by Kuhan Paramsothy March 5, 2007.
Paper Survey of DHT Distributed Hash Table. Usages Directory service  Very little amount of information, such as URI, metadata, … Storage  Data, such.
Peer to Peer A Survey and comparison of peer-to-peer overlay network schemes And so on… Chulhyun Park
PROP: A Scalable and Reliable P2P Assisted Proxy Streaming System Computer Science Department College of William and Mary Lei Guo, Songqing Chen, and Xiaodong.
Lecture 12 Distributed Hash Tables CPE 401/601 Computer Network Systems slides are modified from Jennifer Rexford.
1 Distributed Hash Table CS780-3 Lecture Notes In courtesy of Heng Yin.
LightFlood: An Efficient Flooding Scheme for File Search in Unstructured P2P Systems Song Jiang, Lei Guo, and Xiaodong Zhang College of William and Mary.
Chord Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber Google,
BATON A Balanced Tree Structure for Peer-to-Peer Networks H. V. Jagadish, Beng Chin Ooi, Quang Hieu Vu.
Algorithms and Techniques in Structured Scalable Peer-to-Peer Networks
LOOKING UP DATA IN P2P SYSTEMS Hari Balakrishnan M. Frans Kaashoek David Karger Robert Morris Ion Stoica MIT LCS.
A Bandwidth Scheduling Algorithm Based on Minimum Interference Traffic in Mesh Mode Xu-Yajing, Li-ZhiTao, Zhong-XiuFang and Xu-HuiMin International Conference.
NCLAB 1 Supporting complex queries in a distributed manner without using DHT NodeWiz: Peer-to-Peer Resource Discovery for Grids Sujoy Basu, Sujata Banerjee,
Large Scale Sharing Marco F. Duarte COMP 520: Distributed Systems September 19, 2004.
Malugo – a scalable peer-to-peer storage system..
Peer-to-Peer (P2P) File Systems. P2P File Systems CS 5204 – Fall, Peer-to-Peer Systems Definition: “Peer-to-peer systems can be characterized as.
CS694 - DHT1 Distributed Hash Table Systems Hui Zhang University of Southern California.
CS Spring 2010 CS 414 – Multimedia Systems Design Lecture 24 – Introduction to Peer-to-Peer (P2P) Systems Klara Nahrstedt (presented by Long Vu)
Distributed Web Systems Peer-to-Peer Systems Lecturer Department University.
Peer-to-Peer Information Systems Week 12: Naming
Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M
Distributed Hash Tables
Controlling the Cost of Reliability in Peer-to-Peer Overlays
Improving and Generalizing Chord
S-Chord: Using Symmetry to Improve Lookup Efficiency in Chord
Plethora: Infrastructure and System Design
Early Measurements of a Cluster-based Architecture for P2P Systems
DHT Routing Geometries and Chord
5.2 FLAT NAMING.
Distributed P2P File System
Peer-to-Peer (P2P) File Systems
Peer-to-Peer Storage Systems
Dynamic Replica Placement for Scalable Content Delivery
Combinatorial Optimization of Multicast Key Management
A Scalable Content Addressable Network
MIT LCS Proceedings of the 2001 ACM SIGCOMM Conference
Consistent Hashing and Distributed Hash Table
Peer-to-Peer Information Systems Week 12: Naming
Presentation transcript:

SCOPE: Scalable Consistency in Structured P2P Systems Xin Chen1, Shansi Ren, Haining Wang, and Xiaodong Zhang College of William and Mary 1. With AskJeeves, Piscataway, NJ Introduce WM, supported by AskJ

Overview: P2P Systems P2P traffic: P2P users: P2P data size: 50-70% traffic in consumer ISPs 95% upstream traffic (MediaMetrix) P2P users: 75% broadband users (Jupiter Media) 6 millions simultaneous users P2P data size: 10 Petabytes (10,000,000 GB) 11/10/2018 INFOCOM 2005

Consistency in P2P Applications Consistency: provide the most updated object to any peer. No Consistency File sharing Real-time media streaming Weak Consistency P2P-based Web caching DNS systems Partial Strong Consistency Wide-area file systems Font; consistency definition Strong Consistency Publish/subscribe systems Directory services, Online auctions Not well supported 11/10/2018 INFOCOM 2005

Problems of Existing Solutions Reliability: recoverable from node failures, e.g. eliminating single point failure. Scalability: maintaining increasingly large system size. Maintainability: low cost and low overhead. High Reliability Low Scalability High Maintenance Reliability SCOPE Highly scalable Highly reliable Low Cost Graph Path-record Low Reliability Low Scalability Low Maintenance Centralized Time-to-Live Low Reliability High Scalability Low Maintenance Decent Reliability Decent Scalability High Maintenance Scalability 11/10/2018 INFOCOM 2005

Our Objectives High Scalability High Reliability Easy management Low overhead as system scales to large size. High Reliability Recoverable for node failures. Easy management Low maintenance costs General solutions Deployable for all structured P2P systems 11/10/2018 INFOCOM 2005

Outline Background SCOPE Performance Conclusion How SCOPE record the replica locations? How SCOPE operate efficiently? Performance Conclusion font 11/10/2018 INFOCOM 2005

SCOPE: Scalable Consistency in Structured P2P Systems Design Target: High Scalability: Distributed consistency maintenance among all nodes Low maintenance overhead High Reliability: Easy to recover from frequent node failures Able to finish consistency operations with node failures Design Approach Partition the whole ID space into partitions Select a representative in every partition Construct a tree to record the replica locations Revise high level introduction of scope 11/10/2018 INFOCOM 2005

SCOPE: ID Space Partitioning 000 01 10 Partition 0: [000,011] + 00 11 111 001 010 110 10 Partition 1: [100,111] 1 + 00 01 11 Sigcomm 2004 DN SIGCOMM 2004 DNS 101 011 100 3-bit ID space: [000,111] 11/10/2018 INFOCOM 2005

Key Mapping 001 01 101 1 01 + + 000 Representative 00 10 11 111 + 10 11 001 111 Partition 0 010 110 Partition 1 1 + 00 01 10 11 101 011 Root/Representative 100 3-bit ID space: [000,111] 11/10/2018 INFOCOM 2005

Replica Partition Tree (RPT) 101 000 101 101 111 001 Y Y 001 Height O(logM) Y Y 010 Y N 110 011 101 011 Every ID is represented by a bit in RPT 100 101 11/10/2018 INFOCOM 2005

RPT Optimization: Leaf Node 101 000 101 101 111 001 Y Y 001 Y Y 010 N 110 Y 011 101 011 100 Partitioning is necessary only when #node > 1 101 11/10/2018 INFOCOM 2005

RPT Optimization: Intermediate Node 000 101 111 001 Y Y 010 110 Y 101 011 101 100 Partitioning is necessary only when #subpartition > 1 101 11/10/2018 INFOCOM 2005

New Operations -- Subscribe, Unsubscribe, and Update 3. Root Node Update the records for the lower level representatives 3 2. Intermediate Representative Update the records for its lower level representatives Inform the next upper level representative of subscriber 2 1. Subscriber Inform its immediate upper level representative Subscribe, unsubscribe, update; font 1 How to find the upper level representatives? 11/10/2018 INFOCOM 2005

Upper Level Representatives 000 1. Find the partition start address Predecessor: 111 Node address: 000 Partition Start: 000 2. Find the partition end address Partition Start: 000 Successor: 100 Partition End: 011 3. Try smaller partition if the end address larger than the successor 111 001 010 110 101 011 100 11/10/2018 INFOCOM 2005

Upper Level Representatives (cont.) 000 1. Find the partition start address Partition predecessor: 101 Partition start address: 100 Upper partition Start: 100 2. Find the partition end address Upper Partition Start: 100 Partition successor: 100 Upper partition End: 100 3. Try smaller partition if the end address larger than the successor 111 001 010 110 101 011 100 11/10/2018 INFOCOM 2005

Level Index Level Index Fast operations Easy updates which level partition should made to identify a node Each node maintain its level index Fast operations no contact with predecessor and successor Easy updates only O(1) nodes updates when a node joins/leaves 000 111 001 [1,0,0] 010 110 [1,0,3] [1,0,3] 101 011 100 11/10/2018 INFOCOM 2005

Outline Background SCOPE Performance Conclusion System Scalability Operation Effectiveness Maintenance Costs Conclusion 11/10/2018 INFOCOM 2005

Experimental environment Simulation Methods ID Space: 160-bit Hash Function: SHA-1 Partitions: 16 at each level Routing Tables: Pastry 40 levels, each with 15 entries 32 entries in each node’s leaf set Performance metrics System Scalability: load distribution, RPT height Operation Effectiveness: routing path of operations Maintenance Costs: node joining/leaving, recovery process 11/10/2018 INFOCOM 2005

System Scalability Most nodes have <3 records Very few nodes have >3 records Left: Records distribution in a 104-node network; Right: Average RPT height changes with number of nodes. 11/10/2018 INFOCOM 2005

Operation Effectiveness When the #subscriber = 1 When the #subscriber changes 200 (2%) Left: Operation path length comparison; Right: Average path length changes with # of subscribers. 11/10/2018 INFOCOM 2005

Maintenance Costs Less than 1 node’s level index changed 90% message reduction Left: Level index update costs; Right: Failure recovery costs. 11/10/2018 INFOCOM 2005

Conclusion SCOPE provides scalable consistent supports for structured P2P systems Scalable structures Effective operations Minimal maintenance overhead Compared with existing solutions Better load balance Better fault tolerance Better consistency support 11/10/2018 INFOCOM 2005