Fault-tolerance Lecture 15 Sam Madden Today: •Finish Chord

Slides:

Advertisements

Similar presentations

P2P data retrieval DHT (Distributed Hash Tables) Partially based on Hellerstein’s presentation at VLDB2004.

Advertisements

Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT and Berkeley presented by Daniel Figueiredo Chord: A Scalable Peer-to-peer.

Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK

Scalable Content-Addressable Network Lintao Liu

Peer-to-Peer (P2P) Distributed Storage 1Dennis Kafura – CS5204 – Operating Systems.

CHORD – peer to peer lookup protocol Shankar Karthik Vaithianathan & Aravind Sivaraman University of Central Florida.

Chord A Scalable Peer-to-peer Lookup Service for Internet Applications Prepared by Ali Yildiz (with minor modifications by Dennis Shasha)

The Chord P2P Network Some slides have been borowed from the original presentation by the authors.

Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications Speaker: Cathrin Weiß 11/23/2004 Proseminar Peer-to-Peer Information Systems.

1 1 Chord: A scalable Peer-to-peer Lookup Service for Internet Applications Dariotaki Roula

Xiaowei Yang CompSci 356: Computer Network Architectures Lecture 22: Overlay Networks Xiaowei Yang

Robert Morris, M. Frans Kaashoek, David Karger, Hari Balakrishnan, Ion Stoica, David Liben-Nowell, Frank Dabek Chord: A scalable peer-to-peer look-up protocol.

Distributed Hash Tables: Chord Brad Karp (with many slides contributed by Robert Morris) UCL Computer Science CS M038 / GZ06 27 th January, 2009.

Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Robert Morris Ion Stoica, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT.

Common approach 1. Define space: assign random ID (160-bit) to each node and key 2. Define a metric topology in this space,  that is, the space of keys.

Scaling Distributed Machine Learning with the BASED ON THE PAPER AND PRESENTATION: SCALING DISTRIBUTED MACHINE LEARNING WITH THE PARAMETER SERVER – GOOGLE,

Peer to Peer File Sharing Huseyin Ozgur TAN. What is Peer-to-Peer?  Every node is designed to(but may not by user choice) provide some service that helps.

1 Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Robert Morris Ion Stoica, David Karger, M. Frans Kaashoek, Hari Balakrishnan.

Topics in Reliable Distributed Systems Lecture 2, Fall Dr. Idit Keidar.

Looking Up Data in P2P Systems Hari Balakrishnan M.Frans Kaashoek David Karger Robert Morris Ion Stoica.

Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications Stoica et al. Presented by Tam Chantem March 30, 2007.

P2P: Advanced Topics Filesystems over DHTs and P2P research Vyas Sekar.

A Scalable Content- Addressable Network Sections: 3.1 and 3.2 Καραγιάννης Αναστάσιος Α.Μ. 74.

Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek and Hari alakrishnan.

Topics in Reliable Distributed Systems Fall Dr. Idit Keidar.

Peer To Peer Distributed Systems Pete Keleher. Why Distributed Systems? l Aggregate resources! –memory –disk –CPU cycles l Proximity to physical stuff.

Wide-area cooperative storage with CFS

1 Peer-to-Peer Networks Outline Survey Self-organizing overlay network File system on top of P2P network Contributions from Peter Druschel.

Lecture 10 Naming services for flat namespaces. EECE 411: Design of Distributed Software Applications Logistics / reminders Project Send Samer and me.

Chord A Scalable Peer-to-peer Lookup Service for Internet Applications Lecture 3 1.

Chord & CFS Presenter: Gang ZhouNov. 11th, University of Virginia.

Overlay network concept Case study: Distributed Hash table (DHT) Case study: Distributed Hash table (DHT)

 A P2P IRC Network Built on Top of the Kademlia Distributed Hash Table.

1 Reading Report 5 Yin Chen 2 Mar 2004 Reference: Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications, Ion Stoica, Robert Morris, david.

Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications Xiaozhou Li COS 461: Computer Networks (precept 04/06/12) Princeton University.

Node Lookup in P2P Networks. Node lookup in p2p networks In a p2p network, each node may provide some kind of service for other nodes and also will ask.

Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications.

Lecture 2 Distributed Hash Table

Peer to Peer A Survey and comparison of peer-to-peer overlay network schemes And so on… Chulhyun Park

1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.

Chord Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber Google,

1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.

Nick McKeown CS244 Lecture 17 Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications [Stoica et al 2001]

Peer-to-Peer (P2P) File Systems. P2P File Systems CS 5204 – Fall, Peer-to-Peer Systems Definition: “Peer-to-peer systems can be characterized as.

Data Structure of Chord  For each peer -successor link on the ring -predecessor link on the ring -for all i ∈ {0,..,m-1} Finger[i] := the peer following.

CS694 - DHT1 Distributed Hash Table Systems Hui Zhang University of Southern California.

Incrementally Improving Lookup Latency in Distributed Hash Table Systems Hui Zhang 1, Ashish Goel 2, Ramesh Govindan 1 1 University of Southern California.

CS 425 / ECE 428 Distributed Systems Fall 2015 Indranil Gupta (Indy) Peer-to-peer Systems All slides © IG.

Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications * CS587x Lecture Department of Computer Science Iowa State University *I. Stoica,

1 Distributed Hash tables. 2 Overview r Objective  A distributed lookup service  Data items are distributed among n parties  Anyone in the network.

CS Spring 2010 CS 414 – Multimedia Systems Design Lecture 24 – Introduction to Peer-to-Peer (P2P) Systems Klara Nahrstedt (presented by Long Vu)

The Chord P2P Network Some slides taken from the original presentation by the authors.

Peer-to-Peer Information Systems Week 12: Naming

Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M

CSE 486/586 Distributed Systems Distributed Hash Tables

The Chord P2P Network Some slides have been borrowed from the original presentation by the authors.

Distributed Hash Tables

Peer-to-Peer Data Management

(slides by Nick Feamster)

Accessing nearby copies of replicated objects

DHT Routing Geometries and Chord

Chord Advanced issues.

Peer-to-Peer (P2P) File Systems

Building Peer-to-Peer Systems with Chord, a Distributed Lookup Service

Chord Advanced issues.

Peer-to-Peer Storage Systems

EECS 498 Introduction to Distributed Systems Fall 2017

Consistent Hashing and Distributed Hash Table

A Scalable Peer-to-peer Lookup Service for Internet Applications

Peer-to-Peer Information Systems Week 12: Naming

Presentation transcript:

Fault-tolerance 6.033 Lecture 15 Sam Madden Today: •Finish Chord Quantifying reliability General approach to fault tolerance Replication Study of disk failures

Chord Distributed Hash Table N-1 N/2 N/4 Objects assigned ids in range 0..N Using hash function of object name Placed on ring Peers assigned range of ID space Responsible for objects in range Successor pointers used to find objects A Successor 1244 1244 stored on A D B C

Object Lookup in Chord lookup(“song.mp3”) A 1244 1244 stored on A D Successor pointers used to find objects lookup(name, dest): id = hash(name) if (id is local): send object to dest else successor.lookup(name, dest) A 1244 object 1244 stored on A D lookup(“song.mp3”) B C lookup(“song.mp3”)

Optimizing lookup A J 1244 I H B C G D F lookup E lookup(“song.mp3”) Problem: lookups require up to M messages, if M peers Idea: store pointers to additional peers in “finger table” Specifically to 2nd, 4th, 8th, …, 2log2M th peers 1244 I H B C G Neigh Range IP F 3500…4000 5.6.7.8 G 4000…6000 1.2.3.4 I 8000…10000 2.3.4.5 C 2500…3000 3.4.5.6 D F lookup E lookup(“song.mp3”)

Optimizing lookup A J 1244 I lookup H B C G D F E lookup(“song.mp3”) Log2M hops First step gets you at least 50% of the way, next step 25%, etc J 1244 I lookup H B C G Neigh Range IP J 10000…12000 4.5.6.7 A 0… 1500 1.2.2.3 C 2500…3000 3.4.5.6 G 4000…6000 1.2.3.4 D F E lookup(“song.mp3”)

Each hop uses next higher entry in finger table A B C D E F G H I J K L M N O P Dest Start Neigh Range IP J x K M A Neigh Range IP N x O A E Neigh Range IP P x A E I Neigh Range IP B x C E I Tables have Log2M entries

Using Chord Objects can be whole files, blocks, or pointers to files Chord used in web-scale file systems, Akamai-like CDNs, and file-sharing applications

Fault-tolerance Concepts: Quantifying reliability General approach to fault tolerance Replication Study of disk failures

Relative frequency of hardware replacement 10,000 machines Pr(failure in 1 year) ~.3 Schroeder and Gibson, FAST 2008

Bairavasundaram et al., SIGMETRICS 2007

Bairavasundaram et al., FAST 2008

Replicated Disks write (sector, data): write(disk1, sector, data) write(disk2, sector, data) read (sector, data): data = read (disk1, sector) if error data = read (disk2, sector) return error return data