Magdalena Balazinska, Hari Balakrishnan, and David Karger

Slides:



Advertisements
Similar presentations
Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT and Berkeley presented by Daniel Figueiredo Chord: A Scalable Peer-to-peer.
Advertisements

Peer to Peer and Distributed Hash Tables
Kademlia: A Peer-to-peer Information System Based on the XOR Metric.
CHORD – peer to peer lookup protocol Shankar Karthik Vaithianathan & Aravind Sivaraman University of Central Florida.
Chord A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert MorrisDavid, Liben-Nowell, David R. Karger, M. Frans Kaashoek,
Technische Universität Yimei Liao Chemnitz Kurt Tutschku Vertretung - Professur Rechner- netze und verteilte Systeme Chord - A Distributed Hash Table Yimei.
Technische Universität Chemnitz Kurt Tutschku Vertretung - Professur Rechner- netze und verteilte Systeme Chord - A Distributed Hash Table Yimei Liao.
Chord: A Scalable Peer-to- Peer Lookup Service for Internet Applications Ion StoicaRobert Morris David Liben-NowellDavid R. Karger M. Frans KaashoekFrank.
The Chord P2P Network Some slides have been borowed from the original presentation by the authors.
CHORD: A Peer-to-Peer Lookup Service CHORD: A Peer-to-Peer Lookup Service Ion StoicaRobert Morris David R. Karger M. Frans Kaashoek Hari Balakrishnan Presented.
Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M
Lecture 5 - Routing On the Flat Labels M.Sc Ilya Nikolaevskiy Helsinki Institute for Information Technology (HIIT)
Chord: A scalable peer-to- peer lookup service for Internet applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashock, Hari Balakrishnan.
Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented.
Robert Morris, M. Frans Kaashoek, David Karger, Hari Balakrishnan, Ion Stoica, David Liben-Nowell, Frank Dabek Chord: A scalable peer-to-peer look-up.
Robert Morris, M. Frans Kaashoek, David Karger, Hari Balakrishnan, Ion Stoica, David Liben-Nowell, Frank Dabek Chord: A scalable peer-to-peer look-up protocol.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Robert Morris Ion Stoica, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion StoicaRobert Morris David Liben-NowellDavid R. Karger M. Frans KaashoekFrank.
Common approach 1. Define space: assign random ID (160-bit) to each node and key 2. Define a metric topology in this space,  that is, the space of keys.
Presented by Elisavet Kozyri. A distributed application architecture that partitions tasks or work loads between peers Main actions: Find the owner of.
1 Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Robert Morris Ion Stoica, David Karger, M. Frans Kaashoek, Hari Balakrishnan.
Topics in Reliable Distributed Systems Lecture 2, Fall Dr. Idit Keidar.
Introduction to Peer-to-Peer (P2P) Systems Gabi Kliot - Computer Science Department, Technion Concurrent and Distributed Computing Course 28/06/2006 The.
Looking Up Data in P2P Systems Hari Balakrishnan M.Frans Kaashoek David Karger Robert Morris Ion Stoica.
Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications Stoica et al. Presented by Tam Chantem March 30, 2007.
Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Lecture 2: Peer-to-Peer.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek and Hari alakrishnan.
Chord-over-Chord Overlay Sudhindra Rao Ph.D Qualifier Exam Department of ECECS.
Secure Overlay Services Adam Hathcock Information Assurance Lab Auburn University.
Topics in Reliable Distributed Systems Fall Dr. Idit Keidar.
Peer To Peer Distributed Systems Pete Keleher. Why Distributed Systems? l Aggregate resources! –memory –disk –CPU cycles l Proximity to physical stuff.
Structured P2P Network Group14: Qiwei Zhang; Shi Yan; Dawei Ouyang; Boyu Sun.
CSE 461 University of Washington1 Topic Peer-to-peer content delivery – Runs without dedicated infrastructure – BitTorrent as an example Peer.
Chord & CFS Presenter: Gang ZhouNov. 11th, University of Virginia.
1 Reading Report 5 Yin Chen 2 Mar 2004 Reference: Chord: A Scalable Peer-To-Peer Lookup Service for Internet Applications, Ion Stoica, Robert Morris, david.
Wireless Networks of Devices (WIND) Hari Balakrishnan and John Guttag MIT Lab for Computer Science NTT-MIT Meeting, January 2000.
Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications Xiaozhou Li COS 461: Computer Networks (precept 04/06/12) Princeton University.
Vincent Matossian September 21st 2001 ECE 579 An Overview of Decentralized Discovery mechanisms.
Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT and Berkeley presented by Daniel Figueiredo Chord: A Scalable Peer-to-peer.
Presentation 1 By: Hitesh Chheda 2/2/2010. Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT Laboratory for Computer Science.
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications.
Presented by: Tianyu Li
Chord: A Scalable Peer-to-peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan Presented.
Lecture 2 Distributed Hash Table
1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.
Chord Fay Chang, Jeffrey Dean, Sanjay Ghemawat, Wilson C. Hsieh, Deborah A. Wallach, Mike Burrows, Tushar Chandra, Andrew Fikes, Robert E. Gruber Google,
Idit Keidar, Principles of Reliable Distributed Systems, Technion EE, Spring Principles of Reliable Distributed Systems Lecture 2: Distributed Hash.
LOOKING UP DATA IN P2P SYSTEMS Hari Balakrishnan M. Frans Kaashoek David Karger Robert Morris Ion Stoica MIT LCS.
Design and implementation of an intentional naming system William Adjie-WinotoElliot Schwartz Hari BalakrishnanJeremy Lilley MIT Laboratory for Computer.
CS 347Notes081 CS 347: Parallel and Distributed Data Management Notes 08: P2P Systems.
1 Secure Peer-to-Peer File Sharing Frans Kaashoek, David Karger, Robert Morris, Ion Stoica, Hari Balakrishnan MIT Laboratory.
Md Tareq Adnan Centralized Approach : Server & Clients Slow content must traverse multiple backbones and long distances Unreliable.
Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications * CS587x Lecture Department of Computer Science Iowa State University *I. Stoica,
The Chord P2P Network Some slides taken from the original presentation by the authors.
A Survey of Peer-to-Peer Content Distribution Technologies Stephanos Androutsellis-Theotokis and Diomidis Spinellis ACM Computing Surveys, December 2004.
Ion Stoica, Robert Morris, David Liben-Nowell, David R. Karger, M
MIT – Laboratory for Computer Science
The Chord P2P Network Some slides have been borrowed from the original presentation by the authors.
A Scalable Peer-to-peer Lookup Service for Internet Applications
(slides by Nick Feamster)
DHT Routing Geometries and Chord
Building Peer-to-Peer Systems with Chord, a Distributed Lookup Service
Internet Indirection Infrastructure
Reading Report 11 Yin Chen 1 Apr 2004
Distributed Hash Tables
MIT LCS Proceedings of the 2001 ACM SIGCOMM Conference
Consistent Hashing and Distributed Hash Table
P2P: Distributed Hash Tables
A Scalable Peer-to-peer Lookup Service for Internet Applications
#02 Peer to Peer Networking
Presentation transcript:

INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery Magdalena Balazinska, Hari Balakrishnan, and David Karger MIT – Laboratory for Computer Science http://nms.lcs.mit.edu/

Problem Description Abundant ubiquitous computation and communication Increasingly large computing environments Dynamic environments Many possible “cool” applications Locate resources using intentional descriptions

INR: Intentional Name Resolver INS Overview Client describes attributes of required resources INR INR INR Self-configuring resolver network INR INR INR Resources advertise their capabilities INR: Intentional Name Resolver

Resource Discovery Goals Allow client applications to locate services and devices Handle sophisticated resource descriptions Handle dynamism in the operating environment Scale to large numbers of resources

Existing Solutions for Scalability Partitioning Cameras Resolver Bldg 3 Resolver ? Floors 1-3 Floors 4-6 Bldg 1 Bldg 2 Resolver Resolver Resolver Resolver Sensor Proxy Sensor Proxy Sensor Proxy

Existing Solutions for Scalability Hierarchies Resolver Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy Sensor Proxy Sensor Proxy

INS/Twine Contributions Collaborating peer resolvers: no content or location constraints on queries Scalability and load distribution via hash-based partitioning of resource descriptions among resolvers Semi-structured resource descriptions with arbitrary attribute-set Network dynamism Designed for an environment where all resources are equally important to users

INS/Twine Approach Overview resource = camera resource = motion sensor Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic

INS/Twine Approach Overview A resource advertises its descriptions and network location to any resolver The description is converted into a canonical form: attribute-value tree (AVTree) Using the content of the advertised description, the resolver determines which resolvers should know about the resource The resolver forwards the description to these resolvers for storage Similarly for queries

Architecture of One Resolver Res adv. Resolver … Strand Mapper a1 v1 a2 v2 Strand h = hash(a1v1-a2v2) h : 0110 1001 0000 Key Router 0110 1001 0000 0110 1001 0000 Key Best(01101001000) K nodes are chosen Distributed Hash Table

Splitting Descriptions into Strands Resource description: AVTrees Six strands traffic root subject resource camera manufacturer ACompany model AModel resource camera subject traffic resource camera manufacturer model resource camera Each strand is then hashed into a 128 bit value which determines the nodes that will store the resource information. All queries, even short stranded queries require asking only one resolver! manufacturer resource camera ACompany model AModel resource camera

Distributed Hash Table: Chord N5 N10 N110 N20 N99 Circular ID Space N32 Stores key-values for keys 21..32 N80 N60 Keys 33..60 Nodes and keys have 160-bit ID’s Chord maps ID’s to “successor” Successor: Node with next highest ID

Basic Lookup N120 N10 N105 N32 K80 N90 N60 “Where is key 80?” Successor pointer N32 “N90 has K80” K80 N90 N60

“Finger table” allows log(N)-time lookups ¼ ½ K = log(n) immediate Successors for robustness Stabilization methods for concurrency 1/8 1/16 1/32 1/64 1/128 finger[i] points to successor (n + 2i) log(n) fingers in all N80

Back to Example Resolver Resolver Resolver Resolver Resolver Resolver resource = camera resource = motion sensor Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic

Properties of INS/Twine For a resource description with a attributes, t at the top-level : Number of strands is : s = 2a – t For R resources, S strands, K replication level, and N resolvers : Storage requirement at each resolver : (RSK)/N Advertisement: SK resolvers contacted (+ O(log N) for key routing) Query: K resolvers contacted (+ O(log N) for key routing) 100% success rate for less than K failures Failure rate decreases exponentially with K

State Management Resources join, move, leave and fail Resolvers join and fail How to maintain consistency while achieving fault tolerance? Hard state Soft state Hybrid solution implemented in INS/Twine

INR: Intentional Name Resolver State Management INR INR INR INR D D INR D d INR INR INR Resource INR: Intentional Name Resolver

INR: Intentional Name Resolver State Management INR INR INR INR Remove Remove INR Remove INR INR INR Resource INR: Intentional Name Resolver

INR: Intentional Name Resolver State Management INR INR INR INR D D INR D INR INR INR Resource d INR: Intentional Name Resolver

INR: Intentional Name Resolver State Management INR Expire INR INR Expire INR INR Expire INR INR INR Resource INR: Intentional Name Resolver

Evaluation: Data Distribution Data distribution rather even. Each resolvers holds a small fraction of resource descriptions

Evaluation: Query Resolution Even distribution of queries among resolvers

Conclusion Intentional resource discovery Scalable peer-to-peer network of resolvers Hash-based mapping of resource descriptions to resolvers Dynamic and even distribution of resource information and queries Handles dynamism of resources and resolvers http://nms.lcs.mit.edu/projects/twine/

Appendix

INR: Intentional Name Resolver INS Overview INR: Intentional Name Resolver

Describing Resources INS name-specifier XML AVTrees

Problems using concatenation If numerous resources share the same prefix, some nodes may receive significantly more load than others Fully solving short stranded queries requires the colaboration of a linearly growing number of resolvers (with respect to network size) 1) and 2) are contradictory requirements!

Distributed Hash Table: Chord A distributed hash-table is used to map keys onto resolvers efficiently: From: Chord: A Peer-to-Peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, Frans Kaashoek, Hari Balakrishnan Proc. ACM SIGCOMM Conf., San Diego, CA, September 2001.

Problems using prefixes More insertions for each resource. Small factor since we expect descriptions to be rather short Very popular prefixes may overload certain nodes : many advertisements and queries => the prefix should then become unusable Nodes stop storing resources for that prefix Nodes answer queries for the prefix specifying that they provide a partial answer due to the vague nature of the query