Download presentation
Presentation is loading. Please wait.
1
INS/Twine : A Scalable Peer-to-Peer Architecture for Intentional Resource Discovery
Magdalena Balazinska, Hari Balakrishnan, and David Karger MIT – Laboratory for Computer Science
2
Problem Description Abundant ubiquitous computation and communication
Increasingly large computing environments Dynamic environments Many possible “cool” applications Locate resources using intentional descriptions
3
INR: Intentional Name Resolver
INS Overview Client describes attributes of required resources INR INR INR Self-configuring resolver network INR INR INR Resources advertise their capabilities INR: Intentional Name Resolver
4
Resource Discovery Goals
Allow client applications to locate services and devices Handle sophisticated resource descriptions Handle dynamism in the operating environment Scale to large numbers of resources
5
Existing Solutions for Scalability
Partitioning Cameras Resolver Bldg 3 Resolver ? Floors 1-3 Floors 4-6 Bldg 1 Bldg 2 Resolver Resolver Resolver Resolver Sensor Proxy Sensor Proxy Sensor Proxy
6
Existing Solutions for Scalability
Hierarchies Resolver Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy Sensor Proxy Sensor Proxy
7
INS/Twine Contributions
Collaborating peer resolvers: no content or location constraints on queries Scalability and load distribution via hash-based partitioning of resource descriptions among resolvers Semi-structured resource descriptions with arbitrary attribute-set Network dynamism Designed for an environment where all resources are equally important to users
8
INS/Twine Approach Overview
resource = camera resource = motion sensor Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic
9
INS/Twine Approach Overview
A resource advertises its descriptions and network location to any resolver The description is converted into a canonical form: attribute-value tree (AVTree) Using the content of the advertised description, the resolver determines which resolvers should know about the resource The resolver forwards the description to these resolvers for storage Similarly for queries
10
Architecture of One Resolver
Res adv. Resolver … Strand Mapper a1 v1 a2 v2 Strand h = hash(a1v1-a2v2) h : 0110 1001 0000 Key Router 0110 1001 0000 0110 1001 0000 Key Best( ) K nodes are chosen Distributed Hash Table
11
Splitting Descriptions into Strands
Resource description: AVTrees Six strands traffic root subject resource camera manufacturer ACompany model AModel resource camera subject traffic resource camera manufacturer model resource camera Each strand is then hashed into a 128 bit value which determines the nodes that will store the resource information. All queries, even short stranded queries require asking only one resolver! manufacturer resource camera ACompany model AModel resource camera
12
Distributed Hash Table: Chord
N5 N10 N110 N20 N99 Circular ID Space N32 Stores key-values for keys N80 N60 Keys Nodes and keys have 160-bit ID’s Chord maps ID’s to “successor” Successor: Node with next highest ID
13
Basic Lookup N120 N10 N105 N32 K80 N90 N60 “Where is key 80?”
Successor pointer N32 “N90 has K80” K80 N90 N60
14
“Finger table” allows log(N)-time lookups
K = log(n) immediate Successors for robustness Stabilization methods for concurrency 1/8 1/16 1/32 1/64 1/128 finger[i] points to successor (n + 2i) log(n) fingers in all N80
15
Back to Example Resolver Resolver Resolver Resolver Resolver Resolver
resource = camera resource = motion sensor Resolver subject = traffic Resolver Resolver Resolver Resolver Resolver Resolver Sensor Proxy subject = traffic resource = motion sensor subject = traffic resource = camera subject = traffic
16
Properties of INS/Twine
For a resource description with a attributes, t at the top-level : Number of strands is : s = 2a – t For R resources, S strands, K replication level, and N resolvers : Storage requirement at each resolver : (RSK)/N Advertisement: SK resolvers contacted (+ O(log N) for key routing) Query: K resolvers contacted (+ O(log N) for key routing) 100% success rate for less than K failures Failure rate decreases exponentially with K
17
State Management Resources join, move, leave and fail
Resolvers join and fail How to maintain consistency while achieving fault tolerance? Hard state Soft state Hybrid solution implemented in INS/Twine
18
INR: Intentional Name Resolver
State Management INR INR INR INR D D INR D d INR INR INR Resource INR: Intentional Name Resolver
19
INR: Intentional Name Resolver
State Management INR INR INR INR Remove Remove INR Remove INR INR INR Resource INR: Intentional Name Resolver
20
INR: Intentional Name Resolver
State Management INR INR INR INR D D INR D INR INR INR Resource d INR: Intentional Name Resolver
21
INR: Intentional Name Resolver
State Management INR Expire INR INR Expire INR INR Expire INR INR INR Resource INR: Intentional Name Resolver
22
Evaluation: Data Distribution
Data distribution rather even. Each resolvers holds a small fraction of resource descriptions
23
Evaluation: Query Resolution
Even distribution of queries among resolvers
24
Conclusion Intentional resource discovery
Scalable peer-to-peer network of resolvers Hash-based mapping of resource descriptions to resolvers Dynamic and even distribution of resource information and queries Handles dynamism of resources and resolvers
25
Appendix
26
INR: Intentional Name Resolver
INS Overview INR: Intentional Name Resolver
27
Describing Resources INS name-specifier XML AVTrees
28
Problems using concatenation
If numerous resources share the same prefix, some nodes may receive significantly more load than others Fully solving short stranded queries requires the colaboration of a linearly growing number of resolvers (with respect to network size) 1) and 2) are contradictory requirements!
29
Distributed Hash Table: Chord
A distributed hash-table is used to map keys onto resolvers efficiently: From: Chord: A Peer-to-Peer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, Frans Kaashoek, Hari Balakrishnan Proc. ACM SIGCOMM Conf., San Diego, CA, September 2001.
30
Problems using prefixes
More insertions for each resource. Small factor since we expect descriptions to be rather short Very popular prefixes may overload certain nodes : many advertisements and queries => the prefix should then become unusable Nodes stop storing resources for that prefix Nodes answer queries for the prefix specifying that they provide a partial answer due to the vague nature of the query
Similar presentations
