Finding Nearby Objects in Peer-to-Peer Networks Kirsten Hildrum

Object Location

The Big Picture

Behind the Cloud ?

Two solutions One directory –One participant stores a lot, but average is low –Stretch is high Everyone has a directory –Low stretch! –Everyone has lots of data –Changes are very expensive

Distribute directory Each peer is responsible for part of directory Q: Who is responsible for my object? A: Hash peers into same namespace as objects. Q: How do I get there? A: Peer-to-peer networks route to names.  Must efficiently self-organize into network

Efficiency is… With respect to network Few neighbors Short routes (few hops & low stretch) Easy routing (No routing tables!) Dynamic With respect to directory Not too many entries (4,4) hard easy

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Related results Future work Outline

SystemNeighbor s Motivating Structure Hops CAN, 2001O(r)gridO(rn 1/r ) Chord, 2001O(log n)hypercubeO(log n) Pastry, 2001O(log n)hypercubeO(log n) Tapestry, 2001O(log n)hypercubeO(log n) Distributed Hash Tables These systems give –Guaranteed location –Join and leave algorithms –Load-balanced storage No stretch guarantees

Low Stretch Approaches SystemStretchSpaceBalancedMetric Awerbuch Peleg, 1991 polylog noGeneral PRR, 1997O(1)O(log n)yesSpecial Thorup-ZwickO(k 2 )O(kn 1/k )yesGeneral RRVV, 2001polylog yesGeneral Not dynamic Result #1: Make a dynamic low-stretch scheme

PRR/Tapestry City State Country

PRR/Tapestry Two object types: red and blue, so two trees Level 1 Level 2 Level 3

PRR/Tapestry Two object types: red and blue, so two trees Level 1 Level 2 Level 3

Neighbor Table For “5471” (Octal) Routing Levels xxx 2xxx 0xxx 3xxx 4xxx 5xxx 6xxx 7xxx 50xx 51xx 52xx 53xx 54xx 55xx 56xx 57xx 540x 541x 542x 543x 544x 545x 546x 547x Ø 5475 Ø 5477 Balancing Load 1 NodeID NodeID 5471 NodeID 5416 NodeID 5061 NodeID 5432 NodeID 5455 NodeID 5470

Building It Every node is a root! Can extend to more than two colors Pefix routing in base b  promoting at rate 1/b b=16  match one hex digit or four bits at a time Space: O(b log b n) –Every node at every level stores b parents We want the links to be of geometrically increasing length

Prefix Routing (PRR97) Nodes at the bottom can go red or blue Repeat at all levels, each node is a root of a tree defined by red/blue Space is O(n log n) –Each node appears in every level –Two parents per level If red=0, blue=1, then trees=binary IDs For object 10101, first send to a node beginning in 1, then 10, then 101….

Building It Repeat at all levels –Every node is a root! Can extend to more than two colors b colors  prefix routing in base b  promoting at rate 1/b b=16  match one hex digit or four bits at a time Space: O(b log b n) –Every node at every level stores b parents We want the links to be of geometrically increasing length

What’s Hard Level 1 Level 2 Level 3 ?

Node Joins Correctness O(log n) insert & delete –Concurrent inserts in a lock-free fashion Keeping low stretch –Required a nearest-neighbor search algorithm as subroutine –Keep O(1) stretch in special metrics Algorithms implemented in version of Tapestry used in OceanStore

Concurrent Insert

Solving the Problem Theorem 1 When peer A is finished inserting, it knows about all relevant peers that have finished insertion. Hildrum, Kubiatowicz, Rao & Zhao, “Distributed Object Location in a Dynamic Network” SPAA2002 Sequential Consistency (without locks) B A

Node Join in Practice large group join ~210 kill nodes RouteToID operations on 40 node cluster Churn: nodes join/leave every 10 seconds, average lifetime = 2mins Constant churn

Dynamic challenge: find parent Need to find closest at level Idea Use routing structure backwards Other approaches Clarkson ‘97 Karger & Ruhl ‘02 Krauthgamer Lee ‘03

Growth Restriction Growth Restriction: –Ball of radius 3r contains only a factor of c more nodes than ball of radius r. –Stronger than “polynomial growth” Base b at least c –Promote less than 1/c r 3r

Algorithm Idea, in pictures new

The Trimming Lemma Circle of radius r around new with at least one level i node. To find level (i-1) must find its parent. Level (i-1) node in little circle must point to a level i node in 3r of new <2r r 3r

How Many? Keep fixed k= O(log n) If k red nodes in little circle, then about ck in big circle Expect only about ck/b blue nodes If c/b < 1, expect less than k r 3r log n per level & log n levels, so total is O(log 2 n)

Using only O(log n) If k red nodes in little circle, then about ck in big circle Expect only about ck/b blue nodes If c/b < 1, expect constant per level But what is r? Guess, and backtrack if needed r 3r Total: O(log n)

Summary In growth-restricted networks (with no additional space!): Theorem 2 [HKRZ02] Can find nearest neighbor with high probability with O(log 2 n) messages Theorem 3 [HKMR04] Can find nearest neighbor, and messages is O(log n) with high probability + more  dynamic, low-stretch network

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Related results Future work Outline

Behind the Cloud Again

Dealing with faults Multiple paths –Castro et. al –One failure along path, path breaks Wide path –Paths faulty at the same place to break Exponential difference in width effect “retrofit” Tapestry to do latter in slightly malicious networks Failed! Still good…

Effective even for small overhead Theorem 4 In growth restricted spaces, can make probability of failed route less than 1/n c for width O(clog n) Hildrum & Kubiatowicz, DISC02

Wide path vs. multiple paths

Wide path vs. multiple paths (2)

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to Network Variation Related results Future work Outline

Network not homogeneous Previous schemes assumed global growth rate. But what if there isn’t one? San Francisco Nebraska Paris

b too big

b too small Level 1 Level 2 Level 3

Space as function of base

Using the local c Idea: Determine local c and use it Challenge: Paths still meet up, stretch still low, and space reasonable Technique: “Fix” or match bits –In variable size chunks –PRR97, LAND, Tapestry, Pastry matched bits in fixed size chunks r 3r

Using Local Growth Rate Level 1 We want the links to be of geometrically increasing length Find the rate so that level 1 is evenly spaced.  Bigger b on left, smaller on right

Summary Previous approaches assumed a global c Had to know c in advance of construction Hildrum, Krauthgamer & Kubiatowicz [SPAA04]: Dynamic, locally optimal low-stretch network

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Related results Future work Outline

Congestion Latency might not be right measure Reduce bandwidth usage When demands are known, can use linear program to find a set of paths But what if demands are not known…

Oblivious Routing S1S1 T1T1 S3S3 T3T3 S2S2 T2T2

The Answer: A Tree

Summary Räcke02 showed scheme exists O(log 3 n) Azar, Cohen, Fiat, Kaplan, & Räcke in STOC03 find optimal routing, but not hierarchical Theorem 6 Hierarchical scheme can be constructed in polynomial time. (Within O(log 2 n log log n) of optimal) –Harrelson, Hildrum, & Rao, “A Polynomial-time Tree Decomposition to Minimize Congestion” SPAA03

Online Auctions for Digital Goods Online Answer bidder i before seeing the next bid –Bob offers you 50¢, yes or no? –Don’t get to see Alice’s bid before answering Digital Each additional copy is free –Want to sell to everyone at their bid –Can’t sell to everyone at their bid, or else all bids will be low Bar-Yossef, Hildrum, Wu, “Online Auctions for Digital Goods” SODA 2002: Theorem 7 Constant competitive ratio when opt makes O(maxBid log log maxBid) Theorem 8 When best fixed price sells few, can get:

Low stretch dynamic peer-to-peer network Tolerate failures in network Adapting to network variation Related results Future work Outline

Future Work Object location questions –Tradeoff between degree, stretch, load balance as function of growth rate –Smooth transition between general-metric systems and growth-restricted systems Distributed low-congestion network Peer-to-peer networks with selfish players Sensor networks

Thank You!