Fabián E. Bustamante, Fall 2005 A brief introduction to Pastry Based on: A. Rowstron and P. Druschel, Pastry: Scalable, decentralized object location and routing for large-scale peer-to-peer systems, Middleware, Nov Presenter: Fabián E. Bustamante

CS 343 Operating Systems EECS, Northwestern University 2 Distributed Systems Architectures System architecture – Component placement and their interactions Client/Server –Server implementing a specific service –Client requests the service Peer-to-Peer –All peers are equal –Symmetric interaction between processes –Focusing on data-sharing systems

CS 343 Operating Systems EECS, Northwestern University 3 Data Sharing P2P Data sharing P2P apps rely on overlay network protocols for object storage/retrieval & message routing Classes of overlay protocols –Unstructured Few constraints on overlay construction & data placement Could support arbitrary complex queries, highly resilient to churn Restricted to inefficient, near-blind search strategies –Structured (DHT ~ Distributed Hash Tables) Constraining overlay structure & data placement Efficient object discovery Potential problems handling churn, exploiting node heterogeneity & supporting complex queries

CS 343 Operating Systems EECS, Northwestern University 4 Pastry DHT Pastry –“Scalable, decentralized object location and routing for large-scale P2P systems” Nodes assigned a unique 128-bit numeric node identifier (nodeId) (circular id space) NodeID can be expressed in based 2 b, b = 4 Given a msg & its 128-bit key, Pastry routes the msg to the node with numerically closest nodeId (in less than ) At each step, forward to node sharing with a key a 1 larger prefix or is numerically closer

CS 343 Operating Systems EECS, Northwestern University 5 Pastry Node State Each Pastry node maintains –Routing table Log N rows with 2 b - 1 entries Entries at row n share n digit with current node Choice of b involves a trade-off between size of populated portions of the routing table & maximum number of hos to route between two nodes (e.g. b = 4 & 10 6, avg. 75 entries and 5 hops) –Neighborhood set nodeIds and IP addresses of |M| nodes closest (proximity metric) to the local node (|M| ~ 2 * 2 b ) –Leaf set Set of |L|/2 numerically closest larger/smaller nodeIds relative to the present node’s nodeId (|L| ~ 2 b )

CS 343 Operating Systems EECS, Northwestern University 6 Routing in Pastry Routing a message with key D arriving at a node with nodeId A If (D is within range of leafset) Forward to Li such that |D - Li| is minimal Else // use routing table Let l be the length of prefix shared bet/ A and D If != null Forward to Else Forward to T in L U R U M such that the length of prefix between T and D is >= l and |T - D| < |A - D|

CS 343 Operating Systems EECS, Northwestern University 7 Pastry API Pastry exports (downcalls) –nodeId = pastryInit(Credentials, Applications) To join the local node (application is a handle to a set of procedures to invoke when certain events happen) –route(msg, key) Applications must export (upcalls) –deliver(msg, key) Call by Pastry when msg arrived for node –forward(msg, key, nextId) Call by Pastry just before forwarding the msg to node with nodeId = nextId –newLeafs(leafSet) Call by Pastry whenever there’s a change in local node leafset

CS 343 Operating Systems EECS, Northwestern University 8 Dynamics – Node Arrival Node X knows of nearby node A X asks A to route a join msg with key X, A routes it to node Z In response to join msg, A, Z and all in between send Z their state tables X initializes its own state tables –A’s neighborhood set is X’s –Z’s leafset is basis for X’s leafset (closest nodeId) –X’s R 0 (X 0 ) = A 0, X 1 = B 1, … where B is the first node in the route from A to Z X informs everyone who should know of its arrival

CS 343 Operating Systems EECS, Northwestern University 9 Example Applications PAST – A persistent storage service –fileId a function of file’s name & owner –Replicas of a file are stored on the k Pastry nodes with nodeId numerically closest to fileId Scribe – A publish/subscribe system –Subscribers are kept in node with nodeId numerically closest to topicId) Others –SplitStream –Squirrel –…