Small-Scale Peer-to-Peer Publish/Subscribe Vinod Muthusamy, Hans-Arno Jacobsen University of Toronto July 17, 2005 P2PKM 2005 San Diego, CA

Small-Scale P2P Publish/Subscribe Motivation Many distributed hash table (DHT)-based peer-to-peer applications Storage (OceanStore, Past) Database (PIER) Naming service (Chord-DNS) Publish/subscribe (Scribe, P2P-ToPSS, Meghdoot) Typically designed for and evaluated on a large-scale Small-scale performance is important Even large systems start small Benefits even on small scale Small-scale case study of a DHT-based pub/sub protocol July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Agenda Background Publish/subscribe model Distributed hash tables Small-scale benefits Publish/subscribe on DHT Evaluation Conclusions July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Publish/subscribe model TSX Stock markets NASDAQ NYSE Simple interface Decoupled interaction Network efficiency Publisher Publisher AMGN=58 Publications IBM=84 ORCL=12 JNJ=58 HON=24 INTC=19 MSFT=27 Broker Network Notification Subscriptions Subscriber Subscriber Subscriptions: IBM > 85 ORCL < 10 JNJ > 60 July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Distributed hash tables Example: Pastry Distributed version of a hash table data structure Store (key, value) pairs Each node receives at most K/N keys Each node has about O(logN) routing state Lookups resolved with O(logN) hops July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Small-scale benefits Distributed pub/sub DHT Large-scale benefits (traditional focus) Multicast Distributed matching Organic (infrastructureless) scaling Self-organizing Small-scale benefits (untapped) (also large-scale benefits) Decoupled interaction Declarative bindings Incremental service deployment Late binding Effective use of commodity components (fault tolerance, load balancing) Incremental scalability July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Service oriented architecture (SOA) An illustration of small-scale benefits SOA: a distributed architecture Loosely coupled reusable components Enterprise service bus (ESB) comm. fabric Can be realized as a pub/sub broker network Pub/sub support in IBM Websphere, Sonic ESBs Example Three-tier Web server Communication between components through pubs and subs Decoupled interaction Declarative bindings Simple to add a Monitor component Incremental service deployment Late binding Application Server Enterprise Service Bus User Products Monitor Database Database July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Pub/sub case study P2P-ToPSS A pub/sub system built over DHT Large-scale performance previously studied [under submission] Evaluate small-scale benefits (of the same algorithm) now Subscription load balance July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Distributed multidimensional matching (DMM) 1) Pub/sub domain: 3) Network domain: price < 50 weight > 50 Subscription price = 10 weight = 70 Publication - 1 2) Spatial domain: 100 00 01 10 11 01 11 1 weight 00 10 DHT network used to map from tree nodes to peers price 100 July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Distributing the tree Let region r have z-code z, and tree node n(r) Store n(r) at peer p(r), where p(r) is where DHT would store key z Each peer performs mapping locally Hash function ensures even distribution Network price weight Region Subscription 1101 z-code price = … weight = … p/w p_w_1101 name/code AF3483BQ1 peer-id hash July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Load balancing Peers can be overloaded due to Excessive subscriptions storage Excessive publication traffic Overloaded peer delegates subscriptions to children in the DMM tree A local decision Network 00 01 p/w July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Evaluation SimPastry simulator (Microsoft Research) Pastry DHT Simulate network latencies, message costs Nodes connected in a LAN 1ms latency between every pair of nodes Subscriptions Attribute name + range E.g.: “20 < weight < 40” Metric Storage load of subscriptions July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe DHT performance 1000 random keys DHT successfully balances load July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Pub/sub performance weight price 1000 fine-grained subscriptions Load balancing distributes storage load July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Pub/sub performance weight price 1000 coarse-grained subscriptions Load balancing distributes but also increases storage load July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Observation Load balancing technique assumes available resources somewhere in the network Leads to runaway subscription delegation Common assumption in DHT applications CAN DHT’s active caching and replication Meghdoot’s hotspot circumvention In DHT, no knowledge of aggregate available resources Fundamental assumption of DHT applications not valid in small-scale networks July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale P2P Publish/Subscribe Conclusions Pub/sub and DHTs Proven benefits in large-scale network Untapped benefits in small-scale networks Common assumption on DHT applications Sufficient available resources in the network Small-scale networks break this assumption Future work Implement algorithms to estimate global resource availability Curb runaway subscription delegation with this information Evaluate other DHT applications in small-scale networks July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe

Small-Scale Peer-to-Peer Publish/Subscribe Q&A Small-Scale Peer-to-Peer Publish/Subscribe www.msrg.toronto.edu July 17, 2005 (P2PKM ’05) Small-Scale P2P Publish/Subscribe