DIMPLE: DynamIc Membership ProtocoL for Epidemic protocols Jin Sun, Paul Weber, Byung Choi, Roger Kieckhafer bkchoi@mtu.edu Michigan Technological University 11/15/2018

11/15/2018

Epidemic Protocols: Reliable Broadcasting K. Birman, M. Hayden, O. Ozkasap, Z. Xiao, M. Budiu, Y. Minsky, “Bimodal Multicast,” ACM Transactions on Computer Systems, 17(2), 41-88, May 1999. 11/15/2018

Background Two fundamental assumptions of epidemic protocols Random selection of next forwarders Randomly at uniform from the entire membership Nodes know the network size (N) Two approaches Centralized Distributed Entire membership at each and every node Different partial membership at different nodes 11/15/2018

Epidemic protocols on P2P systems? P2P systems can be very large in size, and very dynamic in membership Difficult to maintain a copy of the entire membership at each node Inconsistency Overhead Partial membership at each node? 11/15/2018

Challenges How to maintain partial membership at each node chosen randomly at uniform from the entire membership? How to provide different partial membership at different cycles? How to handle dangling pointers caused by churn? 11/15/2018

General Approach Exchange part of partial membership with part of another partial membership such that, from the node’s perspective, the partial membership is always a random selection of the entire membership at uniform 11/15/2018

Shuffle! A well known randomization method in gambling A practical and easy way of randomization Is it really random at uniform? What should happen if not perfectly random? Network partitioning! 11/15/2018

Resilience of Shuffle Probability of network partitioning is diminishingly small, practically zero! With reasonable sizes of Partial membership (O(log(N)) Shuffle length log(N) Regardless of frequencies A. Allavena, A. Demers, and J.E. Hopcroft. “Correctness of a gossip based membership protocol” , 24th ACM Symposium on Principles of Distributed Computing (PODC ’05). 11/15/2018

Shuffle Properties Convergence Global randomness, comparable to random networks Average shortest path length Clustering coefficient Regardless of frequencies S. Voulgaris, D. Gavidia, and M. van Steen. Cyclon: Inexpensive membership management for unstructured p2p overlays. Journal of Network and Systems Management, 13(2):197–217, June 2005. 11/15/2018

Shuffle with Churn? Not addressed in the previous work, huh? Network partition and churn are different! Measurement study on P2P systems shows: Average stay time under 10 minutes! S. Rhea, D. Geels, T. Roscoe, and J. Kubiatowicz, “Handling Churn in a DHT,” USENIX Technical Conference, 2004. 11/15/2018

Shuffle with Churn? Found not effective Produces a large portion of dangling pointers Would result in poor quality broadcasting Major reasons: Time delay to detect dangling pointers Time delay in join procedures 11/15/2018

DIMPLE Improves Shuffle Reinforcement At the end of each shuffle Quick Join Use a list of visited nodes 11/15/2018

DIMPLE-Shuffle works with churn! 11/15/2018

DIMPLE-Shuffle works with churn! 11/15/2018

DIMPLE-Shuffle works with churn! 11/15/2018

DIMPLE-Shuffle works with churn! 11/15/2018

DIMPLE: detects dangling pointers fast 11/15/2018

In-Degree Distribution 11/15/2018

Out-Degree Distribution 11/15/2018

DIMPLE: better quality of in-degrees 11/15/2018

DIMPLE: better quality of out-degrees 11/15/2018

Conclusions Contributions: Future work: DIMPLE makes shuffle to work with churn A good practical solution to dynamic membership service (for epidemic protocols) Future work: DIMPLE algorithms are improvable Network size (N) estimation utilizing shuffle is next! Self-organizing epidemic protocols 11/15/2018