Peer to peer membership management for gossip based protocols

Slides:



Advertisements
Similar presentations
Sliver: A fast distributed slicing algorithm Ymir Vigfusson Cornell University Ymir Vigfusson Cornell University Vincent Gramoli EPFL & UniNE Switzerland.
Advertisements

Ranveer Chandra Ramasubramanian Venugopalan Ken Birman
12/09/01 1 SCAMP: lightweight membership service for gossip- based protocols Ayalvadi Ganesh, Anne-Marie Kermarrec & Laurent Massoulié Microsoft Research.
University of Würzburg Department of Distributed Systems Prof. Dr. P. Tran-Gia Challenges of (P2P-based) Scalable Multicast in Infrastructure-based Mobile.
A Survey of Key Management for Secure Group Communications Celia Li.
Pastry Peter Druschel, Rice University Antony Rowstron, Microsoft Research UK Some slides are borrowed from the original presentation by the authors.
SIMPLE Presence Traffic Optimization and Server Scalability Vishal Kumar Singh Henning Schulzrinne Markus Isomaki Piotr Boni IETF 67, San Diego.
SocioPlug Polystyrene: Survivable Shape for Self-Organising Data François Taïani Joint work with: Hoel Kervadec (INSA Rennes) Simon Bouget (ENS Rennes)
Decentralized Reactive Clustering in Sensor Networks Yingyue Xu April 26, 2015.
20/10/2006ALPAGE1 Ordered slicing of very large scale overlay networks Mark Jelasity University of Bologna, Italy Anne-Marie Kermarrec INRIA Rennes/IRISA,
Unstructured overlays: construction, optimization, applications Anne-Marie Kermarrec Joint work with Laurent Massoulié and Ayalvadi Ganesh.
Rheeve: A Plug-n-Play Peer- to-Peer Computing Platform Wang-kee Poon and Jiannong Cao Department of Computing, The Hong Kong Polytechnic University ICDCSW.
List servers (listservs) Skills: subscribe to a list server (listserv), send messages to a list and unsubscribe IT concepts: list server, subscription,
Intelligent Storage Project Keqiang Wu October 05, 2002.
Building Low-Diameter P2P Networks Eli Upfal Department of Computer Science Brown University Joint work with Gopal Pandurangan and Prabhakar Raghavan.
Overview Distributed vs. decentralized Why distributed databases
Application Layer Multicast for Earthquake Early Warning Systems Valentina Bonsi - April 22, 2008.
Scalable and Distributed GPS free Positioning for Sensor Networks Rajagopal Iyengar and Biplab Sikdar Department of ECSE, Rensselaer Polytechnic Institute.
Introduction Widespread unstructured P2P network
Communication (II) Chapter 4
RUNNING PARALLEL APPLICATIONS BEYOND EP WORKLOADS IN DISTRIBUTED COMPUTING ENVIRONMENTS Zholudev Yury.
Dominating Set Based and Power-aware Hierarchical Epidemics in P2P Systems Tugba KocEmrah CemOznur Ozkasap Department of Computer Engineering, Koç University,
Probabilistic Broadcast Presented by Keren Censor 1.
Epidemics Michael Ford Simon Krueger 1. IT’S JUST LIKE TELEPHONE! 2.
AXML Transactions Debmalya Biswas. 16th AprSEIW Transactions A transaction can be considered as a group of operations encapsulated by the operations.
CSE 486/586, Spring 2013 CSE 486/586 Distributed Systems Gossiping Steve Ko Computer Sciences and Engineering University at Buffalo.
2007/1/15http:// Lightweight Probabilistic Broadcast M2 Tatsuya Shirai M1 Dai Saito.
Introduction to ZooKeeper. Agenda  What is ZooKeeper (ZK)  What ZK can do  How ZK works  ZK interface  What ZK ensures.
Information-Centric Networks10b-1 Week 10 / Paper 2 Hermes: a distributed event-based middleware architecture –P.R. Pietzuch, J.M. Bacon –ICDCS 2002 Workshops.
Information-Centric Networks Section # 10.2: Publish/Subscribe Instructor: George Xylomenos Department: Informatics.
What can we see in the sky?. IN THE SKY WE CAN SEE MUCH MORE!
School of Electrical Engineering &Telecommunications UNSW Cost-effective Broadcast for Fully Decentralized Peer-to-peer Networks Marius Portmann & Aruna.
What is a distributed system? A network of processes. The nodes are processes, and the edges are communication channels.
CSE 486/586 CSE 486/586 Distributed Systems Gossiping Steve Ko Computer Sciences and Engineering University at Buffalo.
1 Roie Melamed, Technion AT&T Labs Araneola: A Scalable Reliable Multicast System for Dynamic Wide Area Environments Roie Melamed, Idit Keidar Technion.
Chapter 8 Fault Tolerance. Outline Introductions –Concepts –Failure models –Redundancy Process resilience –Groups and failure masking –Distributed agreement.
WCDP: A protocol for web cache consistency Renu Tewari IBM Almaden Research Thirumale Niranjan IBM Software Group
CSE 486/586 Distributed Systems Gossiping
Christian Scheideler Dept. of Computer Science
Resource subscription using DDS in oneM2M
Distributed Cache Technology in Cloud Computing and its Application in the GIS Software Wang Qi Zhu Yitong Peng Cheng
Introduction to Distributed Platforms
.NET Remoting Priyanka Bharatula.
Peer-to-peer networking
On Growth of Limited Scale-free Overlay Network Topologies
Clock-SI: Snapshot Isolation for Partitioned Data Stores
CHAPTER 3 Architectures for Distributed Systems
IBM WEBSPHERE MESSAGE QUEUE online Training | IBM WEBSPHERE MQ Training
#01 Client/Server Computing
SCOPE: Scalable Consistency in Structured P2P Systems
Soumya Sen (UPenn), Supervisors: Carl Livadas, Eve Schooler, DDI team
Providing Secure Storage on the Internet
CS5412: Using Gossip to Build Overlay Networks
RELIABILITY.
CS 4594 Broadband PNNI Signaling.
CS5412: Using Gossip to Build Overlay Networks
Indirect Communication Paradigms (or Messaging Methods)
B. Ramamurthy Based on Paper by Werner Vogels and Chris Re
Indirect Communication Paradigms (or Messaging Methods)
By Werner Vogels and Chris Re
By Werner Vogels and Chris Re
Peer-to-peer networking
SIENA: Wide-Area Event Notification Service
List servers (listservs)
Bing Liu, Xun Xiao, Sheng Jiang, Artur Hecker
#01 Client/Server Computing
CS5412: Using Gossip to Build Overlay Networks
IS 698/800-01: Advanced Distributed Systems Membership Management
Distributed Systems and Algorithms
Presentation transcript:

Peer to peer membership management for gossip based protocols Ayalvadi Ganesh Anne-Marie Kermarrec Laurent Massoulie

Obj 1 Obj 2 Obj 3 Obj 4 Obj 5 Obj 6

Obj 6 Obj 2 Obj 3 Obj 4 Obj 5 The sky is blue Obj 1 Obj 2 Obj 3 Obj 4 Obj 5 Obj 6 Obj 1 Obj 2 Obj 3 Obj 4 Obj 5 The sky is blue

Obj 6 Obj 1 Obj 3 Obj 4 Obj 5 Obj 6 Obj 2 Obj 3 Obj 4 Obj 5 Obj 1 Obj 2 Obj 3 Obj 4 Obj 6 Obj 2 Obj 1 Obj 4 Obj 5 Obj 6 Obj 2 Obj 3 Obj 1 Obj 5 Obj 5 Obj 6 Obj 6 Obj 2 Obj 3 Obj 4 Obj 1 Obj 1 Obj 2 Obj 3 Obj 4 Obj 5

Gossip protocol features Good reliability Good scalability Group membership – scalable ? No, memory requirement No, synchronization

Purpose Partial views Decentralized operation Isolation recovery

Data Structures PartialViewi InViewi Send list node i Receive list node i

Membership mgmt protocol Subscribe Contact New subscription Forwarded subscription Keeping a subscription Unsubscribe

pView = 2,3 pView = 2 pView = 1 inView = 2 inView = 2,3 Obj 1 Obj 2 inView = 1 P = 1 / (1 + size of pView) Obj 3 pView = 1 inView = 1

(c + 1) n log n Subscribe node out degree ‘d’ Generated arcs ‘d + c + 1’ E[Mn] E[Mn] = E[Mn-1] + E[Mn-1]/(n-1) + c + 1 E[Mn] = (c + 1) n log n(approx)

Unsubscribing Send unsubscribe - (1,l’ – c – 1) Send replace – (1,l’ – c – 1) Send unsubscribe – (l’ – c , l’) l’ the number of id’s in inView c the system reliability parameter

Obj 3 pView inView Obj 10 Obj 20 Obj 23 Obj 34 Obj 32 Obj 6 Obj 26 Obj 13 Obj 41 Obj 51

(c + 1) n log n pView = Mn / n E[Mn]/n , c+1 E[Mn-1] = E[Mn] - E[Mn]/n – (c+1)

Isolation Recovery Heartbeat In addition, Lease mechanism

Lease Mechanism Subscription lifetime Re-subscribe pView Serves : Reduces Concentration Automatic unsubscribe

Indirection Data Structures Forwarding rule Stopping rule pred(i), succ(i), wIJ, counter Forwarding rule Decrement counter Stopping rule

Distribution of partial view size n=50000

Indirection vs Random contact