Distributed Software Engineering Middleware for Reconfigurable Systems Gianpaolo Cugola Dipartimento di Elettronica e Informazione Politecnico di Milano,

Slides:



Advertisements
Similar presentations
Peer-to-Peer Infrastructure and Applications Andrew Herbert Microsoft Research, Cambridge
Advertisements

DISTRIBUTED COMPUTING PARADIGMS
REDS: A Reconfigurable Event Dispatching Infrastructure Gianpaolo Cugola Dipartimento di Elettronica e Informazione Politecnico di Milano, Italy
8 Feb 2006WPG Meeting1 WPG – PUG meeting 8 February 2006.
Chapter 10: Designing Databases
1 Mobile IPv6-Based Ad Hoc Networks: Its Development and Application Advisor: Dr. Kai-Wei Ke Speaker: Wei-Ying Huang.
Interest Management Objectives – –Understand what is meant by the term interest management. –Realise how interest management schemes may be deployed. –Understand.
1 EuroIMSA 2007 Chamonix, March th 2007 A PUBLISH SUBSCRIBE SUPPORT FOR NETWORKED MULTIPLAYER GAMES IASTED European Conference on INTERNET AND MULTIMEDIA.
Improving TCP Performance over Mobile Ad Hoc Networks by Exploiting Cross- Layer Information Awareness Xin Yu Department Of Computer Science New York University,
Ranveer Chandra , Kenneth P. Birman Department of Computer Science
Xyleme A Dynamic Warehouse for XML Data of the Web.
Rheeve: A Plug-n-Play Peer- to-Peer Computing Platform Wang-kee Poon and Jiannong Cao Department of Computing, The Hong Kong Polytechnic University ICDCSW.
Secure Multicast Xun Kang. Content Why need secure Multicast? Secure Group Communications Using Key Graphs Batch Update of Key Trees Reliable Group Rekeying.
Beneficial Caching in Mobile Ad Hoc Networks Bin Tang, Samir Das, Himanshu Gupta Computer Science Department Stony Brook University.
© nCode 2000 Title of Presentation goes here - go to Master Slide to edit - Slide 1 Reliable Communication for Highly Mobile Agents ECE 7995: Term Paper.
Design and Evaluation of a Wide-Area Event Notification Service Antonio Carzaniga David S. Rosenblum Alexander L. Wolf.
1 AINA 2006 Wien, April th 2006 DiVES: A DISTRIBUTED SUPPORT FOR NETWORKED VIRTUAL ENVIRONMENTS The IEEE 20th International Conference on Advanced.
1 Efficient Retrieval of User Contents in MANETs Marco Fiore, Claudio Casetti, Carla-Fabiana Chiasserini Dipartimento di Elettronica, Politecnico di Torino,
Building Low-Diameter P2P Networks Eli Upfal Department of Computer Science Brown University Joint work with Gopal Pandurangan and Prabhakar Raghavan.
Slide Set 15: IP Multicast. In this set What is multicasting ? Issues related to IP Multicast Section 4.4.
Anonymous Gossip: Improving Multicast Reliability in Mobile Ad-Hoc Networks Ranveer Chandra (joint work with Venugopalan Ramasubramanian and Ken Birman)
Hermes: A Distributed Event- Based Middleware Architecture Peter Pietzuch and Jean Bacon 1st DEBS Workshop, Vienna,
NFS. The Sun Network File System (NFS) An implementation and a specification of a software system for accessing remote files across LANs. The implementation.
Background Notification services in LAN Provides Notification Selection Notification Delivery Done on a centralized server (hence not scalable) Challenge.
Peer-to-peer file-sharing over mobile ad hoc networks Gang Ding and Bharat Bhargava Department of Computer Sciences Purdue University Pervasive Computing.
CS401 presentation1 Effective Replica Allocation in Ad Hoc Networks for Improving Data Accessibility Takahiro Hara Presented by Mingsheng Peng (Proc. IEEE.
Middleware for P2P architecture Jikai Yin, Shuai Zhang, Ziwen Zhang.
Client-Server Computing in Mobile Environments
Distributed Publish/Subscribe Network Presented by: Yu-Ling Chang.
GMD German National Research Center for Information Technology Innovation through Research Jörg M. Haake Applying Collaborative Open Hypermedia.
Network Design Essentials. Guide to Networking Essentials, Fifth Edition2 Contents 1. Examining the Basics of a Network Layout 2. Understanding Standard.
Chapter 2 Network Design Essentials Instructor: Nhan Nguyen Phuong.
Decentralized Scattering of Wake-up Times in Wireless Sensor Networks Amy L. Murphy ITC-IRST, Trento, Italy joint work with Alessandro Giusti, Politecnico.
On P2P Collaboration Infrastructures Manfred Hauswirth, Ivana Podnar, Stefan Decker Infrastructure for Collaborative Enterprise, th IEEE International.
GeoGrid: A scalable Location Service Network Authors: J.Zhang, G.Zhang, L.Liu Georgia Institute of Technology presented by Olga Weiss Com S 587x, Fall.
An affinity-driven clustering approach for service discovery and composition for pervasive computing J. Gaber and M.Bakhouya Laboratoire SeT Université.
Active Monitoring in GRID environments using Mobile Agent technology Orazio Tomarchio Andrea Calvagna Dipartimento di Ingegneria Informatica e delle Telecomunicazioni.
1 BitHoc: BitTorrent for wireless ad hoc networks Jointly with: Chadi Barakat Jayeoung Choi Anwar Al Hamra Thierry Turletti EPI PLANETE 28/02/2008 MAESTRO/PLANETE.
SAMANVITHA RAMAYANAM 18 TH FEBRUARY 2010 CPE 691 LAYERED APPLICATION.
Distributed Systems: Concepts and Design Chapter 1 Pages
1 AutoconfBOF2.PPT / Aug / Singh,Perkins,Clausen IETF Not Confidential Ad hoc network autoconfiguration: definition and problem statement (draft-singh-autoconf-adp-00.txt)
Scalable Web Server on Heterogeneous Cluster CHEN Ge.
DISTRIBUTED COMPUTING PARADIGMS. Paradigm? A MODEL 2for notes
This paper appears in: Computer Communications and Networks, ICCCN Proceedings.15th International Conference on 指導教授 : 許子衡 報告者 : 黃群凱 1.
Data Sharing. Data Sharing in a Sysplex Connecting a large number of systems together brings with it special considerations, such as how the large number.
Information-Centric Networks10b-1 Week 10 / Paper 2 Hermes: a distributed event-based middleware architecture –P.R. Pietzuch, J.M. Bacon –ICDCS 2002 Workshops.
PRIN WOMEN PROJECT Research Unit: University of Naples Federico II G. Ferraiuolo
AMQP, Message Broker Babu Ram Dawadi. overview Why MOM architecture? Messaging broker like RabbitMQ in brief RabbitMQ AMQP – What is it ?
Information-Centric Networks Section # 10.2: Publish/Subscribe Instructor: George Xylomenos Department: Informatics.
E81 CSE 532S: Advanced Multi-Paradigm Software Development Venkita Subramonian, Christopher Gill, Ying Huang, Marc Sentany Department of Computer Science.
Efficient Resource Allocation for Wireless Multicast De-Nian Yang, Member, IEEE Ming-Syan Chen, Fellow, IEEE IEEE Transactions on Mobile Computing, April.
Improving Fault Tolerance in AODV Matthew J. Miller Jungmin So.
By Nitin Bahadur Gokul Nadathur Department of Computer Sciences University of Wisconsin-Madison Spring 2000.
Addressing Data Compatibility on Programmable Network Platforms Ada Gavrilovska, Karsten Schwan College of Computing Georgia Tech.
CSE333 CSE333 DDS.1 A System for Creating Specialized DDS Architectures Research and Work By: Tom Puzak Nathan Viniconis Solomon Berhe Jeffrey Peck Project.
Network Topologies for Scalable Multi-User Virtual Environments Lingrui Liang.
Topic 4: Distributed Objects Dr. Ayman Srour Faculty of Applied Engineering and Urban Planning University of Palestine.
Supporting Mobile Collaboration with Service-Oriented Mobile Units
A Study of Group-Tree Matching in Large Scale Group Communications
Peer-to-peer networking
Wireless Sensor Network Architectures
Effective Replica Allocation
Distributed Publish/Subscribe Network
SAMANVITHA RAMAYANAM 18TH FEBRUARY 2010 CPE 691
JINI ICS 243F- Distributed Systems Middleware, Spring 2001
Message Queuing.
Market-based Dynamic Task Allocation in Mobile Surveillance Systems
Indirect Communication Paradigms (or Messaging Methods)
Indirect Communication Paradigms (or Messaging Methods)
Presentation transcript:

Distributed Software Engineering Middleware for Reconfigurable Systems Gianpaolo Cugola Dipartimento di Elettronica e Informazione Politecnico di Milano, Italy joint work with Gian Pietro Picco, Paolo Costa, Davide Frey, Matteo Migliavacca, and Amy Murphy

Politecnico di Milano IS-MANET kick-off meeting2 Outline Sources of reconfiguration Publish-subscribe middleware (Jedi) –Managing reconfigurations in distributed publish-subscribe middleware –Reducing event loss during reconfigurations by using epidemic algorithms State-based, peer-to-peer middleware (PeerWare) –A model combining state based and reactive communication –A middleware which adopts such model Future work

Politecnico di Milano IS-MANET kick-off meeting3 Sources of reconfiguration Modern distributed applications must be able to operate in dynamic environments At the network level, dynamicity comes from: –Organizational changes in companies, which results in changes in the company network New departments, merging among companies, … –Mobile computing Nomadic users, base-station, and ad-hoc scenarios –Peer-to-peer networking Peers come and go during the application lifetime Building reconfigurable systems is a challenge for software engineering

Politecnico di Milano IS-MANET kick-off meeting4 Middleware for reconfigurable systems - 1 To efficiently build reconfigurable applications developers need new middleware –Applications must be able to reconfigure themselves... –... and cannot rely on common notions like the address of remote components Standard C/S middleware, like RMI and CORBA, present several drawbacks –Being based on a strong coupling among clients and servers... –... which reduces the possibility of dynamically changing the configuration of the application to cope with changes in connectivity –Even if some services have been added to the above mentioned middleware to try to overcome these problems

Politecnico di Milano IS-MANET kick-off meeting5 Middleware for reconfigurable systems - 2 In the last years, different middleware have shown their benefit in a dynamic environment –Publish-Subscribe middleware –State-based, Linda like, middleware The SE group at Politecnico di Milano has gained a good experience with both –Jedi: A publish-subscribe middleware for large scale applications –Lime: A state-based middleware based on the Linda paradigm, explicitly conceived for mobile computing Recently we got even further by extending Jedi and developing PeerWare, a new state-based, p2p middleware

Politecnico di Milano IS-MANET kick-off meeting6 Publish-Subscribe Middleware Communication is based on asynchronous event notifications, published by clients Clients subscribe to event classes by issuing a subscription Client components are attached to an event dispatcher, responsible for dispatching events to subscribers Publish-subscribe middleware varies in terms of: –Event format & subscription language –Architecture Dispatcher CCC C Temperature > 20 o CTemperature = 25 o C

Politecnico di Milano IS-MANET kick-off meeting7 Subscription Language The expressivity of the subscription language allows to distinguish between: –Subject-based The set of subjects is determined a priori Analogous to multicast –Content-based Subscriptions contain expressions (event patterns) that allow clients to filter events based on their content The pattern set is determined by client subscriptions A single event may match multiple subscriptions

Politecnico di Milano IS-MANET kick-off meeting8 Architecture of the event dispatcher Most publish-subscribe middleware adopt a centralized event dispatcher To improve scalability and performance, most advanced publish-subscribe middleware offer a distributed event dispatcher –Where a set of interconnected dispatching servers cooperate to distribute events Distributed publish-subscribe middleware differ in the way such dispatching servers are connected ad in the way subscriptions are propagated

Politecnico di Milano IS-MANET kick-off meeting9 Subscription Forwarding Every dispatcher propagates subscriptions to the others –Subscriptions are never sent twice through the same link Events follow the routes laid by subscriptions

Politecnico di Milano IS-MANET kick-off meeting10 Reconfiguration To operate in dynamic environments, distributed publish- subscribe middleware must provide mechanisms to reconfigure the event dispatching network –No publish-subscribe system to date deals with this kind of reconfiguration Supporting reconfiguration implies solving three problems: –How to reconstruct the overlay network Open issue… to be solved in is-manet –How rearrange the subscription information Content-based makes the problem very different from existing subject- based solutions, inspired by multicast –How to minimize event loss during reconfiguration Epidemic algorithms We considered the three problems as orthogonal

Politecnico di Milano IS-MANET kick-off meeting11 Removing a link The graph is partitioned in two Unsubscriptions are propagates along the two subgraphs, starting at the broken link A B C D

Politecnico di Milano IS-MANET kick-off meeting12 Adding a link The two subgraphs are merged back into one Subscriptions are propagated across the new link A B C D

Politecnico di Milano IS-MANET kick-off meeting13 Drawbacks In the case of a link substitution, the ordering of operations affects the efficiency of the reconfiguration All the subscriptions in the left subtree are removed, but the blue ones are reinserted immediately after A B C D

Politecnico di Milano IS-MANET kick-off meeting14 Understanding Propagation A subscription is propagated following the unique route up to the closest splitter, if it exists; to the whole tree, otherwise –A splitter is either a subscriber or a dispatcher whose routing table has two or more entries for the given subscription A B C D Adding a subscription is usually cheap The more splitters, the shorter the path a subscription must follow

Politecnico di Milano IS-MANET kick-off meeting15 The Idea To appear in Int. Conf. on Distributed Computing Systems (ICDCS03) The idea is to perform unsubscriptions first, and subscriptions after –This way, the network is kept dense of subscriptions, and the impact of reconfiguration is reduced Unsubscriptions are managed through a timer, which is set when a link breakage is detected Other optimizations are provided, e.g., for the case where a dispatcher belongs to a broken link and a new link The idea is very simple but: –It is based on a thorough understanding of the dispatching strategy –It provides significant improvements

Politecnico di Milano IS-MANET kick-off meeting16 Simulations: Event Delivery Goal: understand whether the algorithm converges, even under a barrage of reconfigurations non-overlapping overlapping

Politecnico di Milano IS-MANET kick-off meeting17 Simulations: Overhead Reduction Better performance with a sparse graph The impact of misrouted events is relevant

Politecnico di Milano IS-MANET kick-off meeting18 Reducing event loss during reconfiguration We need a mechanism to reduce event loss during reconfiguration –It should be scalable, light, and resilient to reconfigurations The idea: why not try to apply epidemic (gossip) algorithms to recover lost messages? Epidemic algorithms: –Each process periodically communicates its view of the system state to a random subset of the other processes –Information propagates epide- mically up to a stable state

Politecnico di Milano IS-MANET kick-off meeting19 Gossip & reconfigurable publish- subscribe middleware Content-based routing introduces new problems Issues to solve –How to route gossip messages? Content-based routing schema do not have any notion of group –How to detect event loss? Simple numbering schemas are not enough Different approaches –Push –Pattern-based pull –Source-based pull Same approaches can be used to recover event lost during reconfigurations or because of channel failures

Politecnico di Milano IS-MANET kick-off meeting20 Simulations Overlapping reconfigurations

Politecnico di Milano IS-MANET kick-off meeting21 Simulations Unreliable channels

Politecnico di Milano IS-MANET kick-off meeting22 Simulations Delivery rate vs. size

Politecnico di Milano IS-MANET kick-off meeting23 Simulations Overhead (gossipMsgs/events)

Politecnico di Milano IS-MANET kick-off meeting24 PeerWare: Some facts and an idea Some facts –Sometimes it is necessary to have state based communication... –... other times messages are the best form of communication –As demonstrated by applications like gnutella, peer-to-peer architectures perfectly fit the needs of flexibility and reconfigurability of several classes of distributed applications The idea –Merge in a single, integrated model the two forms of communication A model that could be implemented by a scalable middleware –Develop a peer to peer middleware that adopts this model

Politecnico di Milano IS-MANET kick-off meeting25 PeerWare: The general model Based on the notion of global virtual data structure –Each mobile unit carries a piece of a global data structure, e.g., a subtree, a subset, a submatrix, … –When connectivity is available, the portion of the global data structure available to the unit is larger, and determined by the other connected units Middleware carries the burden of maintaining the illusion of local access to a global data structure Wide (and largely unexplored) range of design choices –Rules for breaking/merging the data structure, proactive vs. reactive primitives, definition of connectivity, …

Politecnico di Milano IS-MANET kick-off meeting26 PeerWare: The specific model PeerWare maintains a gvds organized as a graph of nodes and documents –Each node may be part of zero or one parent nodes –Each document may belong to one or more nodes Each component connected to PeerWare contributes the gvds with a set of connected nodes and documents The gvds managed by PeerWare is built by merging these documents and nodes

Politecnico di Milano IS-MANET kick-off meeting27 PeerWare: Engagement Nodes with the same identifier (i.e., path) are merged together n1 n2n3 d1 d3 n7 n8 n10 d7 d8 n9 d6 Component 1 n1 n2n3 d1 d2 n4n5 n6 d5 d4 Component 2 n7 n8 n10 d7 d8 n9 d6 n1 n2n3 d2 n4n5 n6 d5 d4 d3 dds

Politecnico di Milano IS-MANET kick-off meeting28 PeerWare: Primitives PeerWare provides primitives to: –Join and leave the system –Manipulate the local repository of each component i.e., the set of nodes and documents held by each component –Query the distributed data structure –Register subscriptions to events occurring on data items –Publish event notifications for events related to data items

Politecnico di Milano IS-MANET kick-off meeting29 PeerWare: Global primitives Item[] execute(NodeFilter, DataFilter, Action, Callback, Mode) –The query is distributed to the set of hosts holding at least one node matching NodeFilter –Locally, the set M of items that belong to these nodes and match the DataFilter is determined –M is passed (locally) to the user-provided Action that manipulates it (e.g., by filtering out part of their content) –The set of items returned by the action is given back to the caller Through the Callback if the asynchronous mode is specified As a single array of data if the synchronous mode is specified

Politecnico di Milano IS-MANET kick-off meeting30 PeerWare: Global primitives SubscriptionID subscribe(NodeFilter, DataFilter, EventFilter, Callback) –Events matching NodeFilter,... –... occurring on a data item matching DataFilter,... –... and belonging to a node matching NodeFilter... –... are returned to the caller through the specified Callback

Politecnico di Milano IS-MANET kick-off meeting31 PeerWare: Global primitives executeAndSubscribe( NodeFilter, DataFilter, EventFilter, Action, Callback, Mode) –Equivalent to an execute atomically followed (on each single host) by a subscribe –It allows programmers to hook upon some data and provide strong consistency on the content of that data Through event notifications that inform the caller of any further change on that data

Politecnico di Milano IS-MANET kick-off meeting32 PeerWare: Different scenarios Peer to peer with base stations Dynamic fringe of mobile or disconnected nodes Fixed backbone of PeerWare servers

Politecnico di Milano IS-MANET kick-off meeting33 PeerWare: Different scenarios Strongly peer to peer Mobile hosts connects in an ad- hoc network Querying and publishing/subscribing proceed as before

Politecnico di Milano IS-MANET kick-off meeting34 PeerWare: The prototype Current prototype is oriented to enterprise networks –Base station scenario Provides limited mechanisms to reconfigure the overlay network of peers –Only leaf nodes may join and leave at run-time

Politecnico di Milano IS-MANET kick-off meeting35 PeerWare: Conclusions PeerWare brings together the best of two worlds... –It is possible to mix together message-based and state-based communication In a single model (with nodes acting as a unique mechanism to limit the scope of queries and subscriptions) –It does not suffer of the scalability problems of Linda-like middleware Access to the distributed data structure is not atomic Events combined with the executeAndSubscribe primitive provide great expressive power... in a peer to peer architecture Applicability –Within the EU project MOTION PeerWare has been used to develop a collaborative application in an enterprise scenario

Politecnico di Milano IS-MANET kick-off meeting36 Future work Publish-subscribe middleware –Find some mechanisms to maintain the overlay network –Generalize the approaches to other architectures (generic graphs) –Introduce these mechanisms in Jedi PeerWare –Port the mechanisms above to PeerWare to support more dynamic scenarios –Validate the middleware by using it in different environments