NaradaBrokering – Building P2P Grids Access Grid meeting – August 21st, 2002. PTLIU Laboratory for Community Grids Geoffrey Fox, Shrideep Pallickara Computer Science, Informatics, Physics Indiana University, Bloomington IN 47404 http://www.naradabrokering.org http://grids.ucs.indiana.edu/ptliupages 11/12/2018 uri="http://www.naradabrokering.org" email="gcf@indiana.edu"

http://www.naradabrokering.org/ gcf, spallick@indiana.edu JXTA and Grids JXTA and Grid architectures can be implemented as Web Services interacting with (XML-based) messages We built a “Grid Messaging System” NaradaBrokering that implements generalized publish-subscribe mechanism in a network of “brokers/rendezvous peers” NaradaBrokering can replace Java Message Service – Grid-like system Used to run our synchronous collaboration system Garnet supporting shared display, text chats, Jabber instant messenger …. NaradaBrokering is integrated with JXTA (as a proxy to rendezvous peers) and can provide reliable messaging between peer groups (and inside?) We are building Collaboration (shared application and audio-video conferencing) as a Web Service XGSP (XML General Session Protocol) is meant to include H323 SIP and (later) JXTA sessions (peer groups) JXTA will be able to invoke Access Grid, Polycom, Shared Display sessions 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Different Web Service Organizations Everything is a resource implemented as a Web Service, whether it be: back end supercomputers and a petabyte data Microsoft PowerPoint and this file Web Services communicate by messages ….. Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport ….. Gives a Peer-to-Peer Grid NaradaBrokering is an example of Event or Message Service linking web services together 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Peer to Peer Grid Peers Integrate P2P and Grid/WS Peers Database Database Peers Resource Facing Web Service Interfaces Event/ Message Brokers Integrate P2P and Grid/WS Peer to Peer Grid Web Service Interfaces Peers User Facing Web Service Interfaces A democratic organization Peer to Peer Grid 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Role of Event/Message Brokers We will use events and messages interchangeably An event is a time stamped message Our systems are built from clients, servers and “event brokers” These are logical functions – a given computer can have one or more of these functions In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications NaradaBrokering is designed to provide these functionalities MPI provided such functionalities for all parallel computing 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

NaradaBrokering implements an Event Web Service (Virtual) Queue Web Service 2 Destination Source Matching Filter Routing workflow WSDL Ports Broker Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor Workflow implements message process Routing illustrated by JXTA Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Engineering Issues Addressed by Event / Messaging Service Application level Quality of Service – give audio highest priority Tunnel through firewalls Filter messages to slow (collaborative or real time) clients Hardware multicast is erratically implemented (Event service can dynamically use software multicast) Scaling of software multicast Elegant implementation of Collaboration in a Groove Networks (done better) style Integrate synchronous and asynchronous collaboration 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Features of Event Service I Based on a network of cooperating broker nodes Cluster based architecture allows system to scale to arbitrary size. Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S Synchronous systems: B acts as a real-time router/filterer Messages can be archived and software multicast Asynchronous systems: B acts as an XML database and workflow engine Subscription is in each case, roughly equivalent to a database query Aims at millisecond latencies (MPI aims at microsecond latencies) 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Features of Event Web Service II In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism All Web Services are specified in XML but can be implemented quite differently Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints One can sometimes compile message streams specified in GXOS to MPI or to local method call Event Service must support dynamic heterogeneous protocols 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Features of Event Web Service III The event web service is naturally implemented as a dynamic distributed network Required for fault tolerance and performance A new classroom joins my online lecture A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students Company X sets up a firewall The event service sets up brokers either side of firewall to optimize transport through the firewall Note all message based applications use same message service Web services imply ALL applications are (possibly virtual) message based 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu JMS Compliance Features: JMS clients written while conforming to spec JMS clients are vendor agnostic JMS providers do not interoperate with each other Rationale Providing support for JMS clients within the system. Mature specification with several existing applications Opens NaradaBrokering to these applications Bring NaradaBrokering functionality to JMS systems Distributed solution, load balancing, scaling and failure resiliency. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu Providing JMS support JMS Interactions Supported locally or mapped to corresponding NaradaBrokering calls JMS Interconnection Bridge Operations supported locally or mapped to corresponding NaradaBrokering interactions One bridge instance per connection Maintains list of registered sessions Responsible for routing events to appropriate sessions Support for Creation of different message types e.g. ObjectMessage, BytesMessage etc. Operations that can be invoked on these message types. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu Providing JMS Support Topic NaradaBrokering topics are created as <tag,value> pairs. JMS Topics are generally “/” separated. JMS selector mechanism We augment NaradaBrokering’s topic matching with the selector mechanism implemented by openJMS. JMS subscriptions Mapped to corresponding NaradaBrokering subscription requests. The Narada JMS Event. Encapsulates the entire JMS message as a payload for the event. Matching is done based on the topic name contained in the message. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Towards a distributed JMS solution Features in NaradaBrokering best exploited in distributed settings. Clients of distributed solution to inherit NaradaBrokering features Routing efficiencies, load balancing, scaling etc. Eliminate the Single point of failure model Highly Available System Existing systems should easily be replaced with distributed system 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Broker Locators: Distributed JMS Solution Primary Function Discovery of brokers that clients can connect to Obviates need for client to keep track of broker states within the broker network. Keeps track of Broker additions and removals Changes to network fabric Published Limit on concurrent connections at a broker node Set during broker initializations Active connections at a broker node. Individual brokers notify changes to broker locator. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Broker Locators: Features & Constraints Dynamic Real time Load Balancing Connection requests forked to best available broker. Incorporation of new brokers into solution A newly added broker is among the best brokers to handle a connection request. Slower clients could all be hosted on specific brokers Eliminates broker choking resulting from servicing very slow clients. Constraints Availability (multiple per domain) Should not constitute a bottleneck or single point of failure. Minimal logic Should not affect processing pertaining to any node in the system. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Broker Locator: Decision Metrics IP-address of requesting client Published limit on concurrent connections Number of active connections still possible Availability of broker A simple ping test If broker is not available, remove broker from list of available brokers. Computing capabilities at a broker CPU speed, RAM etc. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Broker Locator: Sequence of Operations Locate valid broker Propagate broker information to client Hostname/IP-address information Port number on which it listens for connections Transport protocol over which it communicates Client then uses info to establish communication channel with broker Done transparently. Clients with multiple connections A client could sometimes have connections to multiple brokers. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu JMS Performance Data SonicMQ (version 3.0) and NaradaBrokering broker Dual CPU (Pentium 3, 1 GHz, 256 MB RAM) machine. 100 subscribers Over 10 different JMSTopicConnections All hosted on a Dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine. Publisher and Measuring Subscriber Hosted on another dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine. Operating System and Run time Environment Linux (version 2.2.16) Java 2 JRE (Java 1.3.1, Blackdown-FCS, mixed-mode) 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Performance Data:Factors Measured Transit Delay No need for clock synchronization and accounting for clock drifts. Standard Deviation in the transit delay for the sample of messages received. System Throughput Measured in terms of rate at which messages are received. Factors measured under varying publish rates message payload sizes. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Performance: Transit Delay & Standard Deviation Lower Payloads & Low Publish Rates 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Performance: Transit Delay & Standard Deviation Higher Payloads & Low Publish Rates 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Performance: Transit Delay & Standard Deviation Lower Payloads & High Publish Rates 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Performance: System Throughput Lower Payloads & Higher Publish Rates 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu Why P2P Core features Resource Sharing & Discovery CPU cycles: SETI@home, Folding@HOME File Sharing: Napster, Gnutella Deployments user driven No dedicated management Management of resources Expose resources & specify security strategy Replicate resources based on demand Dynamic peer groups, fluid memberships Sophisticated search mechanisms Peers respond based on their interpretations Responses do not conform to traditional templates. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu What are the downsides? Interactions are attenuated Localized Fragmented world of multiple P2P subsystems Routing not very sophisticated Inefficient network utilization (Tragedy of Commons) Simple forwarding Peer Traces (to eliminate echoing) Attenuations (to suppress propagation) 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu JXTA Set of protocols to support P2P interactions. Planned bridges to (and from) other P2P systems. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

NaradaBrokering & JXTA Interaction Model Based on proxy model Acts as both Rendezvous peer (JXTA routers) and NaradaBrokering client. No changes to JXTA core or straitjacketing of interactions Change made to Rendezvous layer Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

What we plan to accomplish Better routing and network utilization Bridge isolated P2P subsystems Efficiently locate/replicate shared resources Since interactions are routed through the broker network, brokers can build efficient resource maps. Incorporate GXOS support in NaradaBrokering Then use JXTA search to locate resources stored in the distributed XML database. Provide the notion of reliable interactions (for peers attached to Narada-JXTA proxies) Grid/WebServices generated dynamically when complex tasks are initiated could be managed by peer groups. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu Narada-JXTA Proxy Glean relevant information from JXTA interactions. Peer group advertisements Requests/Responses to be part of peer group. Messages sent to a peer group. Subscribe to relevant topics to ensure delivery Construct corresponding Narada-JXTA event from interactions. 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu Applications Integrated NaradaBrokering-JXTA environment tested under JXTA shell and myJxta (InstantP2P) Plan to integrate myJxta into Anabas with NaradaBrokering managing P2P and middle-tier (JMS) style interactions. JXTA-Jabber interface via NaradaBrokering/Anabas. (Jabber, PDA, Anabas) linkage to be combined with (Jxta, NaradaBrokering, Anabas linkage). 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

PDA Collaboration Event Filter GMS = JMS or Narada 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Event (Message) Service Shared Input Port (Replicated WS) Collaboration Collaboration as a WS Set up Session Web Service F I U O R WS Display WS Viewer Master Web Service F I U O R WS Display WS Viewer Event (Message) Service Other Participants Web Service F I U O R WS Display WS Viewer 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Event (Message) Service Shared Output Port Collaboration Collaboration as a WS Set up Session Web Service Message Interceptor Master Application or Content source WSDL Web Service F I U O R WS Display WS Viewer WS Viewer WS Display Event (Message) Service Other Participants WS Viewer WS Display 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

Web Service Architecture for Audio Video Conferencing 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu XGSP: Introduction Registration Method Session Command Query Session Channel Binding Method Registration Method registration server with its alias name and current location Session Command Method Membership Control Commands, Session Control Commands Query Method discover various properties about the system Session Channel Binding Method bind the RTP channels of a client into the media server 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

http://www.naradabrokering.org/ gcf, spallick@indiana.edu XGSP: Example <SessionDes> <SessionName> PervasiveTech Seminar </SessionName> <SessionID> 1234567 </SessionID> <SessionCreator> Ahmet@indiana.edu </SessionCreator> <SessionInfo> this is a meeting on the XGSP </SessionInfo> <SessionPlace> Lobby Room </SessionPlace> <SessionTime> <StartTime> (EastTime) 10:00AM </StartTime> <EndTime> (EastTime) 12:00AM </EndTime> </SessionTime> <SessionURI> http://grids.ucs.indiana.edu/~ag </SessionURI> <SessionParticipants> <Participant> Wenjun@156.56.103.129 </Participant> <Participant> Hasan@156.56.103.27 </Participant> <Participant> Shrideeper@156.56.103.111 </Participant> </SessionParticipants> <ContactInfo> wewu@indiana.edu </ContactInfo> </SessionDes> 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu

NaradaBrokering Futures Higher Performance – reduce minimum transit time to around one millisecond Substantial operational testing Security – allow Grid (Kerberos/PKI) security mechanisms Support of more protocols with dynamic switching as in JXTA – SOAP, RMI, RTP/UDP Have prototype RTP/UDP support Integration of simple XML database model using JXTA Search to manage distributed archives More formal specification of “native mode” and dynamic instantiation of brokers General Collaborative Web services 11/12/2018 http://www.naradabrokering.org/ gcf, spallick@indiana.edu