1. NaradaBrokering – Building P2P Grids
Access Grid meeting – August 21st, 2002.
PTLIU Laboratory for Community Grids
Geoffrey Fox, Shrideep Pallickara
Computer Science, Informatics, Physics
Indiana University, Bloomington IN
11/12/2018
uri="

2. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
JXTA and Grids
JXTA and Grid architectures can be implemented as Web Services interacting with (XML-based) messages
We built a “Grid Messaging System” NaradaBrokering that implements generalized publish-subscribe mechanism in a network of “brokers/rendezvous peers”
NaradaBrokering can replace Java Message Service – Grid-like system
Used to run our synchronous collaboration system Garnet supporting shared display, text chats, Jabber instant messenger ….
NaradaBrokering is integrated with JXTA (as a proxy to rendezvous peers) and can provide reliable messaging between peer groups (and inside?)
We are building Collaboration (shared application and audio-video conferencing) as a Web Service
XGSP (XML General Session Protocol) is meant to include H323 SIP and (later) JXTA sessions (peer groups)
JXTA will be able to invoke Access Grid, Polycom, Shared Display sessions
11/12/2018
gcf,

3. Different Web Service Organizations
Everything is a resource implemented as a Web Service, whether it be:
back end supercomputers and a petabyte data
Microsoft PowerPoint and this file
Web Services communicate by messages …..
Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport …..
Gives a Peer-to-Peer Grid
NaradaBrokering is an example of Event or Message Service linking web services together
11/12/2018
gcf,

4. Peer to Peer Grid Peers Integrate P2P and Grid/WS Peers
Database
Database
Peers
Resource Facing Web Service Interfaces
Event/ Message Brokers
Integrate P2P
and Grid/WS
Peer to Peer Grid
Web Service Interfaces
Peers
User Facing Web Service Interfaces
A democratic organization
Peer to Peer Grid
11/12/2018
gcf,

5. Role of Event/Message Brokers
We will use events and messages interchangeably
An event is a time stamped message
Our systems are built from clients, servers and “event brokers”
These are logical functions – a given computer can have one or more of these functions
In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers
Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services
There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications
NaradaBrokering is designed to provide these functionalities
MPI provided such functionalities for all parallel computing
11/12/2018
gcf,

6. NaradaBrokering implements an Event Web Service
(Virtual) Queue
Web Service 2
Destination Source Matching
Filter
Routing
workflow
WSDL Ports
Broker
Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor
Workflow implements message process
Routing illustrated by JXTA
Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism
11/12/2018
gcf,

7. Engineering Issues Addressed by Event / Messaging Service
Application level Quality of Service – give audio highest priority
Tunnel through firewalls
Filter messages to slow (collaborative or real time) clients
Hardware multicast is erratically implemented (Event service can dynamically use software multicast)
Scaling of software multicast
Elegant implementation of Collaboration in a Groove Networks (done better) style
Integrate synchronous and asynchronous collaboration
11/12/2018
gcf,

8. Features of Event Service I
Based on a network of cooperating broker nodes
Cluster based architecture allows system to scale to arbitrary size.
Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S
Synchronous systems: B acts as a real-time router/filterer
Messages can be archived and software multicast
Asynchronous systems: B acts as an XML database and workflow engine
Subscription is in each case, roughly equivalent to a database query
Aims at millisecond latencies (MPI aims at microsecond latencies)
11/12/2018
gcf,

9. Features of Event Web Service II
In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism
All Web Services are specified in XML but can be implemented quite differently
Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP
There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints
One can sometimes compile message streams specified in GXOS to MPI or to local method call
Event Service must support dynamic heterogeneous protocols
11/12/2018
gcf,

10. Features of Event Web Service III
The event web service is naturally implemented as a dynamic distributed network
Required for fault tolerance and performance
A new classroom joins my online lecture
A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students
Company X sets up a firewall
The event service sets up brokers either side of firewall to optimize transport through the firewall
Note all message based applications use same message service
Web services imply ALL applications are (possibly virtual) message based
11/12/2018
gcf,

11. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
JMS Compliance
Features:
JMS clients written while conforming to spec
JMS clients are vendor agnostic
JMS providers do not interoperate with each other
Rationale
Providing support for JMS clients within the system.
Mature specification with several existing applications
Opens NaradaBrokering to these applications
Bring NaradaBrokering functionality to JMS systems
Distributed solution, load balancing, scaling and failure resiliency.
11/12/2018
gcf,

12. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
Providing JMS support
JMS Interactions
Supported locally or mapped to corresponding NaradaBrokering calls
JMS Interconnection Bridge
Operations supported locally or mapped to corresponding NaradaBrokering interactions
One bridge instance per connection
Maintains list of registered sessions
Responsible for routing events to appropriate sessions
Support for
Creation of different message types e.g. ObjectMessage, BytesMessage etc.
Operations that can be invoked on these message types.
11/12/2018
gcf,

13. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
Providing JMS Support
Topic
NaradaBrokering topics are created as <tag,value> pairs.
JMS Topics are generally “/” separated.
JMS selector mechanism
We augment NaradaBrokering’s topic matching with the selector mechanism implemented by openJMS.
JMS subscriptions
Mapped to corresponding NaradaBrokering subscription requests.
The Narada JMS Event.
Encapsulates the entire JMS message as a payload for the event.
Matching is done based on the topic name contained in the message.
11/12/2018
gcf,

14. Towards a distributed JMS solution
Features in NaradaBrokering best exploited in distributed settings.
Clients of distributed solution to inherit NaradaBrokering features
Routing efficiencies, load balancing, scaling etc.
Eliminate the Single point of failure model
Highly Available System
Existing systems should easily be replaced with distributed system
11/12/2018
gcf,

15. Broker Locators: Distributed JMS Solution
Primary Function
Discovery of brokers that clients can connect to
Obviates need for client to keep track of broker states within the broker network.
Keeps track of
Broker additions and removals
Changes to network fabric
Published Limit on concurrent connections at a broker node
Set during broker initializations
Active connections at a broker node.
Individual brokers notify changes to broker locator.
11/12/2018
gcf,

16. Broker Locators: Features & Constraints
Dynamic Real time Load Balancing
Connection requests forked to best available broker.
Incorporation of new brokers into solution
A newly added broker is among the best brokers to handle a connection request.
Slower clients could all be hosted on specific brokers
Eliminates broker choking resulting from servicing very slow clients.
Constraints
Availability (multiple per domain)
Should not constitute a bottleneck or single point of failure.
Minimal logic
Should not affect processing pertaining to any node in the system.
11/12/2018
gcf,

17. Broker Locator: Decision Metrics
IP-address of requesting client
Published limit on concurrent connections
Number of active connections still possible
Availability of broker
A simple ping test
If broker is not available, remove broker from list of available brokers.
Computing capabilities at a broker
CPU speed, RAM etc.
11/12/2018
gcf,

18. Broker Locator: Sequence of Operations
Locate valid broker
Propagate broker information to client
Hostname/IP-address information
Port number on which it listens for connections
Transport protocol over which it communicates
Client then uses info to establish communication channel with broker
Done transparently.
Clients with multiple connections
A client could sometimes have connections to multiple brokers.
11/12/2018
gcf,

19. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
JMS Performance Data
SonicMQ (version 3.0) and NaradaBrokering broker
Dual CPU (Pentium 3, 1 GHz, 256 MB RAM) machine.
100 subscribers
Over 10 different JMSTopicConnections
All hosted on a Dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine.
Publisher and Measuring Subscriber
Hosted on another dual CPU (Pentium 3, 866 MHz, 256 MB RAM) machine.
Operating System and Run time Environment
Linux (version )
Java 2 JRE (Java 1.3.1, Blackdown-FCS, mixed-mode)
11/12/2018
gcf,

20. Performance Data:Factors Measured
Transit Delay
No need for clock synchronization and accounting for clock drifts.
Standard Deviation in the transit delay for the sample of messages received.
System Throughput
Measured in terms of rate at which messages are received.
Factors measured under varying
publish rates
message payload sizes.
11/12/2018
gcf,

21. Performance: Transit Delay & Standard Deviation
Lower Payloads & Low Publish Rates
11/12/2018
gcf,

22. Performance: Transit Delay & Standard Deviation
Higher Payloads & Low Publish Rates
11/12/2018
gcf,

23. Performance: Transit Delay & Standard Deviation
Lower Payloads & High Publish Rates
11/12/2018
gcf,

24. Performance: System Throughput
Lower Payloads & Higher Publish Rates
11/12/2018
gcf,

25. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
Why P2P
Core features
Resource Sharing & Discovery
CPU cycles:
File Sharing: Napster, Gnutella
Deployments user driven
No dedicated management
Management of resources
Expose resources & specify security strategy
Replicate resources based on demand
Dynamic peer groups, fluid memberships
Sophisticated search mechanisms
Peers respond based on their interpretations
Responses do not conform to traditional templates.
11/12/2018
gcf,

26. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
What are the downsides?
Interactions are attenuated
Localized
Fragmented world of multiple P2P subsystems
Routing not very sophisticated
Inefficient network utilization (Tragedy of Commons)
Simple forwarding
Peer Traces (to eliminate echoing)
Attenuations (to suppress propagation)
11/12/2018
gcf,

27. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
JXTA
Set of protocols to support P2P interactions.
Planned bridges to (and from) other P2P systems.
11/12/2018
gcf,

28. NaradaBrokering & JXTA Interaction Model
Based on proxy model
Acts as both Rendezvous peer (JXTA routers) and NaradaBrokering client.
No changes to JXTA core or straitjacketing of interactions
Change made to Rendezvous layer
Peers are not aware that they interact with a Narada-JXTA proxy or Rendezvous peer.
11/12/2018
gcf,

29. What we plan to accomplish
Better routing and network utilization
Bridge isolated P2P subsystems
Efficiently locate/replicate shared resources
Since interactions are routed through the broker network, brokers can build efficient resource maps.
Incorporate GXOS support in NaradaBrokering
Then use JXTA search to locate resources stored in the distributed XML database.
Provide the notion of reliable interactions (for peers attached to Narada-JXTA proxies)
Grid/WebServices generated dynamically when complex tasks are initiated could be managed by peer groups.
11/12/2018
gcf,

30. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
Narada-JXTA Proxy
Glean relevant information from JXTA interactions.
Peer group advertisements
Requests/Responses to be part of peer group.
Messages sent to a peer group.
Subscribe to relevant topics to ensure delivery
Construct corresponding Narada-JXTA event from interactions.
11/12/2018
gcf,

31. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
Applications
Integrated NaradaBrokering-JXTA environment tested under JXTA shell and myJxta (InstantP2P)
Plan to integrate myJxta into Anabas with NaradaBrokering managing P2P and middle-tier (JMS) style interactions.
JXTA-Jabber interface via NaradaBrokering/Anabas.
(Jabber, PDA, Anabas) linkage to be combined with (Jxta, NaradaBrokering, Anabas linkage).
11/12/2018
gcf,

32. PDA Collaboration Event Filter
GMS = JMS or Narada
11/12/2018
gcf,

33. Event (Message) Service
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS Set up Session
Web Service F I U O R
WS Display
WS Viewer
Master
Web Service F I U O R
WS Display
WS Viewer
Event (Message) Service
Other Participants
Web Service F I U O R
WS Display
WS Viewer
11/12/2018
gcf,

34. Event (Message) Service
Shared Output Port Collaboration
Collaboration as a WS Set up Session
Web Service Message
Interceptor
Master
Application or Content source
WSDL
Web Service F I U O R
WS Display
WS Viewer
WS Viewer
WS Display
Event (Message) Service
Other Participants
WS Viewer
WS Display
11/12/2018
gcf,

35. Web Service Architecture for Audio Video Conferencing
11/12/2018
gcf,

36. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
XGSP: Introduction
Registration
Method
Session Command
Query
Session Channel
Binding Method
Registration Method
registration server with its alias name and current location
Session Command Method
Membership Control Commands, Session Control Commands
Query Method
discover various properties about the system
Session Channel Binding Method
bind the RTP channels of a client into the media server
11/12/2018
gcf,

37. http://www.naradabrokering.org/ gcf, spallick@indiana.edu
XGSP: Example
<SessionDes>
<SessionName> PervasiveTech Seminar </SessionName>
<SessionID> </SessionID>
<SessionCreator> </SessionCreator>
<SessionInfo> this is a meeting on the XGSP </SessionInfo>
<SessionPlace> Lobby Room </SessionPlace>
<SessionTime>
<StartTime> (EastTime) 10:00AM </StartTime>
<EndTime> (EastTime) 12:00AM </EndTime>
</SessionTime>
<SessionURI> </SessionURI>
<SessionParticipants>
<Participant> </Participant>
<Participant> </Participant>
<Participant> </Participant>
</SessionParticipants>
<ContactInfo> </ContactInfo>
</SessionDes>
11/12/2018
gcf,

38. NaradaBrokering Futures
Higher Performance – reduce minimum transit time to around one millisecond
Substantial operational testing
Security – allow Grid (Kerberos/PKI) security mechanisms
Support of more protocols with dynamic switching as in JXTA – SOAP, RMI, RTP/UDP
Have prototype RTP/UDP support
Integration of simple XML database model using JXTA Search to manage distributed archives
More formal specification of “native mode” and dynamic instantiation of brokers
General Collaborative Web services
11/12/2018
gcf,