1. Performance of a possible Grid Message Infrastructure
Edinburgh e-Science Centre December
PTLIU Laboratory for Community Grids
Geoffrey Fox, Shrideep Pallickara
Computer Science, Informatics, Physics
Indiana University, Bloomington IN 47404
9/17/2018
uri="

2. Some Basic Observations/Goals
Grids manage and share asynchronous resources in a rather centralized fashion
Peer-to-peer networks are “just like” Grids with different implementations of services like registration and look-up
Web Services interact with messages and Everything will be a Web Service
Microsoft Office will be a WS?
Computers are fast and getting faster. One can afford many strategies that used to be unrealistic
Software/Application-level message processing/routing sensitive to performance information
Uniform messaging framework for Middle-tier, “High performance layer”, Multimedia (audio-video) etc.
9/17/2018
uri="

3. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Deduction?
Grids or P2P Networks consist of a sea of message-based Services
Services inject and extract messages whose transport and manipulation is support by a logically distinct “MessageGrid” of brokers/routers supplying “Grid Message Layer”
The message brokers support adaptive routing, filtering, workflow
They have publish/subscribe queued connection semantics
They separate logical and actual transport
They form a federated XML database and support asynchronous collaboration
They process real-time messages in about a millisecond and support synchronous collaboration
NaradaBrokering is a prototype of this idea
9/17/2018
uri="

4. Peer to Peer Grid Peers Peers
Database
Database
Peers
Service Facing Web Service Interfaces
Event/ Message Brokers
Peer to Peer Grid
Peers
User Facing Web Service Interfaces
Peer to Peer Grid of Resources supported by internal “MessageGrid” of message or event brokers
Events are “just” Time-stamped messages
9/17/2018
uri="

5. Event (Message) Service
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS Set up Session with XGSP
Web Service F I U O R
WS Display
WS Viewer
Master
Web Service F I U O R
WS Display
WS Viewer
Event (Message) Service
Other Participants
Web Service F I U O R
WS Display
WS Viewer
9/17/2018
uri="

6. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
NaradaBrokering
Based on a network of cooperating broker nodes
Cluster based architecture allows system to scale to arbitrary size
Originally designed to provide uniform software multicast to support real-time collaboration linked to publish-subscribe for asynchronous systems.
Now has four major core functions
Message transport (based on performance measurement) in heterogeneous multi-link fashion
General publish-subscribe including JMS & JXTA and support for RTP-based audio/video conferencing
Filtering for heterogeneous clients
Federation of multiple instances of Grid services
9/17/2018
uri="

7. Role of Event/Message Brokers
We will use events and messages interchangeably
An event is a time stamped message
Our systems are built from clients, servers and “event brokers”
These are logical functions – a given computer can have one or more of these functions
In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers
Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services
There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications
NaradaBrokering is designed to provide these functionalities
MPI provided such functionalities for all parallel computing
9/17/2018
uri="

8. Engineering Issues Addressed by Event / Messaging Service
Application level Quality of Service – e.g. give audio highest priority
Tunnel through firewalls & proxies
Filter messages to slow (collaborative/real-time) clients
Choose Hardware or Software multicast
Scaling of software multicast
Efficient calculation of destinations and routes.
Integrate synchronous and asynchronous collaboration with same messaging, control, archiving for all functions
Supports local broker accesses
Transparently replace single server JMS systems with a distributed solution.
Provides reliable inter-peer group messaging for JXTA
9/17/2018
uri="

9. Features of Event Service
MPI nowadays aims at a microsecond latency
The Event Web Service aims at a millisecond (computer) latency
Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites)
Different performance/functionality trade-off
Messages are not sent directly from P to S but rather from P to Broker B and from Broker B (via other brokers) to subscriber S.
Synchronous systems: B acts as a real-time router/filterer
Messages can be archived and software multicast
Asynchronous systems: B acts as a database & workflow engine
Subscription is in each case, roughly equivalent to a database query
Company X sets up a firewall
The event service sets up brokers either side of firewall to optimize transport through the firewall.
9/17/2018
uri="

10. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Narada Broker Network
(P2P) Community
For message/events service
Broker
Broker
(P2P) Community
Resource
Hypercube topology
for brokers?
Tree for distance education
with teacher at root
Broker
Broker
Broker
Data
base
(P2P) Community
Software multicast
Broker
(P2P) Community
9/17/2018
uri="

11. Performance in Message-based Architecture I
Satellite UDP
Firewall HTTP
Dial-up Filter A B1
Hand-Held Protocol
Fast Link
Software Multicast B2
Useful analogies with transportation grids and parallel computing
In traveling from cities A to B (say 3 separate passengers), one chooses between and changes transport mechanism at waystations to optimize cost, time, comfort, scenic beauty …
Waystations are now NB brokers where one chooses transport protocol
Able to choose between car, type of car, plane, train etc
Able to dynamically create waystations to cope with problems and acts as hubs for multicast messages
Knows about traffic jams and can assign the “HOV lane” B3
9/17/2018
uri="

12. Performance in Message-based Architecture II
Application level QoS – can optimize among managed streams (audio versus video) using performance subsystem
This is just a variant of the NP complete load balancing problem of parallel computing where all reasonable heuristics worked
Load-balance in Space-time (strings) not just Space (particles)
“Performance” needs to measured carefully as includes QoS
I delayed shared application update to ensure audio quality and filtered image to lower resolution
So “application” has changed to satisfy performance constraints
9/17/2018
uri="

13. NaradaBrokering Communication
Applications interface to NaradaBrokering through UserChannels which NB constructs as a set of links between NB Broker waystations which may need to be dynamically instantiated
UserChannels have publish/subscribe semantics with XML topics
Links implement a single conventional “data” protocol.
Interface to add new transport protocols within the Framework
Administrative channel negotiates the best available communication protocol for each link
Different links can have different underlying transport implementations
Implementations in the current release include support for TCP,UDP, Multicast, SSL and RTP. HTTP, HTTPS support will be available in Feb 2003 release.
Supports communication through proxies such as iPlanet, Netscape and Apache.
Supports communication through firewalls such as Microsoft ISA, Checkpoint.
9/17/2018
uri="

14. Types of Performance Measurement
“Core Performance”: Equivalent of node Clock Cycle and node to node MPI Ping/Bandwidth measurements
Cosmic single number which does not satisfy anybody but nevertheless dominates discussion: Equivalent to bisection bandwidth or LINPACK
Detailed performance numbers for particular Grid applications: e.g. Time to assimilate global weather data and simulate hurricane
We will present “core performance” numbers for NaradaBrokering running in different modes
Will not discuss “Web Service” performance – rather performance of Message substrate
9/17/2018
uri="

15. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Note on Optimization
Note in parallel computing, couldn’t do much dynamic optimization as aiming at microsecond latency
Natural to use hardware routing
In Grid, time scales are different
100 millisecond quite normal network latency
30 millisecond typical packet time sensitivity (this is one audio or video frame) but even here can buffer frames on client (conferencing to streaming)
1 millisecond is time for a Java server to “think”
Jitter in latency (transit time) due to routing, processing (in NB) or packet loss recovery is important property
Grid needs and can tolerate significant dynamic optimization
9/17/2018
uri="

16. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Sender/receiver/broker - (Pentium-3, 1 GHz, 256 MB RAM). 100 Mbps LAN. JDK-1.3, Red Hat Linux 7.3
9/17/2018
uri="

17. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Sender/receiver/broker - (Pentium-3, 1 GHz, 256 MB RAM). 100 Mbps LAN. JDK-1.3, Red Hat Linux 7.3
9/17/2018
uri="

18. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

19. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

20. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

21. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
H.263 video file Average bit-rate of 600Kbps. Frame-rate of 30 frames/sec
Jitter J = J + (|D(i-1, i)| - J)/16
Jitter J is defined by the RTP RFC as the mean deviation (smoothed absolute value) of the difference D in packet spacing at the receiver compared to the sender for a pair of packets
Jitter represents the jitter value up until that point in the sample.
9/17/2018
uri="

22. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
SSL Overhead I
The test program simulated a SSL client transferring a 5 MB chunk of data to an SSL server.
Test Environment
800 MHz PIII
JDK 1.3, JSSE 1.0.3
Numbers are approximate over 5 iterations.
SSL Results
SSL Socket Creation: ~250 ms
SSL Handshaking: ~500 ms
Transfer of 5 MB: ~2050 ms
9/17/2018
uri="

23. SSL Overhead (ISA HTTPS Proxy)
Test Environment was as before except going through Microsoft ISA’s HTTPS proxy in between the client and the server. The proxy machine’s configuration was:   
Pentium 900 MHz, 512 MB RAM.
Microsoft ISA proxy Enterprise Edition allow HTTPS protocol through
Proxy => Server network bandwidth 10 Mbps.
SSL Socket Creation: ~250 ms
SSL Handshaking (after 1st one):~400 ms
Transfer of 50 MB: ~50 seconds (1 MB / second)
Proxy CPU Usage: ~7% on the average
Client CPU Usage: ~20% on the average
9/17/2018
uri="

24. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
JMS Performance: Transit Delay and Std Deviation (NaradaBrokering & SonicMQ)
Transit Delay
Standard Deviation
NaradaBrokering is Java Message Service (JMS) compliant
Applications ported include
Anabas – JMS based distance education system.
OKC – Online Knowledge Center project at IU
9/17/2018
uri="

25. NaradaBrokering and JXTA
Narada-JXTA provides JXTA guaranteed long distance delivery
Request/Response
Present if targeted at Particular peer
Narada JXTA Event
9/17/2018
uri="

26. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

27. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

28. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
9/17/2018
uri="

29. Narada Performance Web Service
Performance measurements are used by Links in
Reconfiguring Connectivity between nodes
Deciding underlying transport protocol
Determining possible filtering
Each node determines performance of links of which it is endpoint
Individual node web services are aggregated as another Web Service
Probably should replace by a more
sophisticated measurement package
Factors measured include
Transit delays, bandwidth, Jitter, Receiving rates.
Performance measurements are
Spaced out at increasing intervals for healthy channels.
Factors selectively measured for unhealthy channels.
No repeated measurements of bandwidth for example.
Injected into Narada network as XML events
Administrative Interface
9/17/2018
uri="

30. Narada Link Firewall Architecture
Control Channel (TCP)
Specifics tunnel destination, parameters.
[ SOAP port 80 ]
Config
Specifics default tunnel destination, parameters.
Non-Firewall
Proxy
CTL
SSL Tunnel Server Proxy
UDP
TCP
SSL Tunnel Client Proxy
SSL
Lib
API
TCP
UDP
Firewall
Proxy
Fake SSL
Impl.
JSSE
Impl.
WinINET
Impl.
Proxy Detection API
Required for MS Authentication support.
Text Config
WinINET Detection
Required for Proxy location detection
Narada Link Firewall Architecture
9/17/2018
uri="

31. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Start
UDP
Works
Stream Media
Types
Connection
Complete
Doesn’t Work
Start
TCP
Works
Reliable Data
Stream
Doesn’t Work
Windows ?
WinINET
Try SSL first then HTTP
Works
NaradaBrokering Link Transport
Firewall Heuristic
Doesn’t Work
Try SSL Over
HTTPS
Proxy
Does HTTPS
Proxy Exist
Works
Yes
Doesn’t Work
Try HTTP
Over HTTP
Proxy
Does HTTP
Proxy Exist
Works
Yes
Doesn’t Work
“Fake” SSL Over
Direct
Try SSL Over
Direct
Try HTTP
Direct
Works
Doesn’t
Work
Doesn’t
Work
Works
9/17/2018
uri="

32. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Futures
Redo performance with insight of this workshop – we desperately need a distributed testbed to support scaling tests
Test Scaling with construction of dynamic NB broker network (what is best topology? Hypercube like?)
Understand clients per broker for various applications including large multimedia sessions
Complete transport/performance infrastructure
Develop nifty “load balancing heuristics”
Implement Web Service message-based Security
Understand better how to implement distributed XML database
Develop NB to support federation of Grid Services
Generalize Filtering mechanism
Download from
9/17/2018
uri="

33. Architecture of Message Layer
Need to optimize not only routing of particular messages but classic publish/subscribe problem of integrating different requests with related topics (subscribe to sports/basketball/lakers and sports)
Related to Akamai, AOL … caching and Server optimization problem
Hypercube of NB Brokers (logical
not physical)
N≈100 for
Distance
Education
Scale to billions of grid nodes?
1-> N Grid Nodes
9/17/2018
uri="

34. NaradaBrokering as an XML database
NB is inevitably a distributed XML database supporting real-time update and access (JMS uses SQL like syntax)
Performance data
Event Logging
Published “properties” of publish/subscribe messages
Publish as XML topic; subscribe using XQuery?
We use NB as a simple XML database for News groups and other “XML nuggets” see
XML Instance==Message; what is difference between a message and a database record except performance but Moore’s law is rewriting rules here
Subscribe a real database (Oracle) to all topics for reliable storage
9/17/2018
uri="

35. Federation of Services
If you have a service – Notification (as with Grid or JXTA advertisements), Search, Scheduling, Audio-Video conferencing ….
With a standard which client and server components
Then can build a “server” that services all clients
Alternatively can hierarchically consider collection of existing Server-client domains
IBM or Globus OGSA islands
Sun Grid Engine Schedulers
Polycom/Access Grid A/V sessions
Enterprise Search Engines
Federation links islands together
JXTA Search has XML specified federation approach – will try to include and generalize as a NaradaBrokering federation framework
DoD High Level Architecture HLA does this for simulation
NB Hub
9/17/2018
uri="