1. Application Web Services and Event / Messaging Systems
NeSC Glasgow July
PTLIU Laboratory for Community Grids
Geoffrey Fox, Shrideep Pallickara, Marlon Pierce
Computer Science, Informatics, Physics
Indiana University, Bloomington IN
1/3/2019
uri="

2. Application Portal in a Minute (box)
Systems like Unicore, GPDK, Gridport (HotPage), Gateway, Legion provide “Grid or GCE Shell” interfaces to users (user portals)
Run a job; find its status; manipulate files
Basic UNIX Shell-like capabilities
Application Portals (Problem Solving Environments) are often built on top of “Shell Portals” but this can be quite time confusing
Application Portal = Shell Portal Web Service + Application (factory) Web service
1/3/2019
uri="

3. Application Web service
Application Web Service is ONLY metadata
Application is NOT touched
Application Web service defined by two sets of schema:
First set defines the abstract state of the application
What are my options for invoking myapp?
Dub these to be “abstract descriptors”
Second set defines a specific instance of the application
I want to use myapp with input1.dat on solar.uits.indiana.edu.
Dub these to be “instance descriptors”.
Each descriptor group consists of
Application descriptor schema
Host (resource) descriptor schema
Execution environment (queue or shell) descriptor schema
1/3/2019
uri="

4. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
1/3/2019
uri="

5. Engineering Application WS
Schema wizard: given a Schema, creates (JSP) web page with form to specify XML Instances
Use for application metadata
AntiSchema wizard: given an HTML form, creates a Schema
Captures input parameters of application
Castor converts Schema to Java
Use Python if you prefer!
Apache converts Java into Web Services
Make this in a portlet for use in favorite portal
Being used today in DoD …….. (with and without Globus)
1/3/2019
uri="

6. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
An entry page for the schema wizard.
You provide a java package name (to be used for generated Castor classes) and selected a schema.
1/3/2019
uri="

7. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
The sample schema test.xsd.
We will select this schema for the wizard.
1/3/2019
uri="

8. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Selecting test.xsd automatically generates the JSP interface on next slide.
User can then fill out the form and generate a schema instance (pictured on the right).
Several mapping rules are illustrated:
String and integer elements map to text fields.
Unbounded strings and integers are mapped to editable dropdown menus.
Complex type displays can be built up from lower level display elements.
And so on.
1/3/2019
uri="

9. Different Web Service Organizations
Everything is a resource implemented as a Web Service, whether it be:
back end supercomputers and a petabyte data
Microsoft PowerPoint and this file
Web Services communicate by messages …..
Grids and Peer to Peer (P2P) networks can be integrated by building both in terms of Web Services with different (or in fact sometimes the same) implementations of core services such as registration, discovery, life-cycle, collaboration and event or message transport …..
Gives a Peer-to-Peer Grid
Here we discuss Event or Message Service linking web services together
1/3/2019
uri="

10. Peer to Peer Grid Peers Integrate P2P and Grid/WS Peers
Database
Database
Peers
Resource Facing Web Service Interfaces
Event/ Message Brokers
Integrate P2P
and Grid/WS
Peer to Peer Grid
Web Service Interfaces
Peers
User Facing Web Service Interfaces
A democratic organization
Peer to Peer Grid
1/3/2019
uri="

11. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
XML Skin
XML Skin
Message
Or Event
Based
Inter Connection
Soft ware
Resource
Soft ware
Resource
Data
base
e-Science/Grid/P2P Networks are XML Specified Resources connected by XML specified messages Implementation of resource and connection may or may not be XML
1/3/2019
uri="

12. Role of Event/Message Brokers
We will use events and messages interchangeably
An event is a time stamped message
Our systems are built from clients, servers and “event brokers”
These are logical functions – a given computer can have one or more of these functions
In P2P networks, computers typically multifunction; in Grids one tends to have separate function computers
Event Brokers “just” provide message/event services; servers provide traditional distributed object services as Web services
There are functionalities that only depend on event itself and perhaps the data format; they do not depend on details of application and can be shared among several applications
NaradaBrokering is designed to provide these functionalities
MPI provided such functionalities for all parallel computing
1/3/2019
uri="

13. NaradaBrokering implements an Event Web Service
(Virtual) Queue
Web Service 2
Destination Source Matching
Filter
Routing
workflow
WSDL Ports
Broker
Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor
Workflow implements message process
Routing illustrated by JXTA
Destination-Source matching illustrated by JMS using Publish-Subscribe mechanism
1/3/2019
uri="

14. Engineering Issues Addressed by Event / Messaging Service
Application level Quality of Service – give audio highest priority
Tunnel through firewalls
Filter messages to slow (collaborative or real time) clients
Hardware multicast is erratically implemented (Event service can dynamically use software multicast)
Scaling of software multicast
Elegant implementation of Collaboration in a Groove Networks (done better) style
Integrate synchronous and asynchronous collaboration
1/3/2019
uri="

15. Features of Event Service I
MPI nowadays aims at a microsecond latency
The Event Web Service aims at a millisecond latency
Typical distributed system travel times are many milliseconds (to seconds for Geosynchronous satellites)
Different performance/functionality trade-off
Messages are not sent directly from P to S but rather from P to Broker B and from Broker B to subscriber S
Actually a network of brokers
Synchronous systems: B acts as a real-time router/filterer
Messages can be archived and software multicast
Asynchronous systems: B acts as an XML database and workflow engine
Subscription is in each case, roughly equivalent to a database query
1/3/2019
uri="

16. Features of Event Web Service II
In principle Message brokering can be virtual and compiled away in the same way that WSDL ports can be bound in real time to optimal transport mechanism
All Web Services are specified in XML but can be implemented quite differently
Audio Video Conferencing sessions could be negotiated using SOAP (raw XML) messages and agree to use certain video codecs transmitted by UDP/RTP
There is a collection of XML Schema – call it GXOS – specifying event service and requirements of message streams and their endpoints
One can sometimes compile message streams specified in GXOS to MPI or to local method call
Event Service must support dynamic heterogeneous protocols
1/3/2019
uri="

17. Features of Event Web Service III
The event web service is naturally implemented as a dynamic distributed network
Required for fault tolerance and performance
A new classroom joins my online lecture
A broker is created to handle students – multicast locally my messages to classroom; handle with high performance local messages between students
Company X sets up a firewall
The event service sets up brokers either side of firewall to optimize transport through the firewall
Note all message based applications use same message service
Web services imply ALL applications are (possibly virtual) message based
1/3/2019
uri="

18. Single Server P2P Illusion
Traditional Collaboration Architecture e.g. commercial WebEx
Data
base
Collaboration Server
1/3/2019
uri="

19. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
Narada Broker Network
(P2P) Community
For message/events service
Broker
Broker
(P2P) Community
Resource
Broker
Broker
Broker
Data
base
(P2P) Community
Software multicast
Broker
(P2P) Community
1/3/2019
uri="

20. NaradaBrokering and JMS (Java Message Service)
Low Rate; Small Messages
(commercial JMS)
1/3/2019
uri="

21. uri="http://www.naradabrokering.org" email="gcf@indiana.edu"
JXTA just got slower
Client  JXTA  JXTA  Client
Client  JXTA  Narada  JXTA  Client
Client  JXTA  JXTA  Client multicast
Narada
Client
Pure Narada 2 hops
Client
Narada
1/3/2019
uri="

22. Event (Message) Service
Shared Input Port (Replicated WS) Collaboration
Collaboration as a WS Set up Session
Web Service F I U O R
WS Display
WS Viewer
Master
Web Service F I U O R
WS Display
WS Viewer
Event (Message) Service
Other Participants
Web Service F I U O R
WS Display
WS Viewer
1/3/2019
uri="

23. Event (Message) Service
Shared Output Port Collaboration
Collaboration as a WS Set up Session
Web Service Message
Interceptor
Master
Application or Content source
WSDL
Web Service F I U O R
WS Display
WS Viewer
WS Viewer
WS Display
Event (Message) Service
Other Participants
WS Viewer
WS Display
1/3/2019
uri="

24. NaradaBrokering Futures
Higher Performance – reduce minimum transit time to around one millisecond
Substantial operational testing
Security – allow Grid (Kerberos/PKI) security mechanisms
Support of more protocols with dynamic switching as in JXTA – SOAP, RMI, RTP/UDP
Integration of simple XML database model using JXTA Search to manage distributed archives
More formal specification of “native mode” and dynamic instantiation of brokers
General Collaborative Web services
1/3/2019
uri="