1. Integration of Portals and Web Services
NCSA Expedition Meeting March
PTLIU Laboratory for Community Grids
Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn
Computer Science, Informatics, Physics
Indiana University, Bloomington IN
11/14/2018
uri="

2. What is a Web Service I
A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in XML (WSDL)
In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever
Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but
The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python
Web Services separate the meaning of a port (message) interface from its implementation
Enhances/Enables Re-usable component model of ANY electronic resource
11/14/2018
uri="

3. What is a Web Service II
Web Services have important implication that ALL interfaces are XML messages based. In contrast
Most Windows programs have interfaces defined as interrupts due to user inputs
Most software have interfaces defined as methods which might be implemented as a message but this is often NOT explicit
Security
Catalog
Payment Credit Card
Warehouse
shipping
WSDL interfaces
11/14/2018
uri="

4. What is a Web Service III
“Everything electronic” is a resource
Computers; Programs; People
Data (from sensors to this presentation to to databases)
“Everything electronic” is a distributed object
All resources have interfaces which are defined in XML for both properties (data-structure) and methods (service, function, subroutine) (Resources are Services)
We can assume that a data-structure property has getproperty() and setproperty(value) methods to act as interface
All resources are linked by messages with structure, which must be specifiable in XML
All resources have a URI such as unique://a/b/c …….
11/14/2018
uri="

5. Classic Grid Architecture
Resources
Database
Database
Content Access
Composition
Middle Tier Brokers Service Providers
Netsolve
Security
Collaboration
Computing
Middle Tier becomes Web Services
Clients
Users and Devices
11/14/2018
uri="

6. Raw Resources Clients Raw Data Raw Data Render to XML Display Format
(Virtual) XML Data Interface
Web Service (WS) WS WS WS
etc.
XML WS to WS Interfaces WS
(Virtual) XML Knowledge (User) Interface
Render to XML Display Format
(Virtual) XML Rendering Interface
Clients
11/14/2018
uri="

7. WSDL Abstractions
WSDL abstracts a program as an entity that does something given one or more inputs with its results defined by streams on one or more outputs.
Functions are defined by method name and parameters methodname(parm1,parm2, … parmN)
Where parameters are “Input” “Output” or both
In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods
Instead of a function call, the “calling routine” sends an XML message to the Web Service specifying methodname and values of the parameters
Note name of function is just another parameter
11/14/2018
uri="

8. Details of WSDL Protocol Stack
UDDI finds where programs are
remote( (distributed) programs are just Web Services
WSFL links programs together (under revision?)
WSDL defines interface (methods, parameters, data formats)
SOAP defines structure of message including serialization of information
HTTP is negotiation/transport protocol
TCP/IP is layers 3-4 of OSI
Physical Network is layer 1 of OSI
UDDI or WSIL
WSFL
WSDL
SOAP or RMI
HTTP or SMTP or IIOP or RMTP
TCP/IP
Physical Network
11/14/2018
uri="

9. Event/ Message Brokers
Database
Database
JXTA
Web Service Interfaces
Event/ Message Brokers
Integrate P2P
and Grid/WS
Peer to Peer Grid
Web Service Interfaces
JXTA
Peer to Peer Grid
11/14/2018
uri="

10. Examples of Web Services I
OGSA (Open Grid Service Architecture)
Integrate Web Service and Grid Concepts and allows Globus to be implemented as Web Services
Audio-Video Conferencing as a Web Service
Integrates H323, SIP, JXTA (etc.) protocols by mapping to single XML Interface
Provides VRVS reflector model from Messaging Web Service
Messaging or Event Web Service provides intelligent routing and buffering of messages
Performance as Web Service (yesterday)
Allow access to distributed monitoring data, analysis, models, and final benchmarks with interoperable XML interfaces
11/14/2018
uri="

11. Examples of Web Services II
Computing as a Web service
More on this today – job submittal, status, composition, data services, visualization etc.
Education as a Web Service
One of easiest to do as object standards well defined (IMS) and little performance issues
Grading, Homework submission, registration, assessment etc.
Universal Access and Web Services
As Web Services allow multiple implementation of a particular interface, one can adjust to needs of particular clients (PDA v. versus, impaired sight etc.)
Can build custom implementations of certain web services for particular communities but re-use others
Collaborative Web Services
As interfaces all message based, relatively easy to share Web Services
11/14/2018
uri="

12. Portals and Web Services
Web Services allow us to build a component model (see CCA) for resources.
Each resource naturally has a user interface (which might be customized for user)
Web Service <--> Portlet
Natural to use a component model for portal building displayed web page from collection of portlets
So can customize each portlet and customize which portlets you want
Apache Jetspeed seems good open source technology supporting this model
JSP model is better than say a client-side Java integration in that also message based so this is “Portal as a Web Service”
11/14/2018
uri="

13. Component –based Portal
Web Services provide a component model for the middleware (see large “common component architecture” effort in Dept. of Energy)
Should match each WSDL component with a corresponding user interface component
Thus one “must use” a component model for the portal with again an XML specification (portalML) of portal component
11/14/2018
uri="

14. Meta-Data and Web Services
Enriching resources with meta-data is critical idea
Enables one to identify and link resources around the globe
Allows one to find out “meaning” of a Web service not just syntax of interface
Semantic Grid implies linkage of Grid/Web services enabled by meta-data leading to “digital brilliance” phase transition
We can experiment with Semantic Web techniques for specifying meta-data RDF DAML OIL
These encompass both straightforward enriched data as well as Artificial Intelligence assertions
11/14/2018
uri="

15. RDF in a Nutshell Predicate Subject Object
The essence of RDF is a statement which has a subject and predicate and an object
The “subject” has the property “predicate” with value “object”
The Web Page has the title “RDF Model and Syntax”
Mary had a husband called John
Statements can specify metadata but more generally any facts are expressible in this way
One can associate multiple objects with a subject and so build up a general node of a tree with parent as subject and objects as children.
DAML and OIL extend RDF with more powerful relations and ontologies
11/14/2018
uri="

16. An Example of RDF and Dublin Core
<rdf:RDF xmlns:rdf=“ xmlns:dc=“
<rdf:Description about=“
<dc:Title>D-Lib Program - Research in Digital Libraries</dc:Title> <dc:Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing. </dc:Description> <dc:Publisher>Corporation For National Research Initiatives</dc:Publisher> <dc:Date> </dc:Date>
<dc:Subject>
<rdf:Bag> <rdf:li> Research; statistical methods</rdf:li> <rdf:li> Education, research, related topics</rdf:li> <rdf:li> Library use Studies </rdf:li> </rdf:Bag>
</dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type>
<dc:Format>text/html</dc:Format>
<dc:Language>en</dc:Language>
</rdf:Description> </rdf:RDF>
11/14/2018
uri="

17. What could one do?
Get applications together and decide on set of common web services
Service and methods within services
At “Grid Computing” and “Application” level
Implement generalized meta-data service at application level – generate, access
Role of RDF DAML OIL IMS etc.
Define both middle tier (WS) and user interface (portal) components
Agree on pilot projects demonstrating both interoperable WS/Portal components and complete applications built from WS/Portal components
Feed lessons into W3C and Grid processes
11/14/2018
uri="

18. Collaborative Web Services
First note that there are two distinct concepts
Collaboration as a Web Service
Such as “Audio-Video Conferencing” as a Web Service or “Text Chat as a Web Service”
Collaborative Web Services
Here we view a Web Service as specifying a (distributed) object and wish to share an object
Object could be a Web page, a Job status form, a scientific visualization, a PowerPoint slide etc. (not all of these are Web Services but all should be)
11/14/2018
uri="

19. Why Web Services for Collaboration
Well everything is meant to be a Web Service but also:
Web Services are MUCH EASIER to make collaborative than other objects because all input and output is defined by uniform XML messages
You need to teach your message service about collaboration!
Note local applications are NOT Web Services – input is things like “user mouse click” represented by “method events” (UI program interrupts) not “XML message events”
The elegance of collaborative web services suggest that it could be easiest to make object X (such as PowerPoint or SVG) collaborative not by traditional direct methods but by converting to a Web Service
11/14/2018
uri="

20. CollabWS: Collaboration as a Web Service
This includes Audio-Video conferencing, Instant Messengers, shared applications and tools
There are basic standards H323 and SIP
There are peer to peer collaboration approaches like JXTA from Sun
There are various commercial products – Groove Networks effectively defines some shared application interfaces
CollabWS defines a superset of collaboration properties and subservices in XML/WSDL
Then you provide several bindings of collabws.wsdl to new transports and old ones (binary H323)
11/14/2018
uri="

21. H323 SIP JXTA
11/14/2018
uri="

22. SIP H323 JXTA
SIP
H323
JXTA
11/14/2018
uri="

23. A Typical SIP Message
REGISTER sip:registrar.biloxi.com Via: SIP/2.0/UDP :5060 To: Bob From: Bob Call-ID: CSeq: 1826 REGISTER Contact: Expires: 7200 Contact-Length: 0
Initially build a wrapper that accepts such messages and converts to …..
11/14/2018
uri="

24. Collaboration as a Web service - now
11/14/2018
uri="

25. SIP as XML
<sipasxml> <register> sip:registrar.biloxi.com</register> <via> SIP/2.0/UDP :5060</via> <to> Bob </to> <from tag="456248" > Bob </from> <call-id> </call-id> <cseq> 1826 REGISTER </cseq> <contact> </contact> <expires> 7200 </expires> <contact-length> 0 </contact-length> </sipasxml>
So this is either an internal format to which we map SIP messages or it is an external format used for Collaboration Web Service
11/14/2018
uri="

26. Future Collaboration Web Service
11/14/2018
uri="

27. Messaging/Events as a Web Service
We can implement messaging subsystem (between WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues
There are many different publish/subscribe models
JMS is a cluster of central servers
JXTA is a very dynamic Peer to Peer model where pipes are queues and topics (metadata) are service advertisements
Implement JMS API with JXTA protocol – different WSDL bindings here have different fault tolerance/reliability semantics
Could use JMS as long distance “carrier” between JXTA peers
JXTA provides higher performance than JMS for nearby recipients
11/14/2018
uri="

28. Destination Source Matching
Event Web Service
Web Service 1
(Virtual) Queue
Web Service 2
Destination Source Matching
Filter
Routing
workflow
WSDL Ports
Filter is mapping to PDA or slow communication channel (universal access) – see our PDA adaptor
Workflow natural as all messages “intercepted” by Event Web Service
Routing illustrated by JXTA
Destination-Source matching illustrated by JMS
11/14/2018
uri="

29. Broker Network Software multicast P2P: Brokers are clients Broker
Resource
Data
base
Broker
Broker
Broker
Broker
Broker
Broker
11/14/2018
uri="

30. Narada JXTA/JMS Integration
Is a network of event brokers which can reliably deliver XML specified events
Using openJMS selection module, becomes a distributed or conventional Java Message Service
Linking special JXTA proxies, it can link JXTA communities
Think of JXTA JMS and Narada as different bindings to event/messaging web service
11/14/2018
uri="

31. Education as a Web Service
Can link to Science as a Web Service and substitute educational modules
“Learning Object” XML standards already exist from IMS/ADL – need to update architecture
Web Services for virtual university include:
Registration
Performance (grading)
Authoring of Curriculum
Online laboratories for real and virtual instruments
Homework submission
Quizzes of various types (multiple choice, random parameters)
Assessment data access and analysis
Synchronous Delivery of Curricula
Scheduling of courses and mentoring sessions
Asynchronous access, data-mining and knowledge discovery
Learning Plan agents to guide students and teachers
11/14/2018
uri="

32. CMS (Content Management) as a Web Service
For meta-data, build federated database as a Web Service
Storage, update, lookup of objects
Registration of different control engines
For content, if W3C DOM and Java API as for SVG and “decent” XHTML browsers, then can wrap DOM as a web service for all interface functions
Can of course do this for IE5 but document object model unreliable
Build collaboration systems as sharing Web Service interfaces (not COM or other object API as proposed by Groove Networks)
11/14/2018
uri="

33. Integration of XML/Web Services into an Enterprise I
You are the CIO of a little known state’s (call it Xanadu) education system and been tasked to position it as a 2005 leader in technology enhanced education
Your initial review reveals a multitude of incompatible existing information systems distributed over Xanadu’s 50 campuses
Your cradle to grave community education mandate (which got the Governor elected) implies integration of all aspects of education in your information system
You note that performance issues are much less important in this application than interoperability (it currently takes months to get anything done so what's an extra millisecond or so at a WSDL interface) , so even in current primitive/immature state Web services look attractive
11/14/2018
uri="

34. Enterprise Web Services II
We start a requirements analysis and a catalog of existing “raw resources”. This will lead to the clean internal XML(WSDL) interfaces XIF and to the external XML interfaces (XEF) to the raw resources
Xanadu Internal Framework (XIF) built on top of XML Messaging, Security etc. Services
We find that The Grid Forum community has defined computing interfaces which we will adopt to support infrastructure needed for use of simulations and data repositories in curriculum
We find that Microsoft, the Liberty Alliance and the Grid have Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them and can define authentication & access control as a Web Service
We join the IMS consortium and adopt their standards for learning meta-data and user related information
We adopt the AICC and ADL SCORM standards for “learning objects” (course pages)
11/14/2018
uri="

35. Enterprise Web Services III
We define Xanadu educational agents which use XIF plus RDF/DAML-OIL to be able to capture educational goals
We expect commercial standards compliant solutions will be available for most other needed Educational web services
We find that we must interface with a dreadful collection of external raw resources coming from legacy PeopleSoft SAP Oracle and Microsoft products
As our new model is object (XML) and open standards based we build a separate XII (Xanadu Information Infrastructure) supporting XIF objects
We analyze structure of legacy systems and their relation to XIF and define the external framework XEF which captures much of the raw resources
This XEF Schema is extensible to capture quirks of each legacy system
We build a set of Web services that convert XEF into XIF
We inform vendors that we will only install XIF compliant systems in all future purchases
11/14/2018
uri="

36. WSDL from a Gateway View
11/14/2018
uri="

37. 7 Primitives in WSDL
types: which provides data type definitions used to describe the messages exchanged.
message: which represents an abstract definition of the data being transmitted. A message consists of logical parts, each of which is associated with a definition within some type system.
operation– an abstract description of an action supported by the service.
portType: which is a set of abstract operations. Each operation refers to an input message and output messages.
binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular portType.
port: which specifies an address for a binding, thus defining a single communication endpoint.
service: which is used to aggregate a set of related ports
11/14/2018
uri="

38. Discussion of 7 WSDL Primitives
types specify data-structures which are equivalent to arguments of methods
message specifies collections of types and is equivalent to set of arguments in a method call. Note that it is an “abstract method” in Java terminology
operation is a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message)
portType represents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java
binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades)
port combines a binding and an endpoint network address (URL) and is like a “class instance”
service consists of multiple ports and is equivalent to a “program” in Java
11/14/2018
uri="

39. Gateway and Web Services
We will use the Gateway Computing Portal as an example (
It is largely built using CORBA with a Java Server Pages front end
Several capabilities have been interfaced using WSDL
Job Submission (11 Methods including execute local and remote command, copy files etc. as well as Submit Job)
Manage WebFlow Session (67 Methods)
Generate Batch Script (just 1 method but two implementations developed – one at SDSC and one at Indiana – with UDDI to manage)
Each is one service – could have used finer grain services
Sample files are at
11/14/2018
uri="

40. SOAP wrapper JSP Servlet
Gateway Architecture
As needed DoD approved Kerberos/SecureID security, does not usually link to standard Grid technology (Globus)
Pre WSDL
Post WSDL
backend
backend
Add Castor Java to XML
Globus or submit to local queue system
Will replace historical CORBA
By EJB
CORBA
Corba middle tier SECIOP with Java modules
WSDL
SOAP wrapper JSP Servlet
JSP SSL
11/14/2018
uri="

41. 11/14/2018
uri="

42. SOAP and Gateway Portal I
Having specified service in WSDL, the run-time is implemented in SOAP which is “just” an XML header (info needed by transport – empty here) and body
Here is SOAP transported by HTTP message
This is execLocalCommand WSDL operation to run one particular command (ls) on current WebFlow directory
HTTP Header
Argument of operation
SOAP Envelope and body
Specify ls as
11/14/2018
uri="

43. SOAP and Gateway Portal II
HTTP
Header
SOAP Envelope and body
SOAP and Gateway Portal II
And this is the result of ls sent back to client in SOAP over HTTP
11/14/2018
uri="

44. WSDL Message Example <message name="submitRequest">
<part name="xmljob" type="xsd:string"/>
</message>
<message name="submitResponse">
<part name="response" type="xsd:string"/>
For the batch script service, we pass the XML description of the job as a string and get back the script as a string.
In general, any XML primitive or complex types can be used in messages.
We could improve our service by defining a BatchScript complex type.
11/14/2018
uri="

45. WSDL portTypes Example
<portType name="BatchScriptServicePortType">
<operation name="batchGen">
<output message="tns:submitResponse"
name="submitResponse"/>
<input message="tns:submitRequest"
name="submitRequest"/>
</operation>
</portType>
A portType corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a BatchScriptServiceBinding.java class.
11/14/2018
uri="

46. WSDL SOAP Binding Example
<binding name="BatchBinding" type="tns:BatchScriptServicePortType">
<soap:binding style="rpc" transport="
<operation name="batchGen">
<soap:operation soapAction=""/>
<input>
<soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle="
</input>
<output>
<soap:body use="encoded" namespace="urn:BatchScriptService“
encodingStyle="
</output>
</operation>
</binding>
note binding’s “type” attribute points back to the portType tag by name
11/14/2018
uri="

47. WSDL Ports and Services
<service name="BatchScriptService">
<documentation>BS stands for Batch Script
</documentation>
<port binding="BatchBinding”
name="BatchPort">
<soap:address location=
"
</port>
</service>
</definitions>
ports are concrete implementations of portTypes and point back to a particular binding (by name).
They also point to the specific location of a server that implements the service.
A service is a collection of one or more ports.
11/14/2018
uri="

48. What is a Virtual XML Interface
We can specify interfaces in XML but we are not required to implement in XML.
Example 1:We aren’t likely to change syntax of mail Reply-to:Geoffrey Fox To: Geoffrey Fox Subject: A Test for Tutorial A simple mail message Geoffrey Fox FAX Phones Cell Home
But we could specify and indeed store in XML with transport done using conventional SMTP.
So conventional mail is easy to give a virtual XML specification for with name:value becoming <name>value</name> but this is not only way
11/14/2018
uri="

49. Mail in XML
<mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” > <smtpheaders> <reply-to >Geoffrey Fox</reply-to> <to >Geoffrey Fox</to> <subject>A Test for Tutorial</subject> </smtpheaders> <smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn://123/45/6789” whitespace=“preserve” > Geoffrey Fox FAX Phones Cell Home </signature> </smtpbody> </mailasxml>
Such an interface could be used by “User Messaging as a Web service” which could perhaps integrate News Groups, , text chat, instant messenger, voic
11/14/2018
uri="

50. Virtual XML for MPI I
In days gone by, we used to study: MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle)
Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node
Need to support distributed and shared memory
We want to define all interfaces in WSDL – at least all of them which could be “external” ports
11/14/2018
uri="

51. Virtual XML for MPI II
datatype becomes an XML Schema specifying structure of transmitted data – could either be a URI for Schema or its realization
There would be a built-in schema suitable for handling standard Fortran/C datatypes
buf is a handle to an XML instance of Schema datatype
count is still an integer (xs:integer where xs is Schema namespace xmlns:xs=
For some bindings count is implied by buf
dest becomes a URI with special notation such as legacympi://procnum/local/nnn for processor numbers
tag becomes an instance of (new) mpitag Schema
comm becomes an instance of (new) mpicommunicator Schema
11/14/2018
uri="

52. Virtual XML for MPI III
So one defines mpi.wsdl with all 100 or so methods defined as (abstract) operations
We will define several bindings including the classic HPCC method calls for Fortran C C++ and Java with either distributed memory or shared memory mode
For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf
Note one can use both static and run-time compilation to map XML specified services into particular implementations
MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH
This brings Web Services to MPI but also MPI to web services
11/14/2018
uri="