Integrating Peer to Peer Networks and Web Services with the Grid

Integrating Peer to Peer Networks and Web Services with the Grid
Part of GGF4 Tutorial Toronto February Grid Programming with Web Services, Software Components and Portal Development Tools PTLIU Laboratory for Community Grids Geoffrey Fox, Marlon Pierce, Shrideep Pallickara, Choonhan Youn Computer Science, Informatics, Physics Indiana University, Bloomington IN 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Classic Multi-Tier Architecture
Back-end Tier Raw Resources Middle Tier Servers Client Tier Relational Database “Flat” File System IIOP Object Broker HTTP Web Server RMI Specialized Java Server SOAP + WSDL Web Service Old and New Useful Backend Systems 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Object Web View of the World I
“Everything electronic” is a resource Computers Programs Data (from sensors to this presentation to to databases) People “Everything electronic” is a distributed object “All External Interfaces” are defined in XML whether this is a property (data-structure) or a method (service, function, subroutine) We can assume that a data-structure property has getproperty() and setproperty(value) methods to act as interface All resources have interfaces All resources are linked by messages with format and transport protocol to be discussed but as interfaces defined in XML, message structure must be specifiable in XML 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Object Web View of the World II
All resources have a URI which can be a URL (location) to URN (property tag) I could be ssn://abc/de/fghk or my homepage 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

The 3 Critical XML Interfaces
Raw Resources Raw Data Data (sort of dumb) Information User Rendering (Virtual) XML Data Interface Processing Server Information/Knowledge (Virtual) XML Information Interface Render to XML Display Format (Virtual) XML Rendering Interface Clients 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

What is a Virtual XML Interface
We can specify interfaces in XML but we are not required to implement in XML. Example 1:We aren’t likely to change syntax of mail Reply-to:Geoffrey Fox To: Geoffrey Fox Subject: A Test for Tutorial A simple mail message Geoffrey Fox FAX Phones Cell Home But we could specify and indeed store in XML with transport done using conventional SMTP. So conventional mail is easy to give a virtual XML specification for with name:value becoming <name>value</name> but this is not only way 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

uri="gxos://gridform/ggf4/tutorial2/fox" email="gcf@indiana.edu"
Mail in XML <mailasxml uri=“gxos://mail/users/gcf/sent/2002/february/290” > <smtpheaders> <reply-to >Geoffrey Fox</reply-to> <to >Geoffrey Fox</to> <subject>A Test for Tutorial</subject> </smtpheaders> <smtpbody> <message whitespace="collapse"> A simple mail message</message> <signature personuri=“ssn://123/45/6789” whitespace=“preserve” > Geoffrey Fox FAX Phones Cell Home </signature> </smtpbody> </mailasxml> Such an interface could be used by “Messaging as a Web service” which could perhaps integrate , text chat, instant messenger, voic 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Quick and Dirty mail as XML
If your boss orders you to produce a quick messaging web service then you might instead define: <kwikmailasxml whitespace=“preserve” > Reply-to:Geoffrey Fox To: Geoffrey Fox Subject: A Test for Tutorial A simple mail message Geoffrey Fox FAX Phones Cell Home </kwikmailasxml> This is allowed but of course relies on an SMTP parser not an XML parser to decode message Our messaging web service would allow either form but put them on different ports. Internally we would convert to the common XML representation 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

XML in a Nutshell So we have seen .xml files – XML Instances or the equivalent of Objects in Java There are corresponding .xsd or Schema files which are the equivalent of Classes in Java Schema happen to be defined in XML syntax Schema totally replace DTD’s Originally Schema defined “things”/”properties”/”data” At times in this tutorial, we will use .wsdl files which define Web Service Interfaces The wsdl specification separates into components that specify structure of service which is equivalent to “Abstract Method or Interface” and those that specify actual service (equivalent of program and its methods) 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

XML Rendering Interfaces
The simplest example is XHTML which is HTML in an XML Syntax See <BR> in HTML MUST become <br/> <p> must become <p>Material in Paragraph</p> etc. Another good example is SVG (Scalable Vector Graphics) which is 2D Vector Graphics and can represent Macromedia Flash PowerPoint and StarOffice Adobe Illustrator/Photoshop etc. See Here is a snippet of SVG from above link illustrating text, matrix transformations, positioning and styles <text transform="matrix( )" style="fill:#66FF00;stroke:none;"> <tspan x="5" y="0">Spring Semester MWF 2:30 pm - 3:20 pm </tspan> </text> 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

XML in a Nutshell Concluded
We can browse some examples at: and on the following page Note Schema are labeled by URI’s – their namespaces which allow one to build class libraries WSDL files are also referenced by their URI’s (namespaces) in the XML (SOAP) message communicating between 2 Web services Note version number in URI are reasonable links Always remember that XML defines (potentially complicated) objects and it can specify both data and methods this could have profound implications for architecture and implementation of Enterprise infrastructure Use Castor ( to link XML & Java “automatically” 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

A Sample of XML <?xml version="1.0"?> <okc version="3" xmlns=" <event> <comment> This is a Test </comment> <sender Fox</sender> <distribution> Community Grids Project Reports </distribution> <organization></organization> <update createuri="gxos://okc/newsgroups/cgreports/$next“ /> <keywords></keywords> <subject></subject> <message/> <filingdate> 2/6/2002 </filingdate> <cg:messageType xmlns:cg=“ > Test </cg:messageType> </event> </okc> Schema are here 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP over HTTP Structure
SOAP defines a message with envelope, header and body The SOAP standard also defines a binding to HTTP which is equivalent to methodology for delivering message The SOAP body can specify data as well as fault processing information SOAP can be transported over standard mail by just putting XML in body of mail – see HTTP Delivery with address 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP Example from W3C I We take November 16 tutorial: This is a SOAP Message – defining a reservation request Header has key meta-data of application (namelist m: and instructions to SOAP (intermediate actor) processors Start Envelope Start Header End Header Start Body 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP Example (Concluded)
Start Body This has the SOAP Body with actual request End Body End Envelope 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP Binding to SMTP from W3C
Same example as before This is very straightforward! 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Remarks on SOAP Structure
If you look at networking protocols, at each layer one specifies a header and a body HTTP has a header telling system where to send information and some high level specification of what to get In example, we see that body of HTTP message is full SOAP envelope and this envelope has SOAP Header and Body interpreted by “application” At a lower level in protocol stack, TCP/IP and ATM etc. have their own headers and bodies It is the standard Russian Doll situation – each doll has a body (contained inside it) which is the full doll of higher level protocol HTTP SOAP SOAP Body 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP Attachments This uses same approach as or HTTP and is a general way of including binary data in XML – but not understood by most parsers! Very important? XML links to MIME Attachment using absolute or relative URI’s 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

What is a Web Service I A web service is a computer program running on either the local or remote machine with a set of well defined interfaces (ports) specified in WSDL In principle, computer program can be in any language (Fortran .. Java .. Perl .. Python) and the interfaces can be implemented in any way what so ever Interfaces can be method calls, Java RMI Messages, CGI Web invocations, totally compiled away (inlining) but The simplest implementations involve XML messages (SOAP) and programs written in net friendly languages like Java and Python Web Services separate the meaning of a port (message) interface from its implementation 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

What is a Web or Grid Service II?
The Web is distributed system allowing communities to access seamlessly heterogeneous resources from heterogeneous clients Resources are web-pages, instruments, Object repositories, Simulation codes running on supercomputers …. A Service is a generic application or capability respecting standards (general web and application specific) allowing multiple providers to compete on a given service For a simulation to be a web service, we want it to have standard interfaces so be run from standard “portals” and access “standard format” data Middle Tier Broker Portal is customizable User interface Back end Capability Resource 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Role of Web Services Define interfaces of web applications so that computer-computer interactions are enabled Defines virtual XML for all system and application services WSDL is XML versions of Class and Method definitions SOAP is XML version of message UDDI or WSIL catalogs WSDL based services enabling precise linkage of them WSFL and WSCL are candidate linkage languages Security Catalog Payment Credit Card Warehouse shipping WSDL interfaces WSDL interfaces 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Raw Resources Clients Raw Data Raw Data Render to XML Display Format
(Virtual) XML Data Interface Web Service (WS) WS WS WS etc. XML WS to WS Interfaces WS (Virtual) XML Knowledge (User) Interface Render to XML Display Format (Virtual) XML Rendering Interface Clients 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Some Sort of XML Protocol Stack
Protocols are built in a layered fashion – one on top of another At higher (highest) level a protocol becomes an API (Application Interface) Some reserve protocol to denote the representation of a process involving some negotiation but to my naïve view there is no consensus as to difference between an API and a protocol We are expanding application (top) layer of OSI protocol stack UDDI or WSIL WSFL WSDL SOAP or RMI HTTP or SMTP or IIOP or RMTP TCP/IP Physical Network 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Details of Application Layer
UDDI finds where programs are remote( (distributed) programs are just Web Services WSFL links programs together WSDL defines interface (methods, parameters, data formats) SOAP defines structure of message including serialization of information HTTP is negotiation/transport protocol TCP/IP is layers 3-4 of OSI Physical Network is layer 1 of OSI UDDI or WSIL WSFL WSDL SOAP or RMI HTTP or SMTP or IIOP or RMTP TCP/IP Physical Network 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Role of UDDI with WSDL UDDI plays a role like find or grep under UNIX or Google on the web or yellow pages in the real world A critical difference is that UDDI aims at finding resources matching precise information Suppose we were a shopping cart service and told by a customer that we need to contact the SuchandSuch online credit card system We would not only need to find but also identify the exact URL’s for “approval” and “payment” channels – we would need to get the syntax correct for exchanging data For instance, we would need to know if system used US Canadian or HK dollars So UDDI knows about WSDL Interfaces and this knowledge enables programs with such interfaces to link together UDDI and WSDL together enable dynamic applications Note “service” just means web-based application built according to agreed interaction rules – such “plug and play” software usually called “components” 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Software Engineering and WSDL/SOAP?
Software Engineering is a field whose goal is agreed by most to be important Namely to codify the building of software so that it is more re-usable, modular, easier to maintain and easier to build with teams However there is not so much agreement as to how to do this and if the field has established itself WSDL is part of component-based software engineering methodology of the web As the Web could be a set of processes and threads on a single PC or Palmtop, WSDL can in fact provide software engineering for conventional sequential or parallel computing. WSDL/SOAP addresses same issues as calling sequences and library methodology in conventional programming 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL and SOAP We will use the Gateway Computing Portal as an example ( It is largely built using CORBA with a Java Server Pages front end Several capabilities have been interfaced using WSDL Job Submission (11 Methods including execute local and remote command, copy files etc. as well as Submit Job) Manage WebFlow Session (67 Methods) Generate Batch Script (just 1 method but two implementations developed – one at SDSC and one at Indiana – with UDDI to manage) Each is one service – could have used finer grain services Sample files are at 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Gateway Architecture As needed DoD approved Kerberos/SecureID security, does not usually link to standard Grid technology (Globus) Pre WSDL Post WSDL backend backend Add Castor Java to XML Globus or submit to local queue system Will replace historical CORBA By EJB CORBA Corba middle tier SECIOP with Java modules WSDL SOAP wrapper JSP Servlet JSP SSL 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Basic Portal SOAP Interaction
Client Web Browser (HTML) User Interface Web Server HTTP(s) UI Server Components (including SOAP Client) SOAP over HTTP SOAP Service Components The deployment descriptor maps the SOAP request to the correct service component (Apache SOAP specific). 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Server-Server Communication for Basic SOAP Requests
We have two servers: User interface (UI) server. Includes components for basic functionality (like managing user sessions) plus SOAP client modules. SOAP server. Converts SOAP requests into calls to the underlying service. User interacts with UI to make requests for services. UI server constructs SOAP request and acts as the SOAP client. For example, if UI server is Tomcat, developer uses Apache SOAP’s Java API implementation to create JavaBean components to act as SOAP clients. SOAP server executes the request (generate batch script) and returns response. Note that the UI’s SOAP component must match the interface on the SOAP server. This is where WSDL will come in. UI server formats display for user. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP and Gateway Portal I
Having specified service in WSDL, the run-time is implemented in SOAP which is “just” an XML header (info needed by transport – empty here) and body Here is SOAP transported by HTTP message This is execLocalCommand WSDL operation to run one particular command (ls) on current WebFlow directory HTTP Header Argument of operation SOAP Envelope and body Specify ls as 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP and Gateway Portal II
HTTP Header SOAP Envelope and body SOAP and Gateway Portal II And this is the result of ls sent back to client in SOAP over HTTP 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL Abstractions WSDL abstracts a program as an entity that does something given one or more inputs with its results defined by streams on one or more outputs. Functions are defined by method name and parameters methodname(parm1,parm2, … parmN) Where parameters are “Input” “Output” or both In WSDL, we will have a Web Service which like a (Java or CORBA Program) can be thought of as a (distributed) object with many methods Instead of a function call, the “calling routine” sends an XML message to the Web Service specifying methodname and values of the parameters Note name of function is just another parameter 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

7 Primitives in WSDL types: which provides data type definitions used to describe the messages exchanged. message: which represents an abstract definition of the data being transmitted. A message consists of logical parts, each of which is associated with a definition within some type system. operation– an abstract description of an action supported by the service. portType: which is a set of abstract operations. Each operation refers to an input message and output messages. binding: which specifies concrete protocol and data format specifications for the operations and messages defined by a particular portType. port: which specifies an address for a binding, thus defining a single communication endpoint. service: which is used to aggregate a set of related ports 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Discussion of 7 WSDL Primitives
types specify data-structures which are equivalent to arguments of methods message specifies collections of types and is equivalent to set of arguments in a method call. Note that it is an “abstract method” in Java terminology operation is a a collection of input output and fault messages; there are 4 types of operation one-way(service just receives a message), request-response(RPC), solicit-response, notification (services pushes out a message) portType represents a single channel that can support multiple operations. It is “abstract” as specified as a set of operations. It is equivalent to a “interface or abstract class” in Java binding tells you transport and message format for a porttype (which can have multiple bindings to reflect say performance-portability trades) port combines a binding and an endpoint network address (URL) and is like a “class instance” service consists of multiple ports and is equivalent to a “program” in Java 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Basic Portal WSDL Interaction
Client Web Browser (HTML) User Interface Web Server HTTP(s) UI Server Components (including WSDL clients) SOAP SOAP over HTTP Service Components WSTK compiles WSDL into client stubs that can be used by UI components 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL-Based Portal Services
Basically the same as SOAP server-server interaction described previously. We can use Web Services Toolkit (WSTK) from IBM to generate java client stubs from the WSDL interface. These stubs can be used to create “local” java objects in the UI server that use SOAP RPC to talk to server. So the WSTK creates CORBA-like remote objects. We still use SOAP+HTTP under the hood to make the method calls. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL Definitions and Namespaces
<?xml version="1.0" encoding="UTF-8" ?> <definitions name="BatchScriptService" targetNamespace=" xmlns=" xmlns:soap=" xmlns:tns=" xmlns:xsd=" > Default Schema (null namespace) is WSDL soap namespace is SOAP binding of WSDL tns namespace is this WSDL file xsd namespace defines (ancient) XML Schema 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL Message Example <message name="submitRequest"> <part name="xmljob" type="xsd:string"/> </message> <message name="submitResponse"> <part name="response" type="xsd:string"/> For the batch script service, we pass the XML description of the job as a string and get back the script as a string. In general, any XML primitive or complex types can be used in messages. We could improve our service by defining a BatchScript complex type. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL portTypes Example
<portType name="BatchScriptServicePortType"> <operation name="batchGen"> <output message="tns:submitResponse" name="submitResponse"/> <input message="tns:submitRequest" name="submitRequest"/> </operation> </portType> A portType corresponds to a Java class, so if we compile this WSDL to make client stubs, we will generate a BatchScriptServiceBinding.java class. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL SOAP Binding Example
<binding name="BatchBinding" type="tns:BatchScriptServicePortType"> <soap:binding style="rpc" transport=" <operation name="batchGen"> <soap:operation soapAction=""/> <input> <soap:body use="encoded“ namespace="urn:BatchScriptService" encodingStyle=" </input> <output> <soap:body use="encoded" namespace="urn:BatchScriptService“ encodingStyle=" </output> </operation> </binding> note binding’s “type” attribute points back to the portType tag by name 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

WSDL Ports and Services
<service name="BatchScriptService"> <documentation>BS stands for Batch Script </documentation> <port binding="BatchBinding” name="BatchPort"> <soap:address location= " </port> </service> </definitions> ports are concrete implementations of portTypes and point back to a particular binding (by name). They also point to the specific location of a server that implements the service. A service is a collection of one or more ports. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Portal UDDI Interaction
Client Web Browser (HTML) User Interface Web Server HTTP(s) SOAP SOAP over HTTP (1) (2) UDDI (3) UI Modules include both WSDL and UDDI clients 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

SOAP+UDDI+WSDL UI server now includes both WSDL and UDDI client components. UI’s UDDI components are, for example JavaBeans using UDDI4J API to interact with the UDDI. The UDDI client components can be used to build administrator HTML/JSP interfaces to add, modify UDDI entries on the server. UDDI client components can also be used to build search interfaces for regular users. Note the UDDI client implementation uses SOAP for RPC to talk to the server. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Steps 1-3 on Figure User requests that the UI server performs a search for a particular service. UI server now acts as a client to the UDDI server, gets back the search results, plus URLs for the SOAP servers and the WSDL for the desired service. User selects the desired service provider from UI’s display. UI then becomes client to the SOAP server and executes the service. 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Virtual XML for MPI I In days gone by, we used to study: MPI_SEND(buf, count, datatype, dest, tag, comm) [ IN buf] initial address of send buffer (choice) [ IN count] number of elements in send buffer (nonnegative integer) [ IN datatype] datatype of each send buffer element (handle) [ IN dest] rank of destination (integer) [ IN tag] message tag (integer) [ IN comm] communicator (handle) Now our SPMD code is a Web service running in some large scale parallel machine – perhaps integrated with an Apache server running on node Need to support distributed and shared memory We want to define all interfaces in WSDL – at least all of them which could be “external” ports 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Virtual XML for MPI II datatype becomes an XML Schema specifying structure of transmitted data – could either be a URI for Schema or its realization There would be a built-in schema suitable for handling standard Fortran/C datatypes buf is a handle to an XML instance of Schema datatype count is still an integer (xs:integer where xs is Schema namespace xmlns:xs= For some bindings count is implied by buf dest becomes a URI with special notation such as legacympi://procnum/local/nnn for processor numbers tag becomes an instance of (new) mpitag Schema comm becomes an instance of (new) mpicommunicator Schema 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

HPCC and WSDL HPC Engine XML based Interfaces WSDL Data base
Java C++ .. Proxy HPC Engine MPI .. SOAP IIOP RMI ….. XML based Interfaces WSDL Data base Persistent Storage Parallel I/O … Simulation Component Data Component HPCC and WSDL Backend Resources MPP Middle Tier Client 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Virtual XML for MPI III So one defines mpi.wsdl with all 100 or so methods defined as (abstract) operations We will define several bindings including the classic HPCC method calls for Fortran C C++ and Java with either distributed memory or shared memory mode For C++ and Java we would implements bindings for either “object” or “packed consecutive byte” interpretation of buf Note one can use both static and run-time compilation to map XML specified services into particular implementations MPI_INIT() might be implemented with JXTA (peer-to-peer advertisments) to find nodes; the communicator might be transmitted by SOAP while buf transmission uses MPICH This brings Web Services to MPI but also MPI to web services 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Compiling for WSDL Web Server Web Application1 Web Application2 Single Address space Method1 Method2 stack Web Server WSDL Messages Shared Memory Distributed System 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

How to Integrate XML/Web Services into an Enterprise I
You are the CIO of a little known country’s (call it Quagmire) education system and been tasked to position it as a 2005 leader in technology enhanced education Your initial review reveals a multitude of incompatible existing information systems distributed over Quagmire’s 50 campuses Your cradle to grave community education mandate (which got the President – your uncle – elected) implies integration of all aspects of education in your information system You note that performances issues are much less important in this application than interoperability (it currently takes months to get anything done so what's an extra millisecond or so at a WSDL interface) , so even in current primitive state Web services look attractive 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Enterprise Web Services II
We start a requirements analysis and a catalog of existing “raw resources”. This will lead to the clean internal XML(WSDL) interfaces QIF and to the external XML interfaces (QEF) to the raw resources Quagmire Internal Framework (QIF) We find that The Grid community has defined computing interfaces which we will adopt to support infrastructure needed for use of simulations and data repositories in curriculum We find that Microsoft, the Liberty Alliance and the Grid have Security schemes but are relieved that with a little bit of XSLT (transformations) we can map between them We join the IMS consortium and adopt their standards for learning meta-data and user related information We adopt the AICC and ADL SCORM standards for “learning objects” (course pages) 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Enterprise Web Services III
We define Quagmire educational agents which use QIF plus RDF/DAML/OIL to be able to capture educational goals We expect commercial standards compliant solutions will be available for most other needed Educational web services We find that we must interface with a dreadful collection of external raw resources coming from legacy PeopleSoft SAP Oracle and Microsoft products As our new model is object (XML) and open standards based we build a new QII (Quagmire Information Infrastructure) supporting QIF objects We analyze structure of legacy systems and their relation to QIF and define an external framework QEF which captures much of the raw resources This Schema is extensible to capture quirks of each legacy system We build a set of Web services that convert QEF into QIF We inform vendors that we will only install QIF compliant systems in all future purchases 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Some Science Web Services
These build on general (community) web services 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Some General Grid Web Services
9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Education as a Web Service
Can link to Science as a Web Service and substitute educational modules “Learning Object” XML standards already exist from IMS/ADL – need to update architecture Web Services for virtual university include: Registration Performance (grading) Authoring of Curriculum Online laboratories for real and virtual instruments Homework submission Quizzes of various types (multiple choice, random parameters) Assessment data access and analysis Synchronous Delivery of Curricula Scheduling of courses and mentoring sessions Asynchronous access, data-mining and knowledge discovery Learning Plan agents to guide students and teachers 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

RDF in a Nutshell Subject Object Predicate The essence of RDF is a statement which has a subject and predicate and an object The “subject” has the property “predicate” with value “object” The Web Page has the title “RDF Model and Syntax” Mary had a husband called John Statements can specify metadata but more generally any facts are expressible in this way One can associate multiple objects with a subject and so build up a general node of a tree with parent as subject and objects as children. DAML and OIL extend RDF with more powerful relations and ontologies 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Same Predicate: Student Same or Different Predicates
RDF Tree Fragment Class General Same Predicate: Student Different Predicates Subject Agatha Henry Fred Same or Different Predicates Nick name grade Object Object Object Freddie B- Specific Object Object Object 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

An Example of RDF and Dublin Core
<rdf:RDF xmlns:rdf=“ xmlns:dc=“ <rdf:Description about=“ <dc:Title>D-Lib Program - Research in Digital Libraries</dc:Title> <dc:Description>The D-Lib program supports the community of people with research interests in digital libraries and electronic publishing. </dc:Description> <dc:Publisher>Corporation For National Research Initiatives</dc:Publisher> <dc:Date> </dc:Date> <dc:Subject> <rdf:Bag> <rdf:li> Research; statistical methods</rdf:li> <rdf:li> Education, research, related topics</rdf:li> <rdf:li> Library use Studies </rdf:li> </rdf:Bag> </dc:Subject> <dc:Type>World Wide Web Home Page</dc:Type> <dc:Format>text/html</dc:Format> <dc:Language>en</dc:Language> </rdf:Description> </rdf:RDF> 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Peer to Peer Community Grid
XML Skin XML Skin Message Or Event Based Inter Connection Resource Data base Soft ware XML Defined Resources connected by XML defined messages Implementation of resource and connection may or may not be XML Peer to Peer Community Grid 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

P2P Grid Architecture Note proposed architecture – resources communicating via messages – is already peer-to-peer as we consider everything democratically – clients, data sources, backend resources Peer-to-peer systems (P2P) include Napster, Gnutella and Avaki discussed in another tutorial JXTA ( is attractive as has very clear architecture and primitives such as peers (users), peer groups , advertisements (XML messages), pipes (message queues), rendezvous (repository of advertisements and other places to find things) JXTA Peers communicate via messages with XML topics used to match service providers with service users Different mechanism than UDDI for registering and discovering resources Resources Queued Events 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Classic Grid Architecture
Resources Database Database Neos Composition Middle Tier Brokers Service Providers Netsolve Security Portal Portal Typically separate Clients Servers Resources Clients Users and Devices 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Resources Database Database Resources Peer-to-Peer Grid Event Broker Event Broker Event Broker Event Broker Resources Resources 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Semantic Grid & Digital Brilliance
Peer to Peer networks teach us that we can build “small worlds” where distance between nodes is logarithmic in number of nodes Consider a Grid of WSDL services linked (through UDDI) together This is spirit of Semantic web – RDF/DAML/OIL metadata enables meaningful linkage We do not need to link everybody but only to establish “small world” routes Physics analogies suggest that phase transitions will occur when “enough” nodes are linked – one will get nodes to align in the direction of new knowledge This suggests ways of quantifying value of metadata induced linkages and areas where one “should” add more WSDL specifications 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Publish/Subscribe as a Web Service
We can implement messaging subsystem (between WSDL resources) with either direct messages or by a queued system where you publish messages to queues and subscribe as receiver to particular queues Natural asynchronous collaboration model which is in fact fast enough for synchronous collaboration There are many different publish/subscribe models JMS is a cluster of central servers JXTA is a very dynamic Peer to Peer model where pipes are queues and topics (metadata) are service advertisements Implement JMS API with JXTA protocol – different WSDL bindings here have different fault tolerance/reliability semantics Could use JMS as long distance “carrier” between JXTA peers JXTA provides higher performance than JMS for nearby recipients Pallickara built an intermediate dynamic message broker subsystem which can link JMS and JXTA see: 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Single Server P2P Illusion
Data base JMS Broker 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Broker Network Software multicast P2P: Brokers are clients Resource Data base Broker Broker Broker Broker Broker Broker 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Community P2P Grid GXOS Server Data base JMS Server Generate “Automatically” GXOS Server JXTA Rendezvous 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Narada JXTA/JMS Integration
Is a network of event brokers which can reliably deliver XML specified events Using openJMS selection module, becomes a distributed or conventional Java Message Service Linking special JXTA proxies, it can link JXTA communities 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Event Web Service Web Service 1 (Virtual) Queue Web Service 2 Filter
Destination Source Matching Filter Routing workflow WSDL Ports 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Some Research Issues for P2P Grid
What happens to programming languages when data structures are defined in XML How do we manage a sea of virtual XML? Register, find and link objects This is “distributed operating system of the world” ? What happens to databases when everything is an Object defined in XML and transformed by Java? How and when do we compile virtual XML Convert slow XML message to super fast method call on stack How do we implement services such as Security and collaboration over a range of grain sizes Supporting all “grain sizes” we get some sort of dynamic fractal world which looks like XML objects exchanging XML messages at all scales Not well supported by centralized services (P2P problem) Semantic Grid: as metadata increases, objects link together forming digital brilliance – a phase transition in information space 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Collaborative Web Resources
Collaboration is “just” sharing objects What about Collaborative Web Services ? You can in some cases do this automatically just by multicasting messages from service to clients This is achieved by service publishing messages and clients subscribing Many applications do not expose all state changes E.g. when I edit PowerPoint slide, PowerPoint does not tell the world by sending an (XML) message Solved by shared event collaboration model and requires one to view user interface as a “port” in WSDL sense and treat “event handlers” (mouseover, click etc.) as messages in WSDL Groove Networks does use XML front end to COM interfaces More elegantly can use W3C DOM for (the few) documents (SVG is one) and “universal event handlers” Interesting research area 9/17/2018 uri="gxos://gridform/ggf4/tutorial2/fox"

Collaboration as a Web service I

Future Collaboration Web Service

Integrating Peer to Peer Networks and Web Services with the Grid

Similar presentations

Presentation on theme: "Integrating Peer to Peer Networks and Web Services with the Grid"— Presentation transcript:

Similar presentations

About project

Feedback

Log in

Auth with social network:

Integrating Peer to Peer Networks and Web Services with the Grid

Similar presentations

Presentation on theme: "Integrating Peer to Peer Networks and Web Services with the Grid"— Presentation transcript:

Similar presentations

About project

Feedback