grid computation and web services1

2 Grid computation. –Motivation, a very simple example. –Architecture and implementation problems. Agenda Web Services –What are they?, why do we need them? –Some technologies involved XML, SOAP, WSDL, UDDI. Putting it all together Grid computation. –Motivation, a very simple example. –Architecture and implementation problems.

grid computation and web services3 Motivation (some history) 95% of CPU power in the world is idle. Multiple scientific tasks like require supercomputing power (weather forecast, weapons simulations, etc). The cost of this supercomputers is millions of dollars. Why not using existing resources to tackle this tasks.

grid computation and web services4 The Idea Use the CPU power available with the help of volunteers ready to contribute to your project. Connect the computers throw existing networks (internet). Get a de facto supercomputer. Save millions.

grid computation and web services5 SETI (Search for Extra-Terrestrial Intelligence). connects more than computers throughout the world having a work- power bigger than any existing super-computer. SETI Statistics. –Users: (the original target was ). –Total CPU time: years. –FPO: e+21. –24 hours FPO TFLOPS/sec. The earth simulator (350 million US$) has 40 TFLOPS/sec.

grid computation and web services6 CRASS (1) The problem. –More than 7000 observed orbiting objects larger than 10 cm. –Between to objects larger than 1 cm. –High speeds can make collisions fatal, making the debris problem worse. –1 on 10 shuttle missions have performed collisions avoidance maneuvers.

grid computation and web services7 CRASS (2) USS space com DB tracks the position of the debris. CRASS uses the DB resources and a model of the forces acting to predict possible collisions. CRASS master assigns to each node the operational spacecraft and some debris.

grid computation and web services8 CRASS (3) The nodes calculate the debris orbit and issue collisions warnings in a specific time scope. Master receives the results and assign another period of time till the simulation is completed.

grid computation and web services9 CRASS (4): Architecture MASTER DB node Scheduler - Test speed of the nodes - Distribute the debris accordingly

grid computation and web services10 Conclusions for CRASS CRASS works because the debris does not interact between them. No need of communication between nodes. Overhead of data is small (the number of operational spacecraft is small in relation to the debris). The problem is easily distributed. But there are other not so simple cases. Grids are more than CRASS.

grid computation and web services11 A Grid is more than that A grid is a seamless computing information environment composed of: –Storage systems. –Networks. –Data servers. –On-line instruments (telescopes, particle accelerators, etc). –Computing processors.

grid computation and web services12 Implementing a Grid Putting together so different elements is a complex task. –Need of a common language between them. –Need of communications protocols. –Security issues. Grid applications are usually distributed and thus hard to program.

grid computation and web services13 Implementing a Grid (2) Component (object) oriented programming has helped. –Reusability of code. –OO design subdivides problems in small ones. Suitable approach to deploy big applications. Making objects work together is still a hard task involving many issues. CORBA was supposed to be the solution

grid computation and web services14 More difficulties Slow communications between computers. Wide geographical area. Different users platforms (WIN, UNIX, MAC). Some problems require strong interaction between the different CPUs. –For these problems supercomputers cannot be replaced (for example: chain reactions, weather simulations...). –So we still need supercomputers.

grid computation and web services15 Dividing the problems It’s very important to divide developers and users. –Scientific programmers (no knowledge of grid technologies). –Grid developers (no knowledge of the scientific problem). –End users (no knowledge at all). Each group needs appropriate tools to solve their problems. We need tools to connect the elements.

grid computation and web services16 Dividing the problems (2) Throughout the presentation we will focus in the second group problems (grid architecture). Some new arising tools called hosting environments help also to deploy highly distributed applications. –The programs (elements, objects) do not run over the OS but over a hosting environment.

grid computation and web services17 Grid Portals Grid portals are a solution for end users. They provide an easy way to use then resources of a grid with no special knowledge of computing. The user will be able to use the portal for: –Locate grid resources. –Set in the portal the task he wants to perform. –Define execution parameters (number of processors, memory needs, etc).

grid computation and web services18 Nimrod Portal Applet

grid computation and web services19 Grid Architecture elements Security Protocols –Authentication and privacy. Information Services –users need to see which resources are available for use. Schedulers –multiple resources can be scheduled concurrently.

grid computation and web services20 An Example of a Grid Portal Architecture Web Browser (1) The user browser connects to the Portal Server. Web Server (1) Certificates Server (2) Application Manager (3) Messages DB (4) (2) The Server checks if the user is a registered grid user (3) The Grid Application Manager is notified of the user presence (4) The Grid Application Manager connects to the user through a messages channel

grid computation and web services21 Towards Grid Services The grid model is developing. –Not only scientific cooperation. –Business want to link their different departments maybe in a very wide geographical area using grid technologies. –Outsourcing: companies are using specialized companies and reducing their IT departments. Economies of scale. Price/performance ratio.

grid computation and web services22 Towards Grid Services (2) We would like to make grids more open. Other people to use the grid capabilities. Redefine the grid elements as services. service: network-enabled entity that provides some capability. The key issue is interoperability between services.

grid computation and web services23 Towards Grid Services (3) Redefine the architecture elements of a grid as services: –Security services: authentication and privacy. –Information services: what resources do we have in the grid (data, processors, instruments). –Job-submission resources. –Co-scheduling service: multiple resources. –Caching: in large grids caching data can improve performance dramatically.

grid computation and web services24 Towards Grid Services (4) The interoperability problem basically consists in: –Defining what the service does. –Giving instructions for it’s use (invocation APIs) We would also like a mechanism to find appropriate services. We need semantic information about the services.

grid computation and web services25 It’s Not Easy We need specific protocols (hopefully universally accepted) to define our services. XML helps with it’s semantic approach but it’s not enough. If we use the Internet as a transport channel how do we avoid firewalls? –RPCs are not firewall-friendly.

grid computation and web services26 Agenda Grid computation. –Basic ideas. –Implementation problems. Web Services –What are they?, why do we need them? –Some technologies involved XML, SOAP, WSDL, UDDI. Putting it all together

grid computation and web services27 What the Web Can Offer Web sites offer a myriad of services. Buy books, check stocks, bank transactions… We would like to be able to do that automatically. Amazon could buy book from providers when it’s DB shows stock is low. Business would like to talk to each other. We need a ways to define and provide services.

grid computation and web services28 Some History Earlier attempts to B2B failed because HTML was not able to give enough semantic information about the services provided. In 2001 several companies got together with W3C and came up a precise set of XML based standards in order to help B2B.

grid computation and web services29 Some Standards WSDL Web Services Description Language. –An XML Schema that defined the service capabilities and how to invoke it. UDDI Universal Description, Discovery and Integration. – Registry specification on how to publish WSDL documents. WSIL Web Services Inspection Language. –How to find WSDL documents on a site. The idea is that these tools will help a user to find a WS and use it (automatically).

grid computation and web services30 A Possible Scenario USER WS Registry Amazon.com BN.com HarperCollins.com Buy books by isbn WS USER wants to buy books by ISBN, searches the registry using WSIL Registry (written in UDDI) contains WSDL documents describing the services Registry sends to user the matching services User checks prices offered by the services User chooses the best price and buys the book BN.com

grid computation and web services31 Using a WS There are two types of web services. –RPC (remote procedure call): the user calls a function provided by the service and gets a value. –DOC: the user invokes the service and gets a documents as a response. In order to use the service we need to know how to call it, i.e under which protocol is it working (TCP, SMTP, HTTP or other).

grid computation and web services32 Understanding a WS A WS is a collection of endpoints. An endpoint is a combination of a binding and an address (URI). A binding is a concrete protocol and data format for a port type The set of message exchanges is called an operation in WSDL terms: –error messages, encoding, parameters type, etc.

grid computation and web services33 Understanding a WS (2) Port types are interfaces (in wsdl 1.2 they are changing the term). Related concrete end points can be grouped in interfaces and thus provide abstract endpoints, also called services

grid computation and web services34 Application (resource) TCPHTTPSMTP Messages endpoints SERVICE interface operation interface operation

grid computation and web services35 SOAP Communication with the WS is done through messages, however we do not want to write the XML messages ourselves. SOAP simple object access protocol solves that. SOAP is simply a standard to send messages. There are others, but Microsoft, W3C and Sun use SOAP so it’s a de facto standard.

grid computation and web services36 SOAP (2) We send a message in a SOAP envelope. Using whatever protocol we like (FTP, HTTP, JMS,...) The SOAP processor is the one in charge of understanding the SOAP message and call the WS. There are Apache packages for java and Microsoft packages designed to use SOAP… we’ll see some examples.

grid computation and web services37 Client Application Firewalls HTTP XML Parser SOAP request XML Parser (SOAP translator) Server Application SOAP response SOAP (3)

grid computation and web services38 Using SOAP to Invoke a WS Barnes and Noble provides a service to get a books price with it’s ISBN –method name: getPrice(String isbn). –The service is located in the Web Server of Xmethods. Send though HTTP a SOAP message requesting the price of isbn = (Harry Potter 5).

grid computation and web services39 SOAP request <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPrice xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " The SOAP envelope header: information about the SOAP specification <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPrice xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPrice xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPrice xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " The information about the method invoked and the encoding type The parameter needed (isbn)

grid computation and web services40 <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPriceResponse xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " <SOAP-ENV:Envelope xmlns:SOAP-ENV= " xmlns:xsi=" instance" xmlns:xsd=" <ns1:getPriceResponse xmlns:ns1="urn:xmethods- BNPriceCheck" SOAP-ENV:encodingStyle= " SOAP answer The return value of the method

grid computation and web services41 Using JAVA with SOAP SOAP packages for JAVA give the possibility to: –Create SOAP envelopes. –Set the envelope data (URL, URI, method name, parameters,… –SEND the envelope. –GET the answer (also a SOAP envelope). –PARSE the envelope and get the relevant data.

grid computation and web services42 import java.net.URL; import java.util.Vector; import org.apache.soap.*; import org.apache.soap.rpc.*; public class Client { public static void main(String[] args) throws Exception { URL url = new URL(" ”BNQuoteService"); // Build the call. Call call = new Call(); call.setTargetObjectURI("urn:xmethods- BNPriceCheck"); call.setMethodName(”getPrice"); import java.net.URL; import java.util.Vector; import org.apache.soap.*; import org.apache.soap.rpc.*; public class Client { public static void main(String[] args) throws Exception { URL url = new URL(" ”BNQuoteService"); // Build the call. Call call = new Call(); call.setTargetObjectURI("urn:xmethods- BNPriceCheck"); call.setMethodName(”getPrice"); Java SOAP The URN of the method and the invoked method name

grid computation and web services43 call.setEncodingStyleURI(Constants.NS_URI_SOAP_ENC); Vector params = new Vector(); params.addElement(new Parameter("name", String.class, args[0], null)); call.setParams(params); // Invoke the call. Response resp = null; try { resp = call.invoke(url, ""); } catch( SOAPException e ) { System.err.println("Caught SOAPException (" + e.getFaultCode() + "): "+ e.getMessage()); System.exit(-1); } call.setEncodingStyleURI(Constants.NS_URI_SOAP_ENC); Vector params = new Vector(); params.addElement(new Parameter("name", String.class, args[0], null)); call.setParams(params); // Invoke the call. Response resp = null; try { resp = call.invoke(url, ""); } catch( SOAPException e ) { System.err.println("Caught SOAPException (" + e.getFaultCode() + "): "+ e.getMessage()); System.exit(-1); } Java SOAP (2) Setting args[0] as the isbn to look

grid computation and web services44 // Check the response. if( !resp.generatedFault() ) { Parameter ret = resp.getReturnValue(); Object value = ret.getValue(); System.out.println(value); } else { Fault fault = resp.getFault(); System.err.println("Generated fault: "); System.out.println (" Fault Code = " + fault.getFaultCode()); System.out.println (" Fault String = " + fault.getFaultString()); // Check the response. if( !resp.generatedFault() ) { Parameter ret = resp.getReturnValue(); Object value = ret.getValue(); System.out.println(value); } else { Fault fault = resp.getFault(); System.err.println("Generated fault: "); System.out.println (" Fault Code = " + fault.getFaultCode()); System.out.println (" Fault String = " + fault.getFaultString()); Java SOAP (3) Getting the returned value

grid computation and web services45 Agenda Grid computation. –Basic ideas. –Implementation problems. Web Services –What are they?, why do we need them? –Some technologies involved XML, SOAP, WSDL, UDDI. Putting it all together

grid computation and web services46 The Idea Use the WS standards to solve the implementation problems of a grid. WS are platform-independent and programming-language independent. Communication protocols are widely accepted (firewalls-friendly). WSDL offers a way to publish the services. UDDI offers a way to find the wanted service.

grid computation and web services47 Grid Services As WS (2) However there are grid needs that are not provided by the WS specifications and tools. We need new interfaces. –Grids need dynamic creation of application instances. (The application instance is created when the user needs it). –Grids need lifetime management. operations. –WS are usually stateless.

grid computation and web services48 Grid Factories The factory creates an instance of a new WS in some resource. –Dynamic creation of WS is supported is hosting environments as J2EE Server,.NET and AXIS. An application manager instance is created and publishes a WSDL document. The client can contact the new instance through the WSDL directly (we do not need the factory any more). –A messages channel is opened. Keep alive messages, destroy...

grid computation and web services49 Grid Factories (2) User Grid Factory Service Authentication Service Grid users DB Resource Broker Service Application Instance Service

grid computation and web services50 Lifetime Management Normally a transient service will be created and will run till it’s task termination. However errors can (and do) happen. Need a ways to kill idle WS that consume resources. A possible solution will be: –The service is created with a specific lifetime. –The user can request to extend the lifetime. –The user needs to send keep-alive messages, or explicitly kill it.

grid computation and web services51 User user DB Mining Factory Service Storage Factory Service Storage Service Instance Mining Service Instance Storage - The user invokes “create service” from the 2 factories. Storage room is allocated. - Keep Alive messages and status messages are sent- As no more keep-alive messages are received the services kill themselves freeing all their resources.

grid computation and web services52 The role of hosting environments A grid service architecture should be independent of the implementations issues. Nowadays most grid application rely in native OS (for example the creation of a new service means the creation of a process). This fact means for example that the factory service has to be platform-aware.

grid computation and web services53 The role of hosting environments (2) WS can be created with more sophisticated tools called hosting environments or containers as J2EE (Sun), WebSphere (IBM) or.NET (Microsoft) –These containers work in a component- based basis. Hosting environments build complex applications by using components that adhere to specific interfaces (i.e. EJB).

grid computation and web services54 The role of hosting environments (3) –Building applications which can have components distributed in different machines and places. –Communications are handled by the hosting environment. –Easy security. –Is not the component that handles communication or creation of instances but the hosting environment. –Messaging between components is also handled by the host no matter where the are.

grid computation and web services55 The role of hosting environments (4) Accountancy bean Clients DB Sales bean An accountancy- bean wants to get some data from the sales department DB. The bean asks the host for a handle to the sales-bean. The container looks in the directory where the bean is and returns a handle The beans communicate as if they where running in the same machine. Communications and security issues are managed from the host Container

grid computation and web services56 The role of hosting environments (5) This approach affects also the third level of users (scientific programmers). The programmer needs to follow specific interfaces for his application. In other words if he is a java programmer he needs to deliver beans.

grid computation and web services57 Conclusion WS provide a powerful tool to implement grid services. New interfaces (hopefully widely accepted) should be created to support specific grid needs: –Factories. –Lifetime management. –Security...

grid computation and web services58 References Articles. –The anatomy of the grid. –The physiology of the grid. –Programming the grid: distributed software components, P2P and grid web services for scientific applications. Grid services for distributed system integration.

grid computation and web services59 References (2) Some links –SOAP: –UDDI: –WDSL: –To see some WS in action: –Microsoft’s WS page: ult.aspx ult.aspx