Internet Technologies (Grid Computing (OGSA, WSRF) ) ICEN2202 PART 4 (Grid Computing (OGSA, WSRF) ) Abdullah Said Alkalbani alklbany@hotmail.com University of Buraimi

Open Grid service Architecture (OGSA) OGSA is a distributed interaction and computing architecture based around services, assuring interoperability on heterogeneous systems so that different types of resources can communicate and share information. OGSA has been described as a refinement of the emerging Web Services architecture, specifically designed to support Grid requirements OGSA some people may pronounce oog-sa A Revised Analysis of the Open Grid Services Infrastructure by Dennis Gannon, Kenneth Chiu, Madhusudhan Govindaraju, Aleksander Slominski OGSA

OGSA An open, service-oriented architecture (SOA) Dynamic service/resource creation and destruction Built on a Web services infrastructure Build grids from small number of standards-based components Customizable Support for dynamic, domain-specific content… …within the same standardized framework OGSA

Why Use an SOA? Logical view of capabilities Relatively coarse-grained functions Reusable and composable behaviors Encapsulation of complex operations Naturally extendable framework Platform-neutral machine and OS OGSA

SOA & Web Services: Key Benefits Flexible Locate services on any server Relocate as necessary Prospective clients find services using registries Scalable Add & remove services as demand varies Replaceable Update implementations without disruption to users Fault-tolerant On failure, clients query registry for alternate services Web Services Interoperable Growing number of industry standards Strong industry support Reduce time-to-value Robust development tools for Web services are available Decrease learning & implementation time Embrace and extend Leverage effort in developing and driving consensus on standards Focus limited resources on augmenting & adding standards as needed OGSA

Virtualizing Resources Access Web services Type-specific interfaces Computers Storage Sensors Applications Information Common Interfaces Create common interfaces through web services that allow us to interact with the logical resources based on their domains This will help us in reducing the complexity of managing heterogeneous resources Resource-specific Interfaces Resources OGSA

A Service-Oriented Grid Grid middleware services Job-Submit Service Registry Service Advertise Brokering Service Notify Virtualized resources CPU Resource Compute Service Data Service Application Service Printer Service OGSA

OGSA example user cases OGSA design is driven by a set of functional requirements, which themselves are informed by the use cases. These use cases cover infrastructure and application scenarios for both commercial and scientific areas. Examples are listed in the next slide OGSA

Use cases User cases Summary Commercial Data Center Manages thousands of IT resources, while reducing management costs and increasing resource utilization. Online Media and Entertainment Delivering an entertainment experience Grid Lite Extends the use of Grids to small devices—PDAs and cell phones. Virtual Organization Gives its members access to various computational, instrument-based data and other types of resources. Grid Portal Provides an end-user view of the collected resources available. IT Infrastructure Management Job execution, cycle sharing and provisioning scenarios. Application Use Cases Peer-to-Peer PC Grid computing, file sharing and content delivery scenarios. For full a detail see GFD-I.080:The Open Grid Services Architecture, Version 1.5 OGSA

Web services foundation OGSA Capabilities Execution Management Job description & submission Scheduling Resource provisioning Data Services Common access facilities Efficient & reliable transport Replication services Resource Management Discovery Monitoring Control OGSA Self-Management Self-configuration Self-optimization Self-healing Information Services Registry Notification Logging/auditing Security Cross-organizational users Trust nobody Authorized access only OGSA “profiles” Web services foundation OGSA

3. Select from or deploy required resources Execution Management The basic problem Provision, execute and manage services/resources in a grid Allow for legacy applications 2. Submit the job 3. Select from or deploy required resources Job 4. Manage the job 1. Describe the job JSDL CDL OGSA

Describing a Job Submission: JSDL Job Submission Description Language (JSDL) A language for describing the requirements of jobs for submission A JSDL document describes the job requirements Job identification information Application (e.g., executable, arguments) Required resources (e.g., CPUs, memory) Input/output files Job IT Infrastructure JSDL OGSA

Data Services The basic problem Some use-cases Issues to address Manage, transfer and access distributed data services and resources Some use-cases Replicating data for performance and reliability using high-performance data transfer Federating distributed data via a common access interface Managing file-based data and corresponding relational metadata Issues to address Many different data “types” and protocols Multiple possible use-cases, from high-energy physics to business How can we describe the data? How can we find the data? Where is the data needed? Federation = an organization formed by merging several groups or parties Federating distributed data = a union of data from multiple places. OGSA

Basic Interfaces of Data Services Storage Management Data Access Data Transfer Metadata catalog Replica management Cache management OGSA

Resource Management Domain-specific capabilities WSDM, WS-Management Provides a framework to integrate resource management functions interfaces, services, information models, etc. Enables integrated discovery, monitoring, control, etc. Application- specific Domain-specific capabilities OGSA High-level management services (GGF) Execution Management services Provide framework for managing service (monitoring , discovery) Data services Security services WSDM, WS-Management Access to manageability (OASIS, DMTF) WSRF/WSN, WS-Transfer/Eventing Resources Information models (DMTF, SNIA, etc.) OGSA

Self-Management Self-configuration: Automatically adapt to changes in the environment: e.g. Deploy/undeploy resources as load changes Self-optimization: Automatically tune system to best meet user or business needs Uses service-level agreements (SLAs) Self-healing: Automatically detect & correct problems Component failures Security violations etc. Self- Management Monitoring Projection Analysis Action Policy SLA How to monitoring resources? If I am not monitoring by myself, how can I configure the system to recover itself back the state that I want OGSA

Asynchronous notification Information Services Provide management and access facilities for information about applications and resources in the grid environment Information Services Execution management Resource reservation Problem determination Accounting Application monitoring Load balancing Service discovery Registry Asynchronous notification Producers There are many events happens in the Grid and several services require these information in order to demine how to provide their services. One example feature is that the information service allow these services to subscribe to only the information that they really want. Consumers Retrieval Reliable Secure Efficient Logger OGSA

OGSA security profiles Security Services Authorization, roles, and access privileges Locally (site) managed Based on SAML and XACML security standards Implementations provide credential mapping Working with GGF Security Area groups Authorization attributes for grids Developing OGSA security profiles PKI certificate WS-Security WS-Addressing OGSA security profiles OGSA

Specifications Landscape: April 2006 SYSTEMS MANAGEMENT GRID COMPUTING UTILITY COMPUTING Use Cases & Applications Distributed query processing Data Centre Collaboration Persistent Archive ASP Multi Media VO Management OGSA Self Mgmt OGSA-EMS WS-DAI Information WSDM Discovery GGF-UR WS-BaseNotification Naming Core Services Privacy Trust GFD-C.16 WSRF-RP WSRF-RL Data Model Web Services Foundation WSRF-RAP WS-Security SAML/XACML X.509 WS-Addressing HTTP(S)/SOAP WSDL CIM/JSIM Data Transport Hole Gap Evolving OGSA Standard

A Short Note about OGSI Open Grid Services Infrastructure (OGSI) is the standard that describe the implementation of the first version of OGSA However, the web service extension described in OGSI is not accepted by the Web service world Therefore, the new version of OGSA leads to the new implementation which is WSRF (Web Service Resource Framework) OGSA

Web Service and OGSA Web service architecture is the best option for implement the Grid. Usually, web service is “stateless”. This means that the Web service can’t "remember” information, or keep state, from one invocation to another. And… there are no standard way to implement a “stateful” web service. However, OGSA requires “stateful” web services. Something had to be done ! WSRF is the result of the effort to standardize the process of creating a stateful web service. However, when this new OGSA architecture is created, it is realized that they needed to choose some sort of distributed middleware on which to base the architecture on. In other words, if OGSA (for example) defines that the JobSubmissionInterface has a submitJob method, there has to be a common and standard way to invoke that method if we want the architecture to be adopted as an industry-wide standard. This base for the architecture could, in theory, be any distributed middleware (CORBA, RMI, or even traditional RPC). For reasons that will be explained further on, Web Services were chosen as the underlying technology. However, although the Web Services Architecture was certainly the best option, it still didn’t meet one of OGSA’s most important requirements: the underlying middleware had to be stateful Unfortunately, although Web services can in theory be either stateless or stateful, they are usually stateless and there is no standard way of making them stateful. So, clearly, something had to be done! OGSA

Web Service Invocation OGSA

Stateless Web Service This means that the Web service can’t "remember” information, or keep state, from one invocation to another. For example, imagine we want to program a very simple Web service which simply acts as an integer accumulator. This accumulator is initialized to zero, and we want to be able to add (accumulate) values in it. Suppose we have an add operation which receives the value to add and returns the current value of the accumulator. As shown in the figure, our first invocation of this operation might seem to work (we request that 5 be added, and we receive 5 in return). However, since a Web service is stateless, the following invocations have no idea of what was done in the previous invocations. So, in the second call to add we get back 6, instead of 11 (which would be the expected value if the Web service was able to keep state). OGSA

Stateful Web Service OGSA

WSRF Web Service Resource Framework A stateful web service State information is kept as Resource Resources have Properties that define state Properties can be add/remove/change dynamically Extends web service standards OGSA

Web Service and Resource More complex information can be include in resource too OGSA

Resource Representation Resource stores its information in Resource Properties (RPs) The structure of Resource is included in a WSDL file of a WSRF web service An instance of a Resource is stored as Resource Property Document OGSA

Resource Property Document <!-- RESOURCE PROPERTIES SECTION --> <xsd:element name="Value" type="xsd:int"/> <xsd:element name="LastOp" type="xsd:string"/> <xsd:element name="MathResourceProperties"> <xsd:complexType> <xsd:sequence> <xsd:element ref="tns:Value" minOccurs="1" maxOccurs="unbounded"/> <xsd:element ref="tns:LastOp" minOccurs="1" maxOccurs="unbounded"/> </xsd:sequence> </xsd:complexType> </xsd:element> WSDL <MathResourceProperties xmlns:tns= "http://www.globus.org/namespaces/examples/core/MathService_instance” <tns:Value>10</tns:Value> <tns:Value>30</tns:Value> <tns:Value>50</tns:Value> <tns:Value>40</tns:Value> <tns:LastOp>ADDITION</tns:LastOp> <tns:LastOp>SUBTRACTION</tns:LastOp> </MathResourceProperties> RP Document OGSA

WSRF Specification WSRF is a collection of specifications Core Related WS-ResourceProperties Specifies how resource properties are defined and accessed WS-ResourceLifetime Provides basic mechanisms to manage the lifecycle of resources WS-ServiceGroup Allow us to group different services together and access them through a single point of entry WS-BaseFaults Provides a standard way of reporting faults Related WS-Notification WS-Addressing WS-ResourceProperties A resource is composed of zero or more resource properties. For example, in the figure shown above each resource has three resource properties: Filename, Size, and Descriptors. WS-ResourceProperties specifies how resource properties are defined and accessed. The resource properties are defined in the Web service’s WSDL interface description. WS-ResourceLifetime Resources have non-trivial lifecycles. In other words, they’re not a static entity that is created when our server starts and destroyed when our server stops. Resources can be created and destroyed at any time. The WS-ResourceLifetime supplies some basic mechanisms to manage the lifecycle of our resources. WS-ServiceGroup We will often be interested in managing groups of Web Services or groups of WS-Resources, and performing operations such as ’add new service to group’, ’remove this service from group’, and (more importantly) ’find a service in the group that meets condition FOOBAR’. The WS-ServiceGroup specifies how exactly we should go about grouping services or WS-Resources together. Although the functionality provided by this specification is very basic, it is nonetheless the base of more powerful discovery services (such as GT4’s IndexService) which allow us to group different services together and access them through a single point of entry (the service group). WS-BaseFaults Finally, this specification aims to provide a standard way of reporting faults when something goes wrong during a WS-Service invocation. WS-Notification WS-Notification is another collection of specifications that, although not a part of WSRF, is closely related to it. This specification allows a Web service to be configured as a notification producer, and certain clients to be notification consumers (or subscribers). This means that if a change occurs in the Web service (or, more specifically, in one of the WS-Resources), that change is notified to all the subscribers (not all changes are notified, only the ones the Web services programmer wants to). WS-Addressing As mentioned before, the WS-Addressing specification provides us a mechanism to address Web services which is much more versatile than plain URIs. In particular, we can use WS-Addressing to address a Web service + resource pair (a WS-Resource). OGSA

WS-ResourceProperties Specifies common operations for interacting with resource properties (RPs) A web service can include these operations (portTypes) into its service to increase functionality As a result, a web service does not have to implement getter/setter-like method for manipulating its resources OGSA

WS-ResourceProperties portTypes GetResourceProperty Provides access to the value of any RPs GetMultipleResourceProperties Provides access to the value of multiple RPs at once SetResourceProperties Provides a uniform way to modify RPs such as update, insert, and delete QueryResourceProperties Allow us to perform queries on the RP document OGSA

Introduction to WSRF Questions so far? Introduction to WSRF What it is, why it was developed Relations to OGSI, OGSA Definitions

What it is? announced at GlobusWorld 04 by the Globus Alliance, IBM and HP WSRF is a set of five Web services specifications to model and manage state in a Web services context ResourceLifetime ResourceProperties BaseFaults RenewableReferences ServiceGroup ... which together with the Notification Spec retain all of the essential functional capabilities present in OGSI

Why it was developed? WSRF effectively completes the convergence of the Web service and Grid computing communities

Why it was developed? Criticisms of OGSI from the Web services community: Too much stuff in one spec => functionality partitioned into a family of composable specifications Does not work well with existing Web services tooling => WSRF tones down the usage of XML Schema Too object oriented: OGSI v1.0 models a stateful resource as a Web service that encapsulates the resource’s state, with the identity and lifecycle of the service and resource state coupled => WSRF makes an explicit distinction between the “service” and the stateful entities acted upon by that service

Relation from WSRF to ... OGSA: WSRF mechanisms will enable OGSA OGSI: WSRF restates OGSI concepts in WS terms OGSI WSRF Grid Service Reference (GSR) WS-Addressing Endpoint Reference Grid Service Handle (GSH) HandleResolver portType WS-RenewableReferences Service data elements (SDE) WS-ResourceProperties GridService lifetime managementt WS-ResourceLifeCycle Notification portTypes WS-Notification Factory portType Treated as a pattern ServiceGroup portTypes WS-ServiceGroup Base fault type WS-BaseFaults

Definitions in WSRF WS-Resource = Web Service + stateful resource which is used in the execution of message exchanges Stateful resource: Specific set of state data expressible as XML doc Well defined lifecycle Known to and acted upon by one or more web services Implied resource pattern = specific kind of relationship between web service and stateful resource Stateful resource implicit input for the execution of the message request (static or dynamic) Pattern means that relationship is codified by a set of conventions – in particular XML, WSDL and WS-Addressing

WSRF in detail WSRF Concepts in Detail how WS-Addressing is used have a closer look on the specs

Usage of WS-Adressing I Service Requestor <wsa:EndpointReference> <wsa:Address> http://someOrg.com/aWebService </wsa:Address> <wsa:ReferenceProperties> <tns:resourceID> C </tns:resourceID> </wsa:ReferenceProperties> </wsa:EndpointReference> Let us assume that the processing of the request resulted in the creation of the stateful resource “C.” We say that the Web service represents an explicit WS-Resource factory. It is a WS-Resource factory because the response message contains the endpoint reference of a WS-Resource which has been composed from the newly created stateful resource and its associated Web service. The stateful resource identifier must be unique within the scope of the Web service!!!! response request WS C B A

Usage of WS-Adressing II Service Requestor C <soap:Envelope> <soap:Header> <tns:resourceID> C </tns:resourceID> </soap:Header> <soap:Body> … some message </soap:Body> </soap:Envelope> ReferenceProperties component of a WS-Addressing message are processed in a binding-specific way. The WS-Addressing specification mandates that the endpoint reference must appear as part of any message sent to the Web service identified by the endpoint reference. Each type of WSDL binding must declare how child elements of the ReferenceProperties element must appear in messages using that binding. E.g. for SOAP must appear as SOAP header elements in the message message WS C B A

Resource-Lifecycle I Creation of a WS-Resource: The lifecycle of a WS-Resource is defined as the period between its instantiation and its destruction. Creation of a WS-Resource: trough any Web service capable of bringing one or more WS-Resources into existence response message typically contains at least one endpoint reference that refers to the new WS-Resource or places it into a registry for later retrival a message exchange is only considered a WS- Resource factory operation if it results in the actual creation of the WS-Resource referred to in the returned WSResource-qualified endpoint reference

Resource-Lifecycle II immediate destruction request message: <wsrl:DestroyRequest /> response message: <wsrl:DestroyResponse /> scheduled destruction mechanisms uses properties of the WS-Resource to query current time Determine current termination time

Resource Lifecycle III Setting initial termination Time via special XML element in the creation request message Requesting Change to Termination Time SetTerminationTimeRequest message Notification of Resource Destruction via subscription to topic ResourceTermination All time specifications are in UTC

Notifications I notification using a topic-based publication/subscription pattern standard set of message exchanges that define the roles of NotificationProducer and NotificationConsumer standard way to name and describe Topics

Notifications II Topic = categorize Notifications and their related NotificationMessage schemas part of the matching process The Web service provides an operation to allow requestors to retrieve the list of topics that it supports, e.g. goingOffline und SystemError A Subscribe C to SystemError WS with topics: goingOffLine SystemError Subscribe B to goingOffLine B C Producer Consumer

Notifications III A B Broker C Consumer msg1 msg2 Publisher A Publish msg1 to topic SystemError WS with topics: goingOffLine SystemError B msg1 C Broker msg2 Publish msg2 to topic SystemError Publisher B Consumer

Notifications IV Broker interface: Demand-based publishing: intermediary Web Service that decouples NotificationConsumers from Publishers Demand-based publishing: producing notifications may be costly Broker subscribes to the Publisher When no subscribers for the messages it pauses its subscription resumes when there are subscribers

Resource Properties I defines the type and values of a WS-Resource’s state that can be viewed and modified Resource properties document acts as a view on the actual state Described using XML Schema

Resource Properties II Defined Messages: GetResourceProperty GetMultipleResourceProperties SetResourceProperties Insert,update,delete QueryResourceProperties Using a query expression such as Xpath

Base Fault I Target: specifying Web services fault messages in a common way defines an XML Schema type for a base fault, along with rules for how this fault type is used

Base Fault II <BaseFault> <Timestamp>xsd:dateTime</Timestamp> <OriginatorReference> wsa:EndpointReferenceType </OriginatorReference> ? <ErrorCode dialect="anyURI">xsd:string</ErrorCode> ? <Description>xsd:string</Description> * <FaultCause>wsbf:BaseFault</FaultCause> * </BaseFault>

Service Groups I defines means by which WS can be grouped together for a domain specific purpose ServiceGroup is a WS-Resource, which represents a collection of other Web services MembershipContentRule: constraints on membership of the service group E.g. membership can be restricted to members that implement a particular interface no MembershipContentRule elements are specified, the members of the ServiceGroup are unconstrained. <wssg:MembershipContentRule MemberInterface="QName"? ContentElements="list of QName" />

Service Groups II ServiceGroupRegistration interface defines the message exchanges allow a requestor to add entries to a ServiceGroup (Add Operation) Notification of ServiceGroup Modification Topic ServiceGroupModification Notification Messages EntryAdditionNotification EntryRemovalNotification

Renewable Reference No specification yet! define mechanisms that can be used to renew an endpoint reference that has become invalid reference may contain not only addressing but also policy information concerning interactions with the service How? Decorating endpoint references with information necessary to retrieve a new endpoint reference

Globus WSRF Preview early preview of the Java WSRF Core implementation none of the higher-level services GT 4.0 based on WSRF should become available in Quartal 4 of 2004

Example I What is required to implement a new service? WSDL Service impl. Resource impl. ResourceHome Client Configuration/Installation

Example II – Counter Scenario

WSDL I - Properties <types> <xsd:schema targetNamespace="http://counter.com" xmlns:tns="http://counter.com" xmlns:xsd="http://www.w3.org/2001/XMLSchema"> … <xsd:element name="Value" type="xsd:int"/> <xsd:element name="CounterRP"> <xsd:complexType> <xsd:sequence> <xsd:element ref="tns:Value" minOccurs="1" maxOccurs="1"/> </xsd:sequence> </xsd:complexType> </xsd:element> </xsd:schema> </types>

WSDL II - Interface <portType name="CounterPortType" gtwsdl:implements="wsnt:NotificationProducer wsrl:ImmediateResourceTermination" wsrp:ResourceProperties ="tns:CounterRP"> <operation name="createCounter"> <input message="tns:CreateCounterRequest"/> <output message="tns:CreateCounterResponse"/> </operation> <operation name="add"> <input message="tns:AddInputMessage"/> <output message="tns:AddOutputMessage"/> </portType>

Service Implementation public _createCounterResponse createCounter(_createCounterRequest request) { ResourceContext ctx = null; CounterHome home = null; ResourceKey key = null; ctx = ResourceContext.getResourceContext(); home = (CounterHome) ctx.getResourceHome(); key = home.create(); EndpointReferenceType epr = AddressingUtils.createEndpointReference(ctx, key); _createCounterResponse response = new _createCounterResponse(); response.setEndpointReference(epr); return response; }

Service Implementation - add public int add(int arg0) throws RemoteException { Object resource = ResourceContext.getResourceContext().getResource(); Counter counter = (Counter) resource; int result = counter.getValue(); result += arg0; counter.setValue(result); return result; }

Resource Implementation public class PersistentCounter extends Counter implements PersistentResource { public void setValue(int value) { super.setValue(value); store(); } public Object create() throws Exception { Object key = super.create(); return key; public void load(ResourceKey key) throws ResourceException { …} public void store() throws ResourceException { … } public void remove() throws ResourceException { … }

ResourceHome public class CounterHome extends PersistentResourceHome { public ResourceKey create() throws Exception { Counter counter = (Counter)createNewInstance(); counter.create(); ResourceKey key = new SimpleResourceKey(keyTypeName, counter.getID()); this.resources.put(key, counter); return key; }

Conclusions WSRF refactors OGSA concepts WS-Resource: some parts are still missing Grid and Web communities can move forward on a common base WS-Resource: Web service that acts upon stateful resources