GSFL: A Workflow Framework for Grid Services Sriram Krishnan Patrick Wagstrom Gregor von Laszewski

Outline Grid Workflow Requirements Technology Survey GSFL Overview Implementation Details Grid Workflow Example Current & Future Work Conclusions

Grid Workflow Requirements Description of the interactions between a set of cooperating Grid Services Presentation of an agglomeration of such Grid Services as a Grid Service Enabling recursive composition of Grid Services A A C C B B D D E E

Grid Workflow Requirements Ability to export entry points that will trigger a set of other activities Enabling peer-to-peer and decentralized data transfer between Grid Services A A B B Entry Point Peer-to-Peer Data Transfer 1 2 3

Grid Workflow Requirements Ability to manage the lifecycle for the individual services in the workflow All services that are part of the workflow need not be alive at the same time Need the ability to instantiate the services as and when necessary A A C C B B D D Time

Technology Survey Web Services Flow Language (WSFL) Flow Model describes the orchestration of the various activities in the workflow Global Model describes how these activities are implemented by the services PlugLinks describe the connection between WSDL operations having dual signatures Service Providers that can be located in a number of ways Static, Local, UDDI, Dynamic

Technology Survey WSFL Pros Exporting individual activities as part of the workflow Recursive composition of Web services WSFL Cons Inadequate peer-to-peer communication mechanism between services Due to the need for solicit-response and notification operations in WSDL The workflow engine has to mediate at every step, and can be a bottleneck Simplistic lifecycle management for the services No implementation available

Technology Survey XLANG Modeling of business processes as Autonomous Agents The unit of actions is a Service Process, which consists of a set of operations according to a defined sequence Sequential and parallel control flow constructs Transaction support Custom correlation of messages Flexible handling of exceptions Dynamic service referral Contracts to agglomerate services

Technology Survey XLANG Pros Defined as an extension to WSDL Availability of complicated control structures and exception handling XLANG Cons Does not enable exporting activities of individual services as part of the workflow Does not explicitly handle recursive composition of Web Services Relies on the ambiguous outgoing operations of WSDL Does not address lifecycle for individual services

Technology Survey Web Services Conversation Language (WSCL) Sequencing of interactions between operations of a single interface Document Type Descriptions Types of documents to be exchanged Interactions The actions of a conversation between participants Transitions Ordering relationships between interactions Conversations Set of interactions and transitions

Technology Survey WSCL Pros Fills the gap between mere IDLs that do not define any choreography and more complex flow languages that describe multi-party conversations WSCL Cons Does not define multi-party conversations Does not handle recursive composition of services Too simplistic for our goals

Technology Survey XCAT (Indiana University) Application Factories for Grid based Components Peer-to-Peer communication between components based on the CCA based Provides and Uses Ports Condor DAGMan Use of a DAG to describe inter-dependencies between jobs Does not deal with either Web or Grid based services

Grid Services Flow Language Language that allows specification of workflow for Grid Services in the OGSA framework Defined using an XML Schema Consists of : Service Providers, that are part of the workflow Activity Model, containing the activities that belong to the workflow Composition Model, describing interactions between the services Lifecycle Model, describing the lifecycle of activities and services

Service Providers List of services that are part of the workflow Name : Identifier for the service in the GSFL document Type : Name of the service, as specified in the WSDL for the service Locator : Information needed to locate the service Static : The service is up and running; the GSH is provided Factory : The service has to be instantiated; the GSH of the factory is provided Registry : The service is up and registered with a Registry Service; the GSH of the registry is provided

Activity Model List of activities that are part of the workflow Name : Identifier for the activity in the GSFL document Source : A reference to the operation which implements the activity The service that acts as the source The port that contains the operation The operation that implements the activity

Composition Model Describes the composition of individual services into a new Grid service Describes the flow of control and data between the various operations of the services Describes the direct communication between services in a peer-to-peer fashion Consists of : Export Model, that models the activities that have to be exported as operations of the workflow Notification Model, that models the direct communication between the services

Export Model List of activities to be exported as operations of the workflow process Each exported operation triggers a set of activities implemented by the individual services For the exported operations : Control Model defines the flow of control between activities of the workflow Data Model defines the flow of data between the individual activities Information to infer the messages and types for the exported operations

Export Model Service Providers : A, B Exported Operation : X Control Model : P -> Q, Q -> R Data Model : P -> Q A A B B Exported Port X P R Q Data Control

Notification Model Models the direct communication between the services Uses the Notification Sources & Sinks provided by OGSA Set of Notification Links connecting individual services, for particular topics Unlike WSFL, there is no need to use poorly defined solicit-response and notification operations

Lifecycle Model Models the lifecycle of the services and activities that are part of the workflow Service Lifecycle Ordering of services as a DAG, using Precedence Links All services need not be active at the same time Activity Lifecycle Possible ordering of the activities for the exported workflow, using Precedence Links Additional semantics to the exported activities, in a similar vein as WSCL

GSFL DEFINITION Name, Target Namespace, Scope IMPORTS List of Imports : Namespace, Location SERVICE PROVIDERS ACTIVITY MODEL List of Providers : Name, Type, Locator List of Activities : Name, Source COMPOSITION MODEL NOTIFICATION MODEL Notification Links Exported Activities Activity Info CONTROL MODELDATA MODEL Control In Control Links Data In, Data Out Data Links EXPORT MODEL LIFECYCLE MODEL Service Lifecycle Precedence Links Activity Lifecycle Precedence Links

Implementation Details Built on top of the OGSI Technology Preview Implementation of the Grid Services Specification Tools : AXIS,.NET, Tomcat, WSDL4J, Java CoG Important components : GSFL Parsing and data binding WSDL auto-generation for workflow instance Generic GSFL Coordinator service, which serves as the back-end

Implementation Details GSFL Parsing & Data Binding Use of Castor to auto-generate Java source from XML schemas Castor is an open source data binding framework for Java Beans generated for every element of the schema (appropriate get and set methods) Had to add wrappers to certain generated classes WSDL auto-generation for workflow instance All information needed for auto-generation is available in the GSFL document, and individual WSDLs Tools used : WSDL4J, Castor, Xerces

Implementation Details GSFL Back-end Every workflow instance is an instance of the generic GSFLCoordinator WSDL for the GSFLCoordinator is auto-generated Specialized GSFLProvider implemented as an extension to the OGSA RPCURIProvider The GSFLProvider intercepts all calls to the GSFLCoordinator, and dispatches them to the generic marshaler operation The marshaler operation of the GSFLCoordinator uses the GSFL information to route the calls to the appropriate services

Implementation Details Client Hosting Environment Client instantiates a workflow instance with a GSFL document Generic GSFL Coordinator GSFL Provider Client calls a method on the workflow instance Exported Virtual Port Marshaler Hosting Environment instantiates a workflow instance

Grid Workflow Example Service Providers : Job Queue Service Resource Manager Service Pool of Execution Services Activities : Queue/Dequeue : Job Queue Service GetResource : Resource Manager Service ExecuteJob : Execution Service

Grid Workflow Example Composition Model Export Model : Exported Operation - execute Identical Control and Data Models Dequeue : Get next job GetResource : Get the best resource for the job ExecuteJob : Execute the job on that resource Notification Model Each of the Execution services send resource information to the Resource Manager service using periodic asynchronous soft-state notifications Set of Notification Links connecting the Execution Services to the Resource Manager service

Grid Workflow Example Executio n Service Executio n Service Job Queue Job Queue Resource Manager Resource Manager Executio n Service Executio n Service Dequeue Queue Get Resource ExecuteJob Notification Link ExecuteSubmitJob

Current and Future Work This is a work in progress Not in release shape yet Continuous updates to keep up with upcoming technologies Investigate the applicability of BPEL4WS Addition of other features, such as : Exception handling Constructs for sequential and parallel control flow Integration of a graphical workflow editor to construct workflow instances visually

Conclusions GSFL provides a framework for defining workflow for Grid services within the OGSA framework Recursive composition of Grid services Effective peer-to-peer communication between the services Lifecycle management for individual services Leverages ideas from emerging Web service technologies More information :