On Using BPEL Extensibility to Implement OGSI and WSRF Grid Workflows Aleksander Slomiski Department of Computer Science Indiana University

Indiana University Extreme! Lab Workflow WfMC defined a workflow as “The automation of a business process, in whole or parts, where documents, information or tasks are passed from one participant to another to be processed, according to a set of procedural rules” –This definition is document office automation oriented Workflow Management System (WFMS): “a system that defines, creates and manages the execution of workflows through the use of software, running on one or more workflow engines, which is able to interpret the process definition, interact with workflow participants and, where required, invoke the use of IT tools and applications”.

Indiana University Extreme! Lab Historical Perspective ’65-’75 Decompose Applications –Data And Code Separated ’75-’85 Database Management –DBMS Used To Share Data ’85-’95 User Interface Management –UIMS User Interface Separated ’95-’05 Workflow Management –Isolate Business Process “Workflow Management” Aalst, van Hee

Indiana University Extreme! Lab Categories Of Workflows CollaborativeProduction Ad HocAdministrative “Production Workflows” Leyman, Roller Repetition  Business Value  Scientific

Indiana University Extreme! Lab Orchestration and Web Services WSFL –IBM: Web Services Flow Language, May 2001 XLANG –Microsoft, May 2001 GSFL –Grid Services Flow Language, July 2002 WSCL / WSCI / W3C WS Choreography WG –HP WS Conversation Language, March 2002 –Web Service Choreography Interface, August 2002 BEA, SAP, Sun, Intalio BPEL4WS / OASIS WSBPEL –Replaces WSFL and XLANG, August 2002

Indiana University Extreme! Lab BPEL4WS OASIS WSBPEL group: BEA, Choreology Ltd, Collaxa, EDS, HP, IBM, Intalio, NEC, Novell, Microsoft, Oracle, SAP, Sun, Sybase, Workflow Management Coalition (WfMC), and many more... Unique mix of two different paradigms –XLANG: hierarchical structure with specialized control constructs –WSFL: graph structure with control patterns based on transition and join conditions.

Indiana University Extreme! Lab BPEL4WS Overview Specifies how to connect multiple web service to provide new web service The same language is defined to define executable and abstract process (contract) Executable process describes everything needed to execute workflow Abstract process describes required observable behavior of workflow based on message exchange (this allows to verify contracts between business partners) Provide support for basic Web Service activities: invoke, receive, reply Implicit lifecycle: workflow process instance is created when a message is marked as "start" and arrives to workflow engine

Indiana University Extreme! Lab Motivation: XML/Web/Grid Services Fusion Grid WS/XML Services Transport (ex. HTTP) Security (XML dsig.) RPC style WSRF Framework Type Mapping Future? WS-Stuff WSRF Notification Reliable Messaging

Indiana University Extreme! Lab Integration Goals Leverage Web Services Workflow Specification –BPEL4WS suitable with built-in extensibility Compatibility with BPEL 1.1 –Extend Not Replace! –Can execute BPEL 1.1 examples without any change Seamless integration with WS-RF (OGSI) –Grid/Semantic Web can use GPEL as “executable”

Indiana University Extreme! Lab Additional Goals Fully Observable State –Monitoring tools Steering: Dynamically Modifiable Modular extensibility –For example: module to work with XCAT components Easy to integrate inside portals

Indiana University Extreme! Lab Example Workflow Process Component Services (Create, Connect) CoG Services (security, GridFTP) Pub/Sub Messaging Grid Services (Grid Partners) Receive Invoke Reply Web Service WSRF Service Another BPEL Process Web Services (Partners)

Indiana University Extreme! Lab BPEL And OGSI Integration Leveraging BPEL abstractions –BPEL Partner is known by interface (PortType) –Very powerful integration mechanism! Understands natively GSH/Locator as service reference –Handle automatically GSH to GSR conversion Supports Explicit and Implicit Lifecycle –GPEL workflow engine is also OGSI Factory

Indiana University Extreme! Lab Issues in OGSI Integration GSH/GSR and WS-Addressing –Endpoint Reference (EPR) is not GSH or GSR … –Semantic differences: EPR has WS-Police, … BPEL and OGSI Factory –BPEL Engine acts as factory –BPEL has already implicit creation model BPEL Engine that is NOT OGSI Factory –Will not fit into common OGSI usage pattern –OGSI Factory is optional …

Indiana University Extreme! Lab WSRF support WS-Addressing (WSA) –Workflow Instance identification WSA is already used by BPEL … –No need to do conversions WS-Resource properties –Similar to OGSI SDEs WS-Notification –Monitoring workflow state

Indiana University Extreme! Lab Integration with Today Grid GridFTP, GSI Security, Condor, … Add set of pseudo partners (services) –No need to extend BPEL syntax! –They represent built-in functionality (handy!) –Can be implemented as real Web Services Will make easy to write workflow that is: –Working on Grid (CoG services) –Manipulating components (XCAT) –Large data transfers (GridFTP, RFT service) –… Similarly could have OGSI partners –Natively understands GSH/GSR/locator –Maintains soft state etc.

Indiana University Extreme! Lab Very Long Running Processes Some processes are short lived and some not –Long running simulations –Iterative Computation –Permanent searches Workflow Engine must be able to survive host restart –All state stored in database –Lends itself to scalability: per instance state is easy to share

Indiana University Extreme! Lab Synchronous vs. Asynchronous  –Typical binding is SOAP/HTTP Use WS-Addressing –User can specify its return address!!! –Receive connection can be one and closed! –Reply can be executed

Indiana University Extreme! Lab Reliability (Under Development) Use WS-ReliableMessaging –WS-RM enabled Workflow endpoint –Invoking external services with WS-RM –What if workflow engine is restarted? Allow to annotate partners –Designate operations as idempotent –Allow to specify exponential back-off retries

Indiana University Extreme! Lab Additional Issues Workflow Monitoring –How much state exposed as XML is needed? –Need a common framework to filter WS-Notification Topics? Fault Tolerance –Selection of alternative services –Dealing with “unexpected” exceptions Supporting Large Data and Streaming –Suitable metaphor (in BPEL)?

Indiana University Extreme! Lab Summary BPEL can be adapted to work in Grids –Key: deployment is not part of BPEL –However there are many levels of adaptation … There are issues beyond OGSI/WSRF –Supporting large data and streaming –Dealing with faults and error recovery –Monitoring and managing workflows –Workflow deployment –Security –…