Lightweight Grid Computing Worksop 2 nd May 2006, Losehill Hall, Derbyshire Requirements and Expectations from Workflows Asif Akram e-Science Grid Technology Group

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire WOSE Project  Workflow Optimisation for Services in e-Science  EPSRC funded project  In collaboration with Imperial College and Cardiff University  CCLRC is investigating the user requirements  Developing Use Cases based on existing e- Science Application  e-HTPX: An e-Science Resources for High Throughput Protein Crystallography

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Best Practices for Workflows  Modular Design  Exception Handling  Compensation Mechanism  Adaptive Workflow  Flexible Workflows  Management of Workflow

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Modular Design  Thorough investigation of different activities  Group similar or related activities  Group activities with minimum side effects  Module components should be scalable  Module components can be replaceable  Reliable to serve as repeatable units  Modules operate as “black boxes” in the overall workflow, with their own variables, computational logic, dependency constraints, and event handlers.

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Exception Handling Exceptions can be due to  inconsistent data,  divergence of tasks from the workflow model,  unexpected contingencies and  un-modeled changes in the environment.  Type of Exceptions: “expected exceptions” (“variations”) and “unexpected exceptions”  Exception Handlers: Global Exception Handlers, Scoped Exception Handlers and Inline Exception Handlers.  Exception handling should be localized

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Compensation Mechanism  “undoing steps in the business process that have already completed successfully”  Reverse the effects of successful activities in the abandoned workflow  Un-handled Exceptions requires compensation  Compensation behavior must be defined by the services to ensure reversal of a operation.  Synchronous operations (i.e. a single connection request/reply) have short lifetimes and do not require compensation to release resources.  Asynchronous communications requires some sort of compensation (unluckily we have normally RPC style Web Services)

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Adaptive Workflow  Adaptive nature of a workflow can be static or dynamic depending on whether changes are accommodated at design time or runtime respectively.  Redesign and optimization of a process to gain greater efficiency and effectiveness requires adjustment in the workflows; in fact the final goal is to constantly improve the workflow/ process by evolutionary redefinition.  Modifications break a pre-defined process in an attempt to solve the problem in better way.  Dynamically adaptive workflows adjust at runtime; i.e. due to unexpected results

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Flexible Workflows  Flexibility is required during the early stages of a workflow design when services are un-stable.  Flexibility in the data types and service endpoint.  Flexibility in data, is achieved by applying generic “base” or “parent” data types  Flexibility in service endpoint, requires integration of additional and re-located services through WS- Addressing.  Potential callouts and logic for selecting partner service has to be built into the business process as external configuration file.  Assuming all the services have “similar” interfaces which may not always be the case.

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Management of Workflow  Workflow management involves the addition of  Breakpoints arbitrary location crucial to the process  Steering capabilities generalizes breakpoints and involves reset, re-schedule, restart, undo, abort, complete, recover, ignore or jump operations.  Persistence of state for fault tolerance, to allow re- execution with different experimental data sets, and to facilitate inspection of intermediate data values  Monitoring for optimization and analysis of internal state, data flow, inspection of values, re-scheduling of tasks and re-ordering of tasks

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Portals & Web Services Internet Repository for Authorisation and Policies Workflow Service End-points Clients

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Business Process Execution Language  Builds on top of XML and Web Services  Abstract, which allow specification of the public message exchange between participating services and doesn’t include the internal details of process flows.  Executable, which allows specification of the exact details of a workflow. Normally BPEL is used for executable processes.  Can utilize other Web Services specification for reliable messages, transactions etc

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire BPEL  Provides different types of activities  Primitive activities can be structured in Modules  BPEL support different types of structured activities i.e. “sequence”, “flows” and “scope”  Extensive support for Exception handling  Can trigger Events from “Exception Handler”  Compensation Handlers can be called from Exception Handler  Limited monitoring is possible through different primitive activities like “onMessage”, “onAlarm” and “pick”

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire BPEL Extension  Different implementation has varying level of extensions  Oracle BPEL engine supports “notification”  Oracle supports “NFlow” for parallel execution of any module “n times”  Oracle has proprietary management capabilities and user interaction functionality  Engines provides API to query and interact with process at runtime  Proprietary API to populate SOAP Headers  Limited support for WS-Security  Support for WSIF and EJB

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Limitations of BPEL  Limited reusability of primitive and structured activities  Can’t re-execute activities defined earlier  or events are not to modify the workflow at runtime  BPEL specification does not explicitly support user interactions and notifications  Difficult to integrate security without support from Vendors  BPEL is fairly complex specification

Presenter Name Facility Name Lightweight Grid Computing Workshop, 2 nd May 2006, Losehill Hall, Derbyshire Future Work  Moving to Document Style Web Services  Evaluating WSRF for e-HTPX  e-HTPX mainly passes URL of images in different services (candidate for Resource Properties)  Moving EPR across services is more efficient  Built in support for Notifications  Support for persistence (flat files but can be extended for DB); e-HTPX is using Hibernate  WS-Core supports XML Database Xindice  Support for different level and type of security