© 2006 Open Grid Forum Workflow Management Research Group - WFM-RG OGF 22 Ian Taylor, Andrew Harrison and Ewa Deelman.

Slides:



Advertisements
Similar presentations
© 2006 Open Grid Forum GHPN-RG Status update co-chairss:Cees de Laat Dimitra Simeonidou GGF22, Boston, February 2008.
Advertisements

© 2006 Open Grid Forum JSDL 1.0: Parameter Sweeps OGF 23, June 2008, Barcelona, Spain.
© 2006 Open Grid Forum Network Services Interface OGF30: Connection Services Guy Roberts, 27 th Oct 2010.
© 2006 Open Grid Forum Ellen Stokes, IBM Michel Drescher, Fujitsu Information Model, JSDL and XQuery: A proposed solution OGF-19 Chapel Hill, NC USA.
© 2006 Open Grid Forum Network Services Interface Introduction to NSI Guy Roberts.
© 2006 Open Grid Forum JSDL 1.0: Parameter Sweeps: Examples OGF 22, February 2008, Cambridge, MA.
© 2006 Open Grid Forum GridRPC Interoperability Test Response to comments Yusuke Tanimura.
© 2006 Open Grid Forum OGF19 Federated Identity Rule-based data management Wed 11:00 AM Mountain Laurel Thurs 11:00 AM Bellflower.
© 2007 Open Grid Forum JSDL-WG Session OGF27 – General Session 10:30-12:00, 14 October 2009 Banff, Canada.
©2010Open Grid Forum OGF28 OGSA-DMI Status Chairs: Mario Antonioletti, EPCC Stephen Crouch, Southampton Shahbaz Memon, FZJ Ravi Madduri, UoC.
© 2006 Open Grid Forum JSDL Session (CIM Job) OGF 21 - Seattle, 17 October 2007.
© 2006 Open Grid Forum Joint Session on Information Modeling for Computing Resources OGF 20 - Manchester, 7 May 2007.
© 2007 Open Grid Forum JSDL-WG Session OGF21 – Activity schema session 17 October 2007 Seattle, U.S.
© 2006 Open Grid Forum 2 nd March 09 Enterprise Grid Requirements Research Group OGF25 EGR-RG Session Group.
Oct 15 th, 2009 OGF 27, Infrastructure Area: Status of FVGA-WG Status of Firewall Virtualization for Grid Applications - Working Group
© 2008 Open Grid Forum Resource Selection Services OGF22 – Boston, Feb
© 2006 Open Grid Forum Network Services Interface OGF29: Working Group Meeting Guy Roberts, 19 th Jun 2010.
Feb OGF22NML-WG: Welcome and Introduction Welcome and Introduction Paola Grosso - UvA Martin Swany - UDelaware.
© 2007 Open Grid Forum JSDL-WG Session 1 OGF25 – General Session 11:00-12:30, 3 March 2009 Catania.
© 2006 Open Grid Forum JSDL Optional Elements OGF 24 Singapore.
© 2007 Open Grid Forum Data/Compute Affinity Focus on Data Caching.
© 2007 Open Grid Forum OGSA-RUS Specification Update, Adoption and WS-RF Profile Discussions (Molly Pitcher) Morris Riedel (Forschungszentrum Jülich –
© 2006 Open Grid Forum Grid Resource Allocation Agreement Protocol GRAAP-WG working session 2 Wenesday, 17 September, 2008 Singapore.
© 2006 Open Grid Forum Joint Session on Information Modeling for Computing Resources (OGSA Modeling Activities) OGF 21 - Seattle, 16 October 2007.
© 2006, 2007 Open Grid Forum Michel Drescher, FujitsuOGF-20, Manchester, UK Andreas Savva, FujitsuOGF-21, Seattle, US (update) Extending JSDL 1.0 with.
© 2006 Open Grid Forum Network Services Interface OGF30: Working Group Meeting Guy Roberts, Inder Monga, Tomohiro Kudoh 27 th Oct 2010.
© 2009 Open Grid Forum Usage Record Working Group Alignment and Production Profile.
1 ©2013 Open Grid Forum OGF Working Group Sessions Security Area – FEDSEC Jens Jensen, OGF Security Area.
OGF26 Grid Information Retrieval Research Group May 26, 2008 Chapel Hill.
© 2006 Open Grid Forum DCI Federation Protocol BoF Alexander Papaspyrou, TU Dortmund University Open Grid Forum March 15-18, 2010, Munich, Germany.
© 2007 Open Grid Forum Data Grid Management Systems: Standard API - community development Arun Jagatheesan, San Diego Supercomputer Center & iRODS.org.
© 2006 Open Grid Forum Service Level Terms Andrew Grimshaw.
© 2010 Open Grid Forum Standards All Hands Meeting OGF28, München, March 2010.
© 2006 Open Grid Forum Network Services Interface OGF 32, Salt Lake City Guy Roberts, Inder Monga, Tomohiro Kudoh 16 th July 2011.
© 2007 Open Grid Forum Enterprise Best (Community) Practices Workshop OGF 22 - Cambridge Nick Werstiuk February 25, 2007.
© 2010 Open Grid Forum OCCI Status Update Alexander Papaspyrou, Andy Edmonds, Thijs Metsch OGF31.
© 2007 Open Grid Forum JSDL-WG Session OGF22 – General Session (11:15-12:45) 25 February 2008 Boston, U.S.
© 2006 Open Grid Forum BES 1.1 Andrew Grimshaw. © 2006 Open Grid Forum 2 OGF IPR Policies Apply “ I acknowledge that participation in this meeting is.
© 2006 Open Grid Forum FEDSEC-CG Andrew Grimshaw and Jens Jensen.
© 2006 Open Grid Forum Activity Instance Schema Philipp Wieder (with the help of the JSDL-WG) Activity Instance Document Schema BoF Monday, 25 February,
© 2006 Open Grid Forum Network Services Interface OGF 33, Lyon Guy Roberts, Inder Monga, Tomohiro Kudoh 19 th Sept 2011.
© 2015 Open Grid Forum ETSI CSC activities Wolfgang Ziegler Area Director Applications, OGF Fraunhofer Institute SCAI Open Grid Forum 44, May 21-22, 2015.
© 2006 Open Grid Forum HPC Job Delegation Best Practices Grid Scheduling Architecture Research Group (GSA-RG) May 26, 2009, Chapel Hill, NC, US.
© 2006 Open Grid Forum GridRPC Working Group 15 th Meeting GGF22, Cambridge, MA, USA, Feb
OGSA-RSS Face-to-Face Meeting Sunnyvale, CA, US Aug 15-16, 2005.
© 2006 Open Grid Forum Joint Session on Information Modeling for Computing Resources OGF 21, Seattle, Tuesday 16 October 2007.
© 2006 Open Grid Forum Network Services Interface CS Errata Guy Roberts, Chin Guok, Tomohiro Kudoh 29 Sept 2015.
© 2006 Open Grid Forum OGSA-WG: EGA Reference Model GGF18 Sept. 12, 4-5:30pm, #159A-B.
© 2006 Open Grid Forum Remote Instrumentation Services in Grid Environment Introduction Marcin Płóciennik Banff, OGF 27 Marcin Płóciennik.
© 2006 Open Grid Forum NML Progres OGF 28, München.
© 2007 Open Grid Forum OGF Management Area Meeting OGF20 7 May, am-12:30pm Manchester, UK.
© 2006 Open Grid Forum VOMSPROC WG OGF36, Chicago, IL, US.
© 2007 Open Grid Forum OGF20 Levels of the Grid Workflow Interoperability OGSA-WG F2F meeting Adrian Toth University of Miskolc NIIF 11 th May, 2007.
© 2008 Open Grid Forum Production Grid Infrastructure WG State Model Discussions PGI Team.
© 2007 Open Grid Forum JSDL-WG Session OGF26 – General Session 11:00-12:30, 28 May 2009 Chapel Hill, NC.
Network Services Interface
RISGE-RG use case template
GridRPC Working Group 13th Meeting
Grid Resource Allocation Agreement Protocol
OGF session PMA, Florence, 31 Jan 2017.
Sharing Topology Information
Network Services Interface
Network Services Interface Working Group
OGSA-Workflow OGSA-WG.
Network Measurements Working Group
WS Naming OGF 19 - Friday Center, NC.
Activity Delegation Kick Off
Network Services Interface Working Group
Introduction to OGF Standards
OGF 40 Grand BES/JSDL Andrew Grimshaw Genesis II/XSEDE
Presentation transcript:

© 2006 Open Grid Forum Workflow Management Research Group - WFM-RG OGF 22 Ian Taylor, Andrew Harrison and Ewa Deelman

© 2006 Open Grid Forum 2 OGF IPR Policies Apply I acknowledge that participation in this meeting is subject to the OGF Intellectual Property Policy. Intellectual Property Notices Note Well: All statements related to the activities of the OGF and addressed to the OGF are subject to all provisions of Appendix B of GFD-C.1, which grants to the OGF and its participants certain licenses and rights in such statements. Such statements include verbal statements in OGF meetings, as well as written and electronic communications made at any time or place, which are addressed to: the OGF plenary session, any OGF working group or portion thereof, the OGF Board of Directors, the GFSG, or any member thereof on behalf of the OGF, the ADCOM, or any member thereof on behalf of the ADCOM, any OGF mailing list, including any group list, or any other list functioning under OGF auspices, the OGF Editor or the document authoring and review process Statements made outside of a OGF meeting, mailing list or other function, that are clearly not intended to be input to an OGF activity, group or function, are not subject to these provisions. Excerpt from Appendix B of GFD-C.1: Where the OGF knows of rights, or claimed rights, the OGF secretariat shall attempt to obtain from the claimant of such rights, a written assurance that upon approval by the GFSG of the relevant OGF document(s), any party will be able to obtain the right to implement, use and distribute the technology or works when implementing, using or distributing technology based upon the specific specification(s) under openly specified, reasonable, non- discriminatory terms. The working group or research group proposing the use of the technology with respect to which the proprietary rights are claimed may assist the OGF secretariat in this effort. The results of this procedure shall not affect advancement of document, except that the GFSG may defer approval where a delay may facilitate the obtaining of such assurances. The results will, however, be recorded by the OGF Secretariat, and made available. The GFSG may also direct that a summary of the results be included in any GFD published containing the specification. OGF Intellectual Property Policies are adapted from the IETF Intellectual Property Policies that support the Internet Standards Process.

© 2006 Open Grid Forum 3 Full Copyright Notice Copyright (C) Open Grid Forum (2008). All Rights Reserved. This document and translations of it may be copied and furnished to others, and derivative works that comment on or otherwise explain it or assist in its implementation may be prepared, copied, published and distributed, in whole or in part, without restriction of any kind, provided that the above copyright notice and this paragraph are included on all such copies and derivative works. The limited permissions granted above are perpetual and will not be revoked by the OGF or its successors or assignees.

© 2006 Open Grid Forum 4 Boston OGF - OGF22 1:45 Session Overview (Taylor) Background on Research Document Past Efforts on interoperability 2:10 OGF Survey on interoperability (Harrison) Responses from community Conclusions 2:30 Discussion of Research Document The way forward Community involvement 3:00 Session Talks/Presentations Farrukh Nadeem - Characterizing, Modeling and Predicting resource availability in the Grid Focus - Research Document

© 2006 Open Grid Forum 5 Motivation for Research Document The Story so far … Focus on sharing and Interoperability Accept co-existing workflow representations/ environments How do we share/reuse workflows? Focus on the scientist performing the experiment How can sharing help him/her Workflow Interoperability Do we need this - use cases? Workflow embedding?

© 2006 Open Grid Forum 6 Motivation: Types of Sharing Intra domain for collaboration on the same workflow Inter domain - reuse of algorithms/services Provenance issues/living documents/logging/storage Sharing strategies for optimising/design of workflows Support full research cycles/iteration Search/discovery of services/workflows Exposing/organising research - workflows/papers/people Tagging/annotation

© 2006 Open Grid Forum 7 WFM-RG Past Efforts Related Meetings - background and further motivation for proposed research

© 2006 Open Grid Forum 8 Manchester - OGF 20 10:35 Levels of the Grid Workflow Interoperability - Adrian Toth, University of Miskolc. 10:45 "Workflow Optimization and Sharing Using Performance Information" - Omer Rana, Cardiff University 10:55 "Scheduling Data Intensive Workflows Onto Storage-Constrained Distributed Resources" - Rizos Sakellariou, University of Manchester 11:05 "myExperiment - social software for workflow sharing" - David De Roure, University of Southampton and OMII-UK 11:15 "The WHIP Plugin for Workflow and Artefact Sharing" - Andrew Harrison, Cardiff University 11:25 "Shibboleth Protection and Management of Workflows" - Richard Sinnott, National e-Science Centre, University of Glasgow 11:35 "A Workflow Mapping Mechanism for establishing Quality of Service Guarantees", Dimosthenis Kyriazis, Telecommunications Laboratory, National Technical University of Athens 11:35 Discussion on Document Focus Focus - Workflow Sharing

© 2006 Open Grid Forum 9 Seattle OGF - OGF21 9:00 Session Overview - Ian Taylor 9:05 Matt Shields – Sharing Workflows Recap of OGF 20 9:15 Research document use cases for workflow sharing, volunteers 9:20 Ewa Deelman NSF Workflow Interoperability workshop and what transpired in the execution breakout 9:35 Andrew Harrison Kepler/Triana integration - interoperability Use Case 1 9:50 Maurizio Melato--- Web Based Grid Workflow System: A- WARE project 10:05 Dave De RouremyExperiment 10:20 All---Future plan (Use cases documents) Focus - Workflow Interoperability

© 2006 Open Grid Forum 10 WORKS 07 The 1st (Paris) and 2 nd Workshop on Workflows in Support of Large-Scale Science (WORKS07) 2nd on 25th HPDC (Monterey, California) Deelman and Taylor, co-chairs Areas: Workflow Systems Adaptation and Integration Workflow Applications and Models Short Papers

© 2006 Open Grid Forum 11 Workshop on Scientific and Scholarly Workflow Cyberinfrastructure Baltimore, Maryland, Oct 3-5, funded by the National Science Foundation The goal of our workshop was to address the issue of interoperability, sustainability and platform convergence in scientific and scholarly workflow. Bring together key parts of the developer/user/supporter communities and see if there are concrete steps that can be taken to a shared, sustainable workflow cyberinfrastructure. Much discussion, current interoperability between systems is weak

Lots and lots of Workflow Systems 27 Chapters - from applications and environments BPEL, Taverna, Triana, Pergasus, Kepler, P-Grade, Sedna, ICENI, Java CoG Workflow, Condor, ASKALON, Swift, Petrinets, and so on … Successfully used - choice depends on requirements and politics :)

© 2006 Open Grid Forum 13 WFM-RG Current State-of-the-art Related past efforts Need more contribution from eScience community on sharing On-going See survey later

© 2006 Open Grid Forum 14 Past Efforts by the WfMC

© 2006 Open Grid Forum 15 WfMC - Types of Interoperability

© 2006 Open Grid Forum 16 WfMC - Types of Interoperability 2

© 2006 Open Grid Forum 17 Existing Workflow Standards Workflow Management Coalition (WfMC) identifies five functional interfaces, that connect a workflow management system with external application systems, of which Interface 4 deals with workflow interoperability, which defines abstract protocol for peer-to-peer interaction of workflow enactment services, potentially across business domain boundaries Supports scenario where an activity in a workflow process hosted by an enactment service is realized by a subprocess hosted by another enactment service. A typical example is an interenterprise workflow, such as those found in supply-chain interactions. Provides interoperability on the workflow-process level only; internals of a process instance need not be known across business domain boundaries.

© 2006 Open Grid Forum 18 WfMC Interface 4 Efforts SWAP (1998) - define an Internet-based workflow access protocol to instantiate, control, and monitor workflow process instances SWAP is a binding of the jointFlow object mode Slow adoption, community if favour of XML-based comms Wf-XML - SWAP successor (1999) interoperability standard that combines the elementary concept of SWAP with the abstract commands defined by the WfMC Interface 4 Defines an interoperability contract HTTP is core transport mechanism for Wf-XML messages

© 2006 Open Grid Forum 19 Wf-XML WfMC defined (2000) a binding of the standard, which uses asynchronous interaction via as the transport, with MIME encoding of the information to be exchanged. Asynchronous Messaging - loosely coupled Other bindings exist

© 2006 Open Grid Forum 20 WfMC Other Related Work - XPDL XML Process Definition Language Describes workflows at an abstract level Use-case - a meta language that acts as a bridge between proprietary languages understood by different engines. Benefits A many-to-one mapping between engines It exists no new specs A number of implementations Drawbacks Complex spec 30 pages of XML Schema Each translation to a workflow must be hand-crafted (by someone who understands XPDL and the engine specific language).

© 2006 Open Grid Forum 21 WfMC - Conclusions We should build on or reuse lessons learned and previous efforts They focus on the sub-process level (embedding) for interoperability Also, defining a meta language for modelling workflows (e.g. XPDL) This is the b2b case Are current eScience scientific workflow systems too different (heterogeneous components, different models of execution and so forth)

© 2006 Open Grid Forum 22 Questionnaire - eScience Input Gathering Data for research Document From eScience community experience on interoperability - requirements analysis We asked the community to contribute to issues surrounding workflow interoperability and sharing Had a handful of responses You can still contribute at:

© 2006 Open Grid Forum 23 Summary Feedback from developers and users of: GridBus Workflow Engine UNICORE Inforsense Triana Taverna Kepler BPEL

© 2006 Open Grid Forum 24 Interoperability Requirement for interoperability between workflow systems Comparing workflow systems with one another Embedding existing workflows that run in a different environment Using desktop interface to design in a chosen language Challenges to workflow interoperabiltiy Language used to describe Scientific Workflows Definition of workflows at an abstract level Semantics of workflow and data Security The term religious was mentioned more than once Benefits of interoperability Reuse A wider audience for workflow designers Achieving interoperability, two approaches idenitified Defining workflow schema at an abstract level Allowing workflow engines to be forked as a sub-process (embedding)

© 2006 Open Grid Forum 25 Sharing Requirement of sharing of workflows Reuse, modification, refining Challenges to workflow sharing Language used to describe Scientific Workflows Native engine enactment of shared workflows (embedding) Trust and security Workflow dependencies/time sensitivity Benefits of sharing Reuse Need driven approach to standardisation - sharing capability as an incentive to achieving interoperability Understanding what can be shared Faster discovery Achieving sharing Standards for semantics/description Community portal

© 2006 Open Grid Forum 26 Conclusions Projects do have a need for interoperability and sharing Not all projects, and not necessarily a need for both Primary reasons: Reuse Extending availability of existing work Primary routes: Embedding workflows as (distributed) sub-processes Shared abstract (meta) descriptions

© 2006 Open Grid Forum 27 Issues Shared meta-language (e.g. XPDL) is a good idea but… Need commitment from workflow developers to implement a binding I want to run a workflow defined in the meta-language Will it really run? Or does it require (aspects of) the environment in which it was initially defined? Depends on the nature of the workflow, e.g. just WS might be easier to port. But what if the client machine expects to receive notifications (i.e. can act as a server)? Introduces concept of modeling/bundling dependencies/runtime Reuse is about getting something for nothing But writing lots of bindings to a meta-language is not getting something for nothing

© 2006 Open Grid Forum 28 Issues Semantics How do I dermine what a workflow is/does? Is it semantically the same but functionally different from another? Even if I can embed a foreign workflow into my favorite tool, what about data mediation/massaging? Understanding/mapping inputs/outputs

© 2006 Open Grid Forum 29 Approaches Try and solve the world? Impose order on chaos Define how it should be and hope individual projects follow suit Try and solve small things? But within a wider context Fragmented, but not ignorant of other fragments. Start with what is there? Sharing e.g. myExperiment Execution e.g. BES, Atom Meta description e.g. XPDL

© 2006 Open Grid Forum 30 Discussion Community Effort? Wed be happy to lead a research document but need volunteers Need at least 5 volunteers not including ourselves … State-of-the-art of workflow interoperability and sharing use cases and proposals Area is vague at present Document would help firm up the area and push things forward.