UvA, Amsterdam June 2007WS-VLAM Introduction presentation WS-VLAM Requirements list known as the WS-VLAM wishlist System and Network Engineering group Institute of informatics University of Amsterdam

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Introduction This presentation contains information requirements that have been driving the development of the WS- VLAM workflow The presentation summarize a long discussion sessions, we have had a couple of years ago (around 2004) with a number of application developers in VL-e. NOTE : Thanks for taking some of your time to read about the WS-VLAM work, any comments are more than welcome WS-VLAM developers

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Needed background … To be able to follow the content of this presentation you will need to know about:  GVLAM= workflow management system developed in the previous project. Details on the WS-VLAM can be found in the section of publication of the WS-VLAM home pageWS-VLAM

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Objective of this presentation… Why you should be interested in this presentations?  This presentation will give you an idea on the requirements, how they have been categorized, and how the support of these requirements have progressed from GVLAM to WS- VLAM

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Outline

UvA, Amsterdam June 2007WS-VLAM Introduction presentation The Requirements Functionality and Capability User interface characteristics Run time capabilities Software engineering aspects

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Functionality and Capability - (1) 1.Encapsulate complex procedures/in-house developed software for novice users  GVLAM: using the PFT and implemented using DBMS  WS-VLAM: same idea re-implemented Semantics & Ontologies 2.Enforce people to work in a certain way (“best practice”)  GVLAM: using the PFT and implemented using DBMS  WS-VLAM: same idea re-implemented Semantics & Ontologies 3.Access to DBMS; a service on(/from) which a workflow entity can store(/retrieve) data  GVLAM: using the PFT Viewer & VIMCO  WS-VLAM: Workflow can be semantically annotated and stored as complex services in a repository

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Functionality and Capability - (2) 4.Access to databases from workflow (storage/retrieval/querying)  GVLAM: using the PFT and implemented using a DBMS,  WS-VLAM: Not supported at this moment 5.Domain independent (biology, medical, etc.)  GVLAM: the PFT Editor & VIMCO are used for all the VL-e applications  WS-VLAM: Engine and the composer as well as the extensions are designed independently of any specific application 6.Platform independent (Linux, Windows, etc)  GVLAM: Workflow editing is platform independent, the workflow execution is only on Linux  WS-VLAM: workflow composer is platform independent, the workflow execution is only on Linux

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Functionality and Capability - (3) 7.Language independent (Python, C, C++, Java, etc.);  GVLAM: has interface for Python, C, C++, Java  WS-VLAM: has interface for Python, C, C++, Java 8.Integration of 3rd party software in a workflow (R, Matlab, VTK, ITK, FSL, etc.)  GVLAM: no support for wrapping Legacy applications  WS-VLAM: supports the execution of Legacy applications 9.Access to existing 3rd web-services  GVLAM: can be added if needed (planned for VL-e)  WS-VLAM: can automatically parse RPC Style WSDL and create workflow components

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Functionality and Capability - (4) 10.Typing mechanism for input/output data;  GVLAM: at composition exact match of the data type is required  WS-VLAM: implements a hierarchical data typing mechanisms, semantic annotation can be added to allow semantic data type checking 11.Fan-in (the input data can come from multiple entities) & fan-out (the output can be passed to multiple entities)  GVLAM: Fan-out supported, Fan-in not supported  WS-VLAM: Fan-out supported, can be support when REO connector will be implemented

UvA, Amsterdam June 2007WS-VLAM Introduction presentation User interface characteristics – (1) 12.User friendly (graphical, wizards etc.)  GVLAM: GUI requires improvement  WS-VLAM: More stable and completely decoupled from the execution, allow monitoring etc. 13.Encapsulation; the ability to create hierarchies of workflow)  GVLAM: Planned Half way implemented  WS-VLAM: support hierarchical workflow composition and distributed execution 14.Capture workflow, “provenance”  GVLAM: not automated, the user has to specify it.  WS-VLAM: Planned

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Run time capabilities - (1) 15.Execution of workflow, controlled (e.g. stepwise; useful in debugging)  GVLAM: Not supported  WS-VLAM: Not supported 16.Distributed execution (e.g. across a Grid of systems)  GVLAM: using GT2.4, and the gvlam-library  WS-VLAM: Using GT4, make use of standards services (delegation services, notification mechanises, etc) 17.Interactive, dynamic execution of workflow, Dynamic workflow  GVLAM: interaction is possible via the parameters at runtime. No for the dynamic execution of the workflow.  WS-VLAM: interaction is possible via the parameters at runtime. No for the dynamic execution of the workflow.

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Run time capabilities - (2) 18.Monitoring execution of workflow, gathering information on execution of workflow (metadata) (also from “inside” a workflow)  GVLAM: long term plan, to be developed  WS-VLAM: support monitoring using GT4 notification framework (WSRF notification ) 19.Maintain history/log of executed workflow for later scrutiny Reproduction of experiment  GVLAM: not automated, the user has to specify it  WS-VLAM: not automated, the user has to specify it 20.Maintain intermediate results?  GVLAM: not automated, the user has to specify it.  WS-VLAM: not automated, the user has to specify it

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Run time capabilities - (3) 21.Checkpointing; both data and process  GVLAM: Not supported  WS-VLAM: Not supported 22.“nohup” execution (being able to execute a workflow “in the background”, without having to be logged in all the time)  GVLAM: long term plan  WS-VLAM: supported using Detach/re-attach feature using GT4 returned EPRs 23.Control flow (while/for/if-then-else, parallel/sequential/recursion)  GVLAM: Not supported  WS-VLAM: Not supported, can be support when REO connector will be implemented

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Run time capabilities - (4) 24.Job Farming and parameter Sweep  GVLAM : Only for one workflow element via Nimord  WS-VLAM: Under development 25.High performance execution  GVLAM : workflow can be distributed all over the Grid nodes and allow data streaming.  WS-VLAM: workflow can be distributed all over the Grid nodes and allow data streaming 26.Resource brokering; given the description of resources required by a workflow entity and the description of abilities provided by a resource: the (automatic) brokering of and entity onto a resource  GVLAM: limited  WS-VLAM: limited

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Software engineering aspects – (1) 27.Compliance to standards on all levels  GVLAM: Partially  WS-VLAM: WSRF standards 28.Based on well-established standards (i.e. Grid software, easy to install, maintain)  GVLAM: Partially  WS-VLAM: WSRF standards 29.Quality-of-Service: fault tolerant, stable, high availability, dependable  GVLAM: limited  WS-VLAM: limited

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Software engineering aspects – (1) 30.Software engineering: maintainability of dependency on 3rd party software  GVLAM: Rudimentary  WS-VLAM: Rudimentary 31.Open source  GVLAM : Database is not open source, it can be replaced  WS-VLAM: uses only Open source software 32.OWL/Ontology, semantic annotation  GVLAM: Not supported  WS-VLAM: semantic extensions are being tested at this time

UvA, Amsterdam June 2007WS-VLAM Introduction presentation Summary The whish list contains 31 whishes, the GVLAM and WS-VLAM scores:  GVLAM (13 Yes)  WS-VLAM (21 Yes)  GVLAM (11 No)  WS-VLAM (5 No)  GVLAM (7 partially)  WS-VLAM (5 partially) Ongoing activities WS-VLAM will probably need to support 4 other requirements (requirements number 4, 11, 14, and 30)

UvA, Amsterdam June 2007WS-VLAM Introduction presentation