1. An Overview of Scientific Workflows: Domains & Applications Laboratoire Lorrain de Recherche en Informatique et ses Applications Presented by Khaled Gaaloul Environments COOperation

2. Plan 1 I. Context & Problematic II. State of Art III. In Progress IV. Conclusion & Perspectives

3. I. Context & Problematic

4. 2 Context: Scientific applications Need of WFMS for the orchestration and optimization of the scientific endeavors. Collecting, generating and analyzing of a large data flow Need of mechanisms supporting interactions between heterogeneous applications Context & Problématic State of ArtIn Progress Conclusion & Perspectives

5. 3 Context: Scientific applications integration Context & Problematic State of ArtIn progress Conclusion & Perspectives Step1 Step2 ANDAND Labo.2 Labo.3 Labo.4 Definition & specification of processes Data flow managing Process orchestration Step5 Step4 Step3 Step6 XORXOR ANDAND Labo.1 Dynamic Scheduling of a Scientific Process

6. 4 Prerequisites for scientific applications High flexibility degree High-performance for resources distribution Workflow ad hoc architecture: moving and hierarchical Data flow Management: - Automate data streaming - Enriching the semantic level - Documentation & reutilisability Context & Problematic State of ArtIn progress Conclusion & Perspectives

7. 5 Problematic: How to optimize and orchester scientific processes execution? Problems in managing shared resources: heterogeneous environment, virtual organizations (VO), etc. Moving Applications: Non-determinism aspect Current approaches: lack of reutilisability and documentations, business process oriented Evolution format within data exchanges Context & Problematic State of ArtIn progress Conclusion & Perspectives

8. 6 Problematic: New requirements Context & Problematic State of ArtIn progress Conclusion & Perspectives Designers Step1 Step2 ANDAND Step5 Step4 Step3 Step6 XORXOR ANDAND sub process1 sub process2 sub process3 To deal with heterogeneity To deal with data exchange

9. State of Art

10. 7 Scientific workflow Definition: the application of workflow technology to scientific endeavors, recognized as a valuable approach for assisting scientists in accessing and analyzing data. Features: - Support for large data flows; - Dynamic environment; - Incomplete workflow: partial definition; - Ad hoc planning; - Reutilisabilty, documentation, etc. Context & ProblematicState of ArtIn progress Conclusion & Perspectives ScientificWorkflow GRID PBIO

11. 8 Scientific Workflow Scientific domain: dedicated to the data flow managing More dynamic: non predefined workflow Traceability and documentation: enriching the semantic level within data exchanges Business Workflow Business domain: dedicated to the processes managing and optimization Lot of constraints: predefined workflow, satisfying end, execution constraints, etc. Lack of formalism: Syntactic level Context & ProblematicState of ArtIn progress Conclusion & Perspectives Scientific Workflow GRID PBIO

12. 9 Scientific Workflow Vs Business Workflow Context & ProblematicState of ArtIn progress Conclusion & Perspectives Scientific Workflow GRID PBIO

13. 10 Solution for intensive computing Virtual organization (VO) - including different users committees - sharing global resources (storing, processing) - Strong impact on organization structure, networks, security Context & ProblematicState of ArtIn progress Conclusion & Perspectives ScientificWorkflow GRID PBIO GRID (Globalization of Informatics' Resources and Data)

14. 11 GridFlow (1) : GRID and Workflow? GRID complexity - Virtual organization - Needs of visualization, managing, and simulation WfMS as a Grid service - Transparent access to one or many GRID regrouping heterogeneous machines - Portals for users Context & ProblematicState of ArtIn progress Conclusion & Perspectives Scientific Workflow GRID PBIO

15. 12 GridFlow (2): Architecture Context & ProblematicState of ArtIn progress Conclusion & Perspectives Scientific Workflow GRID PBIO

16. 13 PBIO: or how to deal with format evolution? Heterogeneous environment, ad hoc solutions - Data exchanges and complex communication - Format evolution: lack of standardization of data streaming PBIO (Portable Binary Input/Output) - Approach to deal with binary data in storage and transmission - Record oriented binary communication mechanism - Data meta-representation - Optimizing data storage/transmission - Improving the communication between processes Context & ProblematicState of ArtIn progress Conclusion & Perspectives Scientific Workflow GRID PBIO

17. In Progress

18. 14 Cooperative processes for scientific workflows Cooperation between applications - Applications more flexible - Working and communicating within the same virtual space of work - Doing common tasks in synchronous or asynchronous way BONITA: a flexible system for cooperative workflow - Define, specify, execute, and coordinate different flows of work - Based on the anticipating model - Ensure an interface for the modeling and the visualization of the processes - Managing flexible data Context & ProblematicState of ArtIn progress Conclusion & Perspectives

19. 15 Motivating Example: Numerizing scenario Context & ProblematicState of ArtIn progress Conclusion & Perspectives

20. 16 Deploying the scenario into Bonita Enhance execution flexibility Anticipation: process optimizing Context & ProblematicState of ArtIn progress Conclusion & Perspectives

21. 17 Mapping Data-Intensive Science into BONITA Considerable data flows Goal: Optimize the data streaming & enhance the data exchange mechanism Context & ProblematicState of ArtIn progress Conclusion & Perspectives Data flow computing

22. 17 Discussions Existing approach: Flow-Based Programming (FBP) - A new/old approach to scientific application development - Data flow Vs. Workflow: which one fit to us? - Anticipating an activity, is it possible with a partial result? PBIO implementation - Interactivity with Bonita services call - Need of middleware like Echo Event to support messages exchange - Portability of the PBIO approach for existing platforms Context & ProblematicState of ArtIn progress Conclusion & Perspectives

24. Conclusions: Cooperative aspect for scientific applications Combining strong concepts (GRID & workflows) Developing a new middleware for scientific process Perspectives: Application onto the GRID: Bonita as a GRID service Adding Non intrusive and user friendly aspects Collaboration with AURARYD on others scenarios (Volkswagen, BP) 18 Context & ProblematicState of ArtIn progress Conclusion & Perspectives