GGF10 Workflow Workshop Summary

Slides:



Advertisements
Similar presentations
Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.
Advertisements

Under the Hood of a Workflow Manager Matthew Shields, BiodiversityWorld GRID workshop, NeSC, 30 June - 1 July T r a ai n.
OMII-UK Steven Newhouse, Director. © 2 OMII-UK aims to provide software and support to enable a sustained future for the UK e-Science community and its.
Tahir Nawaz Introduction to.NET Framework. .NET – What Is It? Software platform Language neutral In other words:.NET is not a language (Runtime and a.
C. Grimme, A. Papaspyrou Scheduling in C3-Grid AstroGrid-D Workshop Project: C3-Grid Collaborative Climate Community Data and Processing Grid Scheduling.
Security Daniel Mallmann MWSG meeting Amsterdam December 2005.
Nadia Ranaldo - Eugenio Zimeo Department of Engineering University of Sannio – Benevento – Italy 2008 ProActive and GCM User Group Orchestrating.
6th Biennial Ptolemy Miniconference Berkeley, CA May 12, 2005 Distributed Computing in Kepler Ilkay Altintas Lead, Scientific Workflow Automation Technologies.
Lock Inference for Systems Software John Regehr Alastair Reid University of Utah March 17, 2003.
UvA, Amsterdam June 2007WS-VLAM Introduction presentation WS-VLAM Requirements list known as the WS-VLAM wishlist System and Network Engineering group.
UNICORE UNiform Interface to COmputing REsources Olga Alexandrova, TITE 3 Daniela Grudinschi, TITE 3.
The Open Grid Service Architecture (OGSA) Standard for Grid Computing Prepared by: Haoliang Robin Yu.
Computer Science 101 The Virtual Machine: Operating Systems.
EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space Cracow Grid Workshop’10 Kraków, October 11-13,
June Amsterdam A Workflow Bus for e-Science Applications Dr Zhiming Zhao Faculty of Science, University of Amsterdam VL-e SP 2.5.
January, 23, 2006 Ilkay Altintas
Data Management Kelly Clynes Caitlin Minteer. Agenda Globus Toolkit Basic Data Management Systems Overview of Data Management Data Movement Grid FTP Reliable.
OS provide a user-friendly environment and manage resources of the computer system. Operating systems manage: –Processes –Memory –Storage –I/O subsystem.
Workflow Project Luciano Piccoli Illinois Institute of Technology.
The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Dataset Caitlin Minteer & Kelly Clynes.
Semantic Web Fred: Project Objectives & SWF Framework Michael Stollberg Reinhold Herzog Peter Zugmann - 07 April
Peter Bajcsy, Rob Kooper, Luigi Marini, Barbara Minsker and Jim Myers National Center for Supercomputing Applications (NCSA) University of Illinois at.
© DATAMAT S.p.A. – Giuseppe Avellino, Stefano Beco, Barbara Cantalupo, Andrea Cavallini A Semantic Workflow Authoring Tool for Programming Grids.
Interoperability between Scientific Workflows Ahmed Alqaoud, Ian Taylor, and Andrew Jones Cardiff University 10/09/2008.
Composing workflows in the environmental sciences using Web Services and Inferno Jon Blower, Adit Santokhee, Keith Haines Reading e-Science Centre Roger.
Distributed System Concepts and Architectures 2.3 Services Fall 2011 Student: Fan Bai
NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.
ICCS WSES BOF Discussion. Possible Topics Scientific workflows and Grid infrastructure Utilization of computing resources in scientific workflows; Virtual.
Interactive Workflows Branislav Šimo, Ondrej Habala, Ladislav Hluchý Institute of Informatics, Slovak Academy of Sciences.
Enabling Grids for E-sciencE Astronomical data processing workflows on a service-oriented Grid architecture Valeria Manna INAF - SI The.
BPEL in Grids Aleksander Slomiski Department of Computer Science Indiana University
Remarks on OGSA and OGSI e-Science All Hands Meeting September Geoffrey Fox, Indiana University.
Workflow Optimisation Services for e-Science Applications David W. Walker Cardiff University.
Qusay H. Mahmoud CIS* CIS* Service-Oriented Computing Qusay H. Mahmoud, Ph.D.
1 Adventures in Web Services for Large Geophysical Datasets Joe Sirott PMEL/NOAA.
On Using BPEL Extensibility to Implement OGSI and WSRF Grid Workflows Aleksander Slomiski Presented by Onyeka Ezenwoye CIS Advanced Topics in Software.
OGCE Workflow and LEAD Overview Suresh Marru, Marlon Pierce September 2009.
David Adams ATLAS ATLAS Distributed Analysis and proposal for ATLAS-LHCb system David Adams BNL March 22, 2004 ATLAS-LHCb-GANGA Meeting.
© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.
Workflow Management Concepts and Requirements For Scientific Applications.
Copyright © 2004 R2AD, LLC Submitted to GGF ACS Working Group for GGF-16 R2AD, LLC Distributing Software Life Cycles Join the ACS Team GGF-16, Athens R2AD,
Chapter 1 Characterization of Distributed Systems
Applications Active Web Documents Active Web Documents.
Deployment of Flows Loretta Auvil
2. OPERATING SYSTEM 2.1 Operating System Function
The Open Grid Service Architecture (OGSA) Standard for Grid Computing
Spark Presentation.
Recap: introduction to e-science
Introduction to Operating System (OS)
Job workflow Pre production operations:
NGS computation services: APIs and Parallel Jobs
Datamining : Refers to extracting or mining knowledge from large amounts of data Applications : Market Analysis Fraud Detection Customer Retention Production.
University of Technology
SDM workshop Strawman report History and Progress and Goal.
Chapter 2: System Structures
Operating Systems Bina Ramamurthy CSE421 11/27/2018 B.Ramamurthy.
Module 01 ETICS Overview ETICS Online Tutorials
What is OGSA? GGF17 OGSA and Alternative Grid Architectures Panel
Introduction to Operating Systems
Large Scale Distributed Computing
Introduction to Operating Systems
Service Oriented Architecture (SOA)
CLARA . What’s new? CLAS Collaboration Meeting. March 6, 2019
A General Approach to Real-time Workflow Monitoring
Chapter 2 Operating System Overview
Operating Systems Structure
Gordon Erlebacher Florida State University
Status of Grids for HEP and HENP
New Tools In Education Minjun Wang
Overview of Computer system
Presentation transcript:

GGF10 Workflow Workshop Summary March 9 2004 Berlin The Organizers

Topics General Issues Application Requirements Language/User Interface Execution Engine (Run-time) 3 Grid Workflow Issues

General Issues Grain Size for “science” and efficiency of distributed service model Hierarchy (workflow of workflows) Data versus Control Security Metadata and Provenance Dynamic/Event based or Static Component Models/Architecture -- CCA, OGSI, WSRF. Web Services Error Handling (Detect, Specify action, Take action) Ease of Use (for real users not Grid hackers) Collaborative use by several users Open Source?

Application Requirements Time of running (seconds to months) People in loop Interactivity: real-time v batch Number of entities (10's to 100000's) Stream-based (communicate via pipes) OR Job-based (communicate via files) Spatial versus temporal interactions Multiple “workflow job” instances handled in or outside workflow

Language/User Interface Abstract versus High level (specification) versus low-level (“workflow virtual machine” ) Virtual Data Abstraction level Language: Kepler, Triana. BPEL WSCL WSCI ….. BPEL is inevitable? Diversity via Different “towers” in BPEL And/Or another language Does “other language” map to BPEL as low level interoperable Workflow VM Scripts: Perl, Python, Ant, Matlab, Specialized Petri Nets Functional Language specification Graphical UI Dataflow (stream) versus Control (message) model Web Service ports can be data and control?

Execution Engine (Run-time) Performance Robustness Support Streams and Messages Discovery of Services and Resources (computers, data repositories, networks) Support Scheduling/Planning of tasks and/or streams and/or data resources (“Towers”?) Support of Monitoring, Factories, Life-times etc Type checking Support Debugging Support "Workflow" (Computational) Steering Distributed versus centralized implementation

3 Grid Workflow Issues 1) Analyze issues such as dataflow, scheduling, virtual data, “science state” Map into WSRF and BPEL Correlation Identifiers/Extensibility or find “inadequacies”? 2) Look at scale and data size, data locality issues in science workflow What are implications for runtime engine? 3) Examine Semantic Grid (metadata/ provenance) issues for workflow 2) and 3) can be examined for both BPEL and “other approaches”