Workflow in Grid Systems Workshop Dave Berry, Research Manager UK National e-Science Centre GGF10, Mar 2004.

Slides:

Advertisements

Similar presentations

Polska Infrastruktura Informatycznego Wspomagania Nauki w Europejskiej Przestrzeni Badawczej Institute of Computer Science AGH ACC Cyfronet AGH The PL-Grid.

Advertisements

LEAD Portal: a TeraGrid Gateway and Application Service Architecture Marcus Christie and Suresh Marru Indiana University LEAD Project (

Experiences with Converting my Grid Web Services to Grid Services Savas Parastatidis & Paul Watson

Abstraction Layers Why do we need them? –Protection against change Where in the hourglass do we put them? –Computer Scientist perspective Expose low-level.

Workflow languages and engines breakout Matthew Addis IT Innovation 5 December 2003 NeSC workshop on workflow services.

Open Grid Service Architecture - Data Access & Integration (OGSA-DAI) Dr Martin Westhead Principal Consultant, EPCC Telephone: Fax:+44.

Protocols in Workflow. Membership First session –Chair: John Brooke –Note Takers: Raj Bose, Mario Antonioletti –Geoff Lusted –Michael Burns Second Session.

European Workshop on Grid-based Virtual Organisations & collaborative e-Enterprise applications Toan NGUYEN May 30th, 2003 London (UK) Business models,

Nmrbox.org NMRbox: TRD 3 A probabilistic core as a coherent inference engine PINE+ Core Extend functionality through the new core PINE+: Assignment, use.

IBM Watson Research © 2004 IBM Corporation BioHaystack: Gateway to the Biological Semantic Web Dennis Quan

WS-VLAM Introduction presentation WS-VLAM Semantic tools Systems, Networking, and Engineering group Institute of informatics University of Amsterdam.

1 Richard White Design decisions: architecture 1 July 2005 BiodiversityWorld Grid Workshop NeSC, Edinburgh, 30 June - 1 July 2005 Design decisions: architecture.

EPrints Workshop, January eBank UK: Dissemination of research data using EPrints Simon Coles, School of Chemistry, University of Southampton.

A PPARC funded project AstroGrid’s Common Execution Architecture Guy Rixon, reporting on behalf of Paul Harrison and the other AstroGrid developers SC4DEVO-1,

John Kewley e-Science Centre GIS and Grid Computing Workshop 13 th September 2005, Leeds Grid Middleware and GROWL John Kewley

CoLaB 22nd December 2005 Secure Access to Service-based Collaborative Workflow for DAME Duncan Russell Informatics Institute University of Leeds, UK.

NextGRID & OGSA Data Architectures: Example Scenarios Stephen Davey, NeSC, UK ISSGC06 Summer School, Ischia, Italy 12 th July 2006.

Ischia, Italy July Session 1 First Part Monday 10 th July Malcolm Atkinson & David Fergusson.

A Semantic Workﬂow Mechanism to Realise Experimental Goals and Constraints Edoardo Pignotti, Peter Edwards, Alun Preece, Nick Gotts and Gary Polhill School.

Annual SERC Research Review - Student Presentation, October 5-6, Extending Model Based System Engineering to Utilize 3D Virtual Environments Peter.

Biology.sdsc.edu CIPRes in Kepler: An integrative workflow package for streamlining phylogenetic data analyses Zhijie Guan 1, Alex Borchers 1, Timothy.

Provenance in my Grid Jun Zhao School of Computer Science The University of Manchester, U.K. 21 October, 2004.

EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space The Capabilities of the GridSpace2 Experiment.

Database Taskforce and the OGSA-DAI Project Norman Paton University of Manchester.

Scientific Workflow reusing and long term big data preservation Salima Benbernou Université Paris Descartes Project.

INFSO-SSA International Collaboration to Extend and Advance Grid Education ICEAGE Forum Meeting at EGEE Conference, Geneva Malcolm Atkinson & David.

User requirements for and concerns about a European e-Infrastructure Steven Newhouse, Director.

Taverna and my Grid Basic overview and Introduction Tom Oinn

Managing Research Data – The Organisational Challenge at Oxford James A J Wilson Friday 6 th December,

CONTENTS Arrival Characters Definition Merits Chararterstics Workflows Wfms Workflow engine Workflows levels & categories.

Designing, Executing, Reusing and Sharing Workflows: Taverna and myExperiment Supporting the in silico Experiment Life Cycle Katy Wolstencroft Paul Fisher.

Taverna and my Grid Open Workflow for Life Sciences Tom Oinn

The ACGT Workflow Editing & Enactment Environment Giorgos Zacharioudakis Institute of Computer Science, Foundation for Research & Technology – Hellas (ICS-FORTH)

Distributed Aircraft Maintenance Environment - DAME DAME Workflow Advisor Max Ong University of Sheffield.

Web: OMII-UK Collaborations Workshop 2009 CSP Meeting 2008 courtesy of Martin Turner.

ILDG Middleware Status Chip Watson ILDG-6 Workshop May 12, 2005.

Large Scale Nuclear Physics Calculations in a Workflow Environment and Data Provenance Capturing Fang Liu and Masha Sosonkina Scalable Computing Lab, USDOE.

NA-MIC National Alliance for Medical Image Computing UCSD: Engineering Core 2 Portal and Grid Infrastructure.

ICCS WSES BOF Discussion. Possible Topics Scientific workflows and Grid infrastructure Utilization of computing resources in scientific workflows; Virtual.

Infrastructures for Social Simulation Rob Procter National e-Infrastructure for Social Simulation ISGC 2010 Social Simulation Tutorial.

GRID Overview Internet2 Member Meeting Spring 2003 Sandra Redman Information Technology and Systems Center and Information Technology Research Center National.

Cooperative experiments in VL-e: from scientific workflows to knowledge sharing Z.Zhao (1) V. Guevara( 1) A. Wibisono(1) A. Belloum(1) M. Bubak(1,2) B.

Uni Innsbruck Informatik - 1 Network Support for Grid Computing... a new research direction! Michael Welzl DPS NSG Team

Managing Documents the Right Way IA354 Amanda Murphy.

Using Federated Services with Triana Matthew Shields Cardiff University.

Remarks on OGSA and OGSI e-Science All Hands Meeting September Geoffrey Fox, Indiana University.

Edinburgh e-Science MSc Bob Mann Institute for Astronomy & NeSC University of Edinburgh.

Data Integration in Bioinformatics Using OGSA-DAI The BioDA Project Shirley Crompton, Brian Matthews (CCLRC) Alex Gray, Andrew Jones, Richard White (Cardiff.

The Globus eXtensible Input/Output System (XIO): A protocol independent IO system for the Grid Bill Allcock, John Bresnahan, Raj Kettimuthu and Joe Link.

Mike Hildreth DASPOS Update Mike Hildreth representing the DASPOS project 1.

11 Researcher practice in data management Margaret Henty.

1 e-Arts and Humanities Scoping an e-Science Agenda Sheila Anderson Arts and Humanities Data Service Arts and Humanities e-Science Support Centre King’s.

Toward a common data and command representation for quantum chemistry Malcolm Atkinson Director 5 th April 2004.

Applications and Requirements for Scientific Workflow May NSF Geoffrey Fox Indiana University.

Example projects using metadata and thesauri: the Biodiversity World Project Richard White Cardiff University, UK

Matthew Farrellee Computer Sciences Department University of Wisconsin-Madison Condor and Web Services.

EUROPEAN UNION Polish Infrastructure for Supporting Computational Science in the European Research Space The Capabilities of the GridSpace2 Experiment.

Application Web Service Toolkit Allow users to quickly add new applications GGF5 Edinburgh Geoffrey Fox, Marlon Pierce, Ozgur Balsoy Indiana University.

Holding slide prior to starting show. Lessons Learned from the GECEM Portal David Walker Cardiff University

Welcome Grids and Applied Language Theory Dave Berry Research Manager 16 th October 2003.

© Geodise Project, University of Southampton, Workflow Support for Advanced Grid-Enabled Computing Fenglian Xu *, M.

Chapter 7: Software Engineering

MIRACLE Cloud-based reproducible data analysis and visualization for outputs of agent-based models Xiongbing Jin, Kirsten Robinson, Allen Lee, Gary Polhill,

Provenance: Problem, Architectural issues, Towards Trust

Grid Portal Services IeSE (the Integrated e-Science Environment)

OGSA Data Architecture Scenarios

PPT1: Basics of software engineering

GGF10 Workflow Workshop Summary

Presentation transcript:

Workflow in Grid Systems Workshop Dave Berry, Research Manager UK National e-Science Centre GGF10, Mar 2004

Outline: Welcome Welcome Aims Programme “E-Science Workflow Services” Background Structure Issues arising

Aims Report on current work Find areas of agreement Identify open issues Look for opportunities Research Development Collaboration Outcomes Sample workflows CP&E special issue Input to RGs and WGs ?

Morning Session Talks Architecture Applications Possible panel topics: Scripting Security Debugging Constraint Modelling Discussion Issues

Afternoon session Talks Languages (esp. BPEL) Tools & Enactment Possible panel topics: Adaptive enactment Workflow inference Event-driven enactment Incorporating devices & human input Discussion Outcomes

Outline: “E-Science Workflow Services” Welcome Aims Programme “E-Science Workflow Services” Background Structure Issues arising

E-Science Workflow Services > 90 participants Industry UK e-Science International e-Science Organisers Dave Berry (NeSC) Savas Parastatidis (NEReSC) Written report In progress UK e-Science Series December 3-5, 2003

e-Science Institute

Speakers Industry WfMC, WS-Choreography UK e-Science MyGrid/Taverna, GeoDise, DiscoveryNet, DAME, ICENI, Planning, RealityGrid, JIGSA, OGSA-DAI, Triana, AstroGrid International e-Science Chimera/Pegasus, BIRN, Kepler/Ptolemy, Thetis, Narada Research Operational research, Workflow and VO’s

Breakout Sessions Scientific Workflow Requirements Carole Goble Protocols in Scientific Workflows John Brooke Workflow Languages and Engines Matthew Addis

User requirements Reflect the modelling paradigm of the scientist. Varies between experiments, disciplines Which user would that be? Creators, users, auditors, validators (I know if its right when I see it) Biologists cf. bioinformaticians, and transitioning between Different users, different environments Appropriate levels of abstraction. User models -> workflow models Simple to use & intuitive creation, deployment, execution and debugging environments

A Scientist Writes… “Work in my problem solving environment so that I don’t need to change the way I work.”

Scientific Workflow lifecycles Incrementally exploratory prototypes Got the data, now get the Nature paper before the next guy Large scale production Got the idea, now get the data for many experiments, teams, communities Migration from one to the other Capture of prototype for later non-interactive replay in a parameterised fashion Different parts of the lifecycle May use different environments and policies Different sorts of users will interact

User interaction Creation & Discovery By example, plagiarism, drag and drop Collaborative multi-user interaction in creation Reusing workflows -> modularisation Reusing workflows with different parameters and data Composing workflows from different areas, disciplines and scales “eXtreme team workflow creation” Single User interaction with workflow execution Choice between paths of execution in specific states Parameter modification mid-run Collaborative multi-user interaction during execution?

Characterising Scientific Workflows Very large amounts of data Files, streams, database queries GridFTP, http, ftp, sockets … Sometimes it's the computation that needs to be moved to the data Data model and types Metadata Provenance Drivers Scientific questions, outcomes and vanity More creators than users in science?

Enactment Stack

Workflow vs. Service “Perform” document Transformed result of query 2 Stored procedure Result of query 1 OGSA-DAI Service

Questions? Presentations from the workshop: contribution.cfm?Title=303