:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 1 :: Workflows and HPC? :: The relation between workflows.

Slides:



Advertisements
Similar presentations
Prepared 7/28/2011 by T. O’Neil for 3460:677, Fall 2011, The University of Akron.
Advertisements

ARCH-05 Application Prophecy UML 101 Peter Varhol Principal Product Manager.
Communication between modules, cohesion and coupling
Design Concepts and Principles
Organization and Environment Upul Abeyrathne, Senior Lecturer in Political Science, University of Ruhuna, Matara.
Solutions to Review Questions. 4.1 Define object, class and instance. The UML Glossary gives these definitions: Object: an instance of a class. Class:
Project Work and Internship Impacts on Labour Market and Society OPEN DISCUSSION FORUM Project Work and Internship Impacts on Labour Market and Society.
Object-Oriented Analysis and Design
GridFlow: Workflow Management for Grid Computing Kavita Shinde.
© 2005 Prentice Hall12-1 Stumpf and Teague Object-Oriented Systems Analysis and Design with UML.
Improving Robustness in Distributed Systems Jeremy Russell Software Engineering Honours Project.
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 8 Slide 1 System models September 29, 2008.
Modified from Sommerville’s originalsSoftware Engineering, 7th edition. Chapter 8 Slide 1 System models.
July 13, “How are Real Grids Used?” The Analysis of Four Grid Traces and Its Implications IEEE Grid 2006 Alexandru Iosup, Catalin Dumitrescu, and.
1 Computer Science, University of Warwick Architecture Classifications A taxonomy of parallel architectures: in 1972, Flynn categorised HPC architectures.
Course Instructor: Aisha Azeem
New Challenges in Cloud Datacenter Monitoring and Management
Mobility in the Virtual Office: A Document-Centric Workflow Approach Ralf Carbon, Gregor Johann, Thorsten Keuler, Dirk Muthig, Matthias Naab, Stefan Zilch.
WORKFLOWS IN CLOUD COMPUTING. CLOUD COMPUTING  Delivering applications or services in on-demand environment  Hundreds of thousands of users / applications.
Resource Fabrics: The Next Level of Grids and Clouds Lei Shi.
Objective Explain What is the Balanced Scorecard
Object-Oriented Analysis and Design
Dr Tina Balke Dr Thomas Centre for Research in Social.
DISTRIBUTED COMPUTING
CONTENTS Arrival Characters Definition Merits Chararterstics Workflows Wfms Workflow engine Workflows levels & categories.
Lecture 9: Chapter 9 Architectural Design
©Ian Sommerville 2000 Software Engineering, 6th edition. Chapter 10Slide 1 Architectural Design l Establishing the overall structure of a software system.
SOFTWARE DESIGN AND ARCHITECTURE LECTURE 09. Review Introduction to architectural styles Distributed architectures – Client Server Architecture – Multi-tier.
High Level Architecture Overview and Rules Thanks to: Dr. Judith Dahmann, and others from: Defense Modeling and Simulation Office phone: (703)
Meta Scheduling Sathish Vadhiyar Sources/Credits/Taken from: Papers listed in “References” slide.
Travis Steel. Objectives What is the Agent Paradigm? What is Agent-Oriented Design and how is it different than OO? When to apply AOD techniques? When.
What are the main differences and commonalities between the IS and DA systems? How information is transferred between tasks: (i) IS it may be often achieved.
20 October 2006Workflow Optimization in Distributed Environments Dynamic Workflow Management Using Performance Data David W. Walker, Yan Huang, Omer F.
SOFTWARE DESIGN.
A Novel Approach to Workflow Management in Grid Environments Frank Berretz*, Sascha Skorupa*, Volker Sander*, Adam Belloum** 15/04/2010 * FH Aachen - University.
Software Design Deriving a solution which satisfies software requirements.
Issues in (Financial) High Performance Computing John Darlington Director Imperial College Internet Centre Fast Financial Algorithms and Computing 4th.
Chapter 6 Architectural Design.
GRASP: Designing Objects with Responsibilities
©Ian Sommerville 2004Software Engineering, 7th edition. Chapter 14 Slide 1 Object-oriented Design.
SmartNets Results Overview SmartNets SmartNets Methods.
CEN5011, Fall CEN5011 Software Engineering Dr. Yi Deng ECS359, (305)
4-1 Copyright © 2009 Pearson Education Canada CHAPTER 4 Strategic Planning Concepts for Marketing Communications.
Framework for MDO Studies Amitay Isaacs Center for Aerospace System Design and Engineering IIT Bombay.
Requirement engineering Good Practices for Requirements Engineering
Michael A. Hitt C. Chet Miller Adrienne Colella Slides by R. Dennis Middlemist Michael A. Hitt C. Chet Miller Adrienne Colella Chapter 4 Learning and Perception.
Objectives Targets & Indicators for Adaptation to Climate Change A presentation to the expert meeting on climate change vulnerability and adaptation indicators,
1© 2010 by Nelson Education Ltd. Chapter Five Training Design.
David Adams ATLAS DIAL: Distributed Interactive Analysis of Large datasets David Adams BNL August 5, 2002 BNL OMEGA talk.
New Perspective Based on how the system is used. What Is a Use Case? A case of how the system is used. –A behaviourally related sequence of interactions.
Principles of MarketingTheocharis Katranis, MBASpring Semester 2013 Principles of Marketing Theocharis Katranis Final Revision Spring Semester 2013 Final.
 An Information System (IS) is a collection of interrelated components that collect, process, store, and provide as output the information needed to.
By: Dr. Mohammed Alojail College of Computer Sciences & Information Technology 1.
The cross-disciplinary approach to education system in the framework of System’s Theory. Dr.Sarantos Psycharis Greek Ministry of Education and Religious.
1 Presentation Methodology Summary B. Golden. 2 Introduction Why use visualizations?  To facilitate user comprehension  To convey complexity and intricacy.
CS223: Software Engineering Lecture 14: Architectural Patterns.
1 TCS Confidential. 2 Objective : In this session we will be able to learn:  What is Cloud Computing?  Characteristics  Cloud Flavors  Cloud Deployment.
From Use Cases to Implementation 1. Structural and Behavioral Aspects of Collaborations  Two aspects of Collaborations Structural – specifies the static.
CSCI 383 Object-Oriented Programming & Design Lecture 7 Martin van Bommel.
Copyright © 2016 Pearson Education, Inc. Modern Database Management 12 th Edition Jeff Hoffer, Ramesh Venkataraman, Heikki Topi CHAPTER 11: BIG DATA AND.
From Use Cases to Implementation 1. Mapping Requirements Directly to Design and Code  For many, if not most, of our requirements it is relatively easy.
18 May 2006CCGrid2006 Dynamic Workflow Management Using Performance Data Lican Huang, David W. Walker, Yan Huang, and Omer F. Rana Cardiff School of Computer.
Spark on Entropy : A Reliable & Efficient Scheduler for Low-latency Parallel Jobs in Heterogeneous Cloud Huankai Chen PhD Student at University of Kent.
Software Design Process. What is software? mid-1970s executable binary code ‘source code’ and the resulting binary code 1990s development of the Internet.
Design and Manufacturing in a Distributed Computer Environment
Unified Modeling Language
JOB DESIGN & JOB ANALYSIS
UML dynamic Modeling (Behavior Diagram)
BPMN - Business Process Modeling Notations
Chapter 5 Architectural Design.
Presentation transcript:

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 1 :: Workflows and HPC? :: The relation between workflows and High Performance Computing :: Contact: Lutz Schubert Stefan Wesner

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 2 :: The relation between workflows and High Performance Computing No workflows in HPC!

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 3 :: The relation between workflows and High Performance Computing No workflows in HPC! Classically, a workflow is – A predefined sequence of tasks / actions – That are exposed in a standard fashion – And communicate via a centralized point Or: “Any program is a workflow” [thanks Mr. Turing]

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 4 :: The relation between workflows and High Performance Computing No workflows in HPC! TODO: Picture of a workflow interaction? At the same time: it’s the least efficient way to execute a program: – Centralized communication – Communication overhead through standards – Next steps need to be identified in a central point –…–…  Just the opposite of what HPC programs try to achieve 1 2 3

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 5 :: The relation between workflows and High Performance Computing But why should workflows act directly on the process level?  Can also act as a means to control higher-order processes I.Initiate a HPC job as part of a larger process II.Trigger jobs upon specific events (that provide data) III.Coupling jobs according to outcome / status No workflows in HPC?

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 6 :: The relation between workflows and High Performance Computing I. Higher-Order Processes with HPC Tasks The “Grid” concept: In particular in engineering cases, extensive calculations need to take place at various stages In large scale, collaborative engineering examples, workflows can reduce the complexity of providing data / jobs A sub-workflow can take over recurring setup and configuration tasks Examples: – Collaborative Engineering (BAE ) – Virtual Engineering (ANSYS)

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 7 :: The relation between workflows and High Performance Computing I. Basic Collaborative Structure

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 8 :: The relation between workflows and High Performance Computing I. High-Level Workflow

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 9 :: The relation between workflows and High Performance Computing II. Event-Based Job Control The “Cloud” Principle: Some recurring jobs need to be executed e.g. with updates in data, under certain environmental conditions etc. These events may trigger recurring configuration workflows (cf. I) Potentially same job offered for multiple users on-demand (with personal data) Not a workflow as such, but easier defined and adapted by making use of workflow descriptions Examples: – Microsoft Financial Computing – Google Market Evaluations

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 10 :: The relation between workflows and High Performance Computing Data pro- visioning Job execution Data sub- mission Job sub- mission II. Financial Calculation Principle specific request changes on the global stock market “personal” results triggers

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 11 :: The relation between workflows and High Performance Computing III. Coupled Applications Multiple HPC jobs that depend on each other Next job is triggered with the conclusion of the previous one (passing the data) The selection of the next job may depend on the results of the previous ones Intermediary results may trigger evaluation jobs in parallel Examples: – Material Stress Test – Virtual Physiological Human

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 12 :: The relation between workflows and High Performance Computing Critical material characteristics Critical points III. Material Stress Test Principle whole modelindividual elements in-depth material test Result feedback Impact on whole model

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 13 :: The relation between workflows and High Performance Computing III. A Virtual Physiological Human Model

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 14 :: The relation between workflows and High Performance Computing III. A Virtual Physiological Human Model Same base model, but different detailed elements No direct coupling between elements New knowledge about one element can impact on other behaviour Simulation of specific diseases may lead to coupling, depending on goal, e.g. – how medicine spreads if the heart muscle is affected – how a muscular disease spreads to and affects the heart – etc.  Data is only exchanged between elements under certain conditions  The workflow must model these events and conditions

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 15 :: The relation between workflows and High Performance Computing To Workflow or Not To Workflow Good For Describing relationships between actors and tasks Recurring configuration and submission processes Result-dependent relationships between jobs Modelling event-based triggers Bad For Tightly coupled process control & execution Distribution of large data between jobs Fast interactions with external processes

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 16 :: The relation between workflows and High Performance Computing Summary  Workflows can be used for higher-order control  Tasks in a workflow could be considered as “triggers”  (Large) datasets should be hosted in a well-known place rather than provided via the workflow  Recurring tasks that are not time-critical can be supported by workflows (e.g. configuration steps)  They are more comprehensible than batch jobs and hence easier to model and adapt

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 17 :: The relation between workflows and High Performance Computing THANK YOU FOR YOUR ATTENTION - ANY QUESTIONS?

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 18 :: The relation between workflows and High Performance Computing Old slides

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 19 :: The relation between workflows and High Performance Computing

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 20 :: The relation between workflows and High Performance Computing To Workflow or Not To Workflow Workflows can particularly support recurring tasks and typical relationships between jobs / tasks – configuration and submission steps – decision logic – event based triggers They are not sensible to control the distributed nature of a HPC job The stronger the coupling between tasks / elements, the less sensible is workflow support  Higher-order support only

:: IDC 2009 :: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: ::::: :: 06/10/2009 :::: 21 :: The relation between workflows and High Performance Computing III. A Virtual Physiological Human Model