Abdulrahman Azab 5, Apr Workflow Management on Abel.

Slides:

Advertisements

Similar presentations

EXAMPLE: X Y Z T 5 VERTICES 6 EDGES S Collection List Set ArrayList LinkedList SortedSet HashSet TreeSet.

Advertisements

What is a graph ? G=(V,E) V = a set of vertices E = a set of edges edge = unordered pair of vertices

Simple Graph Warmup. Cycles in Simple Graphs A cycle in a simple graph is a sequence of vertices v 0, …, v n for some n>0, where v 0, ….v n-1 are distinct,

Chapter 10: Trees. Definition A tree is a connected undirected acyclic (with no cycle) simple graph A collection of trees is called forest.

IKI 10100: Data Structures & Algorithms Ruli Manurung (acknowledgments to Denny & Ade Azurat) 1 Fasilkom UI Ruli Manurung (Fasilkom UI)IKI10100: Lecture10.

Lecture 16 CSE 331 Oct 9, Announcements Hand in your HW4 Solutions to HW4 next week Remember next week I will not be here so.

A tree is a simple graph satisfying: if v and w are vertices and there is a path from v to w, it is a unique simple path. a b c a b c.

1 Graph Introduction Definitions. 2 Definitions I zDirected Graph (or Di-Graph) is an ordered pair G=(V,E) such that yV is a finite, non-empty set (of.

Is the following graph Hamiltonian- connected from vertex v? a). Yes b). No c). I have absolutely no idea v.

CTIS 154 Discrete Mathematics II1 8.2 Paths and Cycles Kadir A. Peker.

Chapter 9: Graphs Basic Concepts

Chapter 4 Graphs.

CS2420: Lecture 36 Vladimir Kulyukin Computer Science Department Utah State University.

Chapter 9 – Graphs A graph G=(V,E) – vertices and edges

Lecture 13 Graphs. Introduction to Graphs Examples of Graphs – Airline Route Map What is the fastest way to get from Pittsburgh to St Louis? What is the.

Computer Science 112 Fundamentals of Programming II Introduction to Graphs.

GRAPH THEORY.  A graph is a collection of vertices and edges.  An edge is a connection between two vertices (or nodes).  One can draw a graph by marking.

Lecture 17 Trees CSCI – 1900 Mathematics for Computer Science Fall 2014 Bill Pine.

Graphs. Definitions A graph is two sets. A graph is two sets. –A set of nodes or vertices V –A set of edges E Edges connect nodes. Edges connect nodes.

Python Arrays. An array is a variable that stores a collection of things, like a list. For example a list of peoples names. We can access the different.

HTCondor and Workflows: An Introduction HTCondor Week 2015 Kent Wenger.

1 12/2/2015 MATH 224 – Discrete Mathematics Formally a graph is just a collection of unordered or ordered pairs, where for example, if {a,b} G if a, b.

COSC 2007 Data Structures II Chapter 14 Graphs I.

Graph Theory. A branch of math in which graphs are used to solve a problem. It is unlike a Cartesian graph that we used throughout our younger years of.

Homework #5 Due: October 31, 2000 Christine Kang Graph Concepts and Algorithms.

DIRECTED ACYCLIC GRAPHS AND TOPOLOGICAL SORT CS16: Introduction to Data Structures & Algorithms Tuesday, March 10,

Graph Concepts and Algorithms Using LEDA By Caroline Moore and Carmen Frerichs (252a-at and 252a-ao) each graph in the presentation was created using gw_basic_graph_algorithms.

Graph Concepts Elif Tosun 252a-aa (All graphics created by using demo/graphwin/gw*)

Graphs Definition: a graph is an abstract representation of a set of objects where some pairs of the objects are connected by links. The interconnected.

Tree - in “math speak” An ________ graph is a set of vertices/nodes and a set of edges, each edge connects two vertices. Any undirected graph in which.

Graph Concepts Illustrated Using The Leda Library Amanuel Lemma CS252 Algorithms.

Advanced topics Cluster Training Center for Simulation and Modeling September 4, 2015.

1 Trees : Part 1 Reading: Section 4.1 Theory and Terminology Preorder, Postorder and Levelorder Traversals.

Modules, Compiling WRF, and Running on CHPC Clusters Adam Varble WRF Users Meeting 10/26/15.

Graph Terms By Susan Ott. Vertices Here are 7 vertices without any edges Each Vertex is labeled a different color and number.

11 Graph Search Algorithms. 2 What parts of the graph are reachable from a given vertex ?

COP 3530 Spring2012 Data Structures & Algorithms Discussion Session.

Spanning Trees Alyce Brady CS 510: Computer Algorithms.

Leda Demos By: Kelley Louie Credits: definitions from Algorithms Lectures and Discrete Mathematics with Algorithms by Albertson and Hutchinson graphics.

Topological Sort. Sorting technique over DAGs (Directed Acyclic Graphs) It creates a linear sequence (ordering) for the nodes such that: –If u has an.

FESR Trinacria Grid Virtual Laboratory Practical using WMProxy advanced job submission Emidio Giorgio INFN Catania.

Hands on training session for core skills

Topological Sorting.

Basic Concepts Graphs For more notes and topics visit:

CS 367 – Introduction to Data Structures

Joker: Getting the most out of the slurm scheduler

Lecture 15 CSE 331 Oct 3, 2016.

Parallel computation with R on TACC HPC server

More Graph Algorithms.

Topological Sort.

Lecture 15 CSE 331 Oct 3, 2016.

Topological Sort.

CCR Advanced Seminar: Running CPLEX Computations on the ISE Cluster

"Learning how to learn is life's most important skill. " - Tony Buzan

CS223 Advanced Data Structures and Algorithms

Chapter 9: Graphs Basic Concepts

Directed Acyclic Graphs && Topological Sorting

Graph Theory By Amy C. and John M..

Graphs Examples on some basic graph concepts and definitions All graphics are taken from the LEDA demos: basic_graph_algorithms, gw_shortest_paths, graphwin.

A Introduction to Computing II Lecture 13: Trees

Graphs By Rajanikanth B.

Graphs G = (V, E) V are the vertices; E are the edges.

CSE 417: Algorithms and Computational Complexity

GRAPHS Lecture 17 CS2110 Spring 2018.

Chapter 9: Graphs Basic Concepts

Chapter 14 Graphs © 2011 Pearson Addison-Wesley. All rights reserved.

Heaps Chapter 6 Section 6.9.

CS639: Data Management for Data Science

DAGs Longin Jan Latecki

Lightflow a lightweight, distributed workflow system

Presentation transcript:

Abdulrahman Azab 5, Apr Workflow Management on Abel

Kiss! Keep It Simple Stupid!

What is a Workflow?  A series of activities that are necessary to complete a task.

What is a Workflow? input.txt A B C D outA.txt outB.txt outC.txt output.txt

Workflow Control input.txt A B C D outA.txt outB.txt outC.txt outD.txt Controller Login Node (abel.uio.no) Abel

Workflow Control input.txt A B C D outA.txt outB.txt outC.txt outD.txt Controller Abel Login Node (abel.uio.no)

SDAG: Direct Acyclic Graph Workflow Submitter for SLURM

Direct Acyclic Graph (DAG)  Directed graph with no directed cycles.  Formed by a collection of vertices and directed edges.  there is no way to start at some vertex and follow a sequence of edges that eventually loops back to the same vertex. No Loops

Direct Acyclic Graph (DAG)

Non-Direct Acyclic Graph (DAG)

SDAG Example JOB A jobA.sbatch JOB B jobB.sbatch JOB C jobC.sbatch PARENT A CHILD B PARENT B CHILD C A B C

SDAG Example JOB A jobA.sbatch JOB B jobB.sbatch JOB C jobC.sbatch JOB D jobD.sbatch PARENT A CHILD B C PARENT B C CHILD D A B C D

SDAG hands-on  On GitHub:

SDAG hands-on  Login  $git clone  $cd sdag/example1  $vim i1 I want to go home :(  $vim i2 I REALLY want to go home :’(

SDAG hands-on  $python test.py i1,i2 out 5 gift  $cat out python test.py i1,i2,… output 30 j- XX codeInput filesoutput fileRun-time Job name Interpreter

SDAG hands-on  $projects  $vim jobA.sbatch #SBATCH --account=  The same with jobB.sbatch, jobC.sbatch, and jobD.sbatch A B C D

SDAG hands-on  $sbatch jobA.sbatch  $squeue -u  $module load sdag  $vim dag.sdag  $sdag dag.sdag  $squeue -u