Grid Computing, B. Wilkinson, 200410.1 C Program Command Line Arguments A normal C program specifies command line arguments to be passed to main with:

Slides:



Advertisements
Similar presentations
Introduction to C Programming
Advertisements

Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Prepared 7/28/2011 by T. O’Neil for 3460:677, Fall 2011, The University of Akron.
Practical techniques & Examples
The Assembly Language Level
CSCI-455/552 Introduction to High Performance Computing Lecture 11.
Partitioning and Divide-and-Conquer Strategies ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, Jan 23, 2013.
Chapter 3. MPI MPI = Message Passing Interface Specification of message passing libraries for developers and users –Not a library by itself, but specifies.
Creating Computer Programs lesson 27. This lesson includes the following sections: What is a Computer Program? How Programs Solve Problems Two Approaches:
Reference: / MPI Program Structure.
Reference: Message Passing Fundamentals.
Introduction to MPI. What is Message Passing Interface (MPI)?  Portable standard for communication  Processes can communicate through messages.  Each.
Data Parallel Algorithms Presented By: M.Mohsin Butt
Embarrassingly Parallel Computations Partitioning and Divide-and-Conquer Strategies Pipelined Computations Synchronous Computations Asynchronous Computations.
Message-Passing Programming and MPI CS 524 – High-Performance Computing.
Comp 422: Parallel Programming Lecture 8: Message Passing (MPI)
ספטמבר 04Copyright Meir Kalech1 C programming Language Chapter 10: Appendices.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Parallel Programming Using Basic MPI Presented by Timothy H. Kaiser, Ph.D. San Diego Supercomputer Center Presented by Timothy H. Kaiser, Ph.D. San Diego.
Task Farming on HPCx David Henty HPCx Applications Support
Introduction to Parallel Programming MapReduce Except where otherwise noted all portions of this work are Copyright (c) 2007 Google and are licensed under.
1 COMPSCI 110 Operating Systems Who - Introductions How - Policies and Administrative Details Why - Objectives and Expectations What - Our Topic: Operating.
Parallel Programming Models Jihad El-Sana These slides are based on the book: Introduction to Parallel Computing, Blaise Barney, Lawrence Livermore National.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Part I MPI from scratch. Part I By: Camilo A. SilvaBIOinformatics Summer 2008 PIRE :: REU :: Cyberbridges.
CS 240A Models of parallel programming: Distributed memory and MPI.
Distribution After Release Tool Natalia Ratnikova.
Parallel Computing A task is broken down into tasks, performed by separate workers or processes Processes interact by exchanging information What do we.
Message Passing Programming with MPI Introduction to MPI Basic MPI functions Most of the MPI materials are obtained from William Gropp and Rusty Lusk’s.
Message Passing Programming Model AMANO, Hideharu Textbook pp. 140-147.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
CPS4200 Unix Systems Programming Chapter 2. Programs, Processes and Threads A program is a prepared sequence of instructions to accomplish a defined task.
Embarrassingly Parallel Computations Partitioning and Divide-and-Conquer Strategies Pipelined Computations Synchronous Computations Asynchronous Computations.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd Edition, by B. Wilkinson & M. Allen, ©
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Chapter 4 Message-Passing Programming. The Message-Passing Model.
Silberschatz, Galvin and Gagne ©2009 Operating System Concepts – 8 th Edition, Chapter 3: Process-Concept.
Slides for Parallel Programming Techniques & Applications Using Networked Workstations & Parallel Computers 2nd ed., by B. Wilkinson & M
Programming distributed memory systems: Message Passing Interface (MPI) Distributed memory systems: multiple processing units working on one task (e.g.
CSCI-455/552 Introduction to High Performance Computing Lecture 9.
M1G Introduction to Programming 2 3. Creating Classes: Room and Item.
Globus Grid Tutorial Part 2: Running Programs Across Multiple Resources.
CSCI-455/552 Introduction to High Performance Computing Lecture 23.
Introduction to OOP CPS235: Introduction.
 Software Development Life Cycle  Software Development Tools  High Level Programming:  Structures  Algorithms  Iteration  Pseudocode  Order of.
Operating Systems (CS 340 D) Dr. Abeer Mahmoud Princess Nora University Faculty of Computer & Information Systems Computer science Department.
1 Running MPI on “Gridfarm” Bryan Carpenter February, 2005.
Project18 Communication Design + Parallelization Camilo A Silva BIOinformatics Summer 2008.
3/12/2013Computer Engg, IIT(BHU)1 INTRODUCTION-1.
INVITATION TO Computer Science 1 11 Chapter 2 The Algorithmic Foundations of Computer Science.
3/12/2013Computer Engg, IIT(BHU)1 MPI-1. MESSAGE PASSING INTERFACE A message passing library specification Extended message-passing model Not a language.
Implementing Processes and Threads CS550 Operating Systems.
CSCI-455/552 Introduction to High Performance Computing Lecture 21.
Message Passing Interface Using resources from
Suzaku Pattern Programming Framework (a) Structure and low level patterns © 2015 B. Wilkinson Suzaku.pptx Modification date February 22,
Visit for more Learning Resources
Operating Systems (CS 340 D)
MPI Message Passing Interface
Operating Systems (CS 340 D)
User-Defined Functions
Using compiler-directed approach to create MPI code automatically
CS 240 – Lecture 18 Command-line Arguments, Typedef, Union, Bit Fields, Pointers to Functions.
Introduction to parallelism and the Message Passing Interface
Monte Carlo Methods A so-called “embarrassingly parallel” computation as it decomposes into obviously independent tasks that can be done in parallel without.
Programming with Shared Memory
Parallel Techniques • Embarrassingly Parallel Computations
Embarrassingly Parallel Computations
Creating Computer Programs
Creating Computer Programs
Presentation transcript:

Grid Computing, B. Wilkinson, C Program Command Line Arguments A normal C program specifies command line arguments to be passed to main with: int main(int argc, char *argv[]) where argc is the argument count and argv[] is an array of character pointers. –First entry is a pointer to program name –Subsequent entries point to subsequent strings on the command line.

Grid Computing, B. Wilkinson, MPI C program command line arguments Implementations of MPI remove from the argv array any command line arguments used by the implementation. Note MPI_Init requires argc and argv (specified as addresses)

Grid Computing, B. Wilkinson, Example Getting Command Line Argument #include “mpi.h” #include int main (int argc, char *argv[]) { int n; /* get and convert character string argument to integer value /* n = atoi(argv[1]); return 0; }

Grid Computing, B. Wilkinson, Executing MPI program with command line arguments mpirun -np 2 myProg argv[1] argv[2] Remember these array elements hold pointers to the arguments. argv[0] Removed by MPI implementation MPI_Init() ?

Grid Computing, B. Wilkinson, More Information on MPI Books: “Using MPI Portable Parallel Programming with the Message-Passing Interface 2nd ed.,” W. Gropp, E. Lusk, and A. Skjellum, The MIT Press,1999. MPICH: LAM MPI:

Grid Computing, B. Wilkinson, Grid-enabled MPI

Grid Computing, B. Wilkinson, Several versions of MPI developed for a grid: MPICH-G, MPI-G2 PACXMPI We will use MPICH-G2, which is based on MPICH for a grid, and uses Globus.

Grid Computing, B. Wilkinson, MPI code for the grid No difference in code from regular MPI code. Key aspect is MPI implementation: Communication methods Resource management

Grid Computing, B. Wilkinson, Communication Methods Implementation should take into account whether messages are between processor on the same computer or processors on different computers on the network. Pack messages into less larger message, even if this requires more computations

Grid Computing, B. Wilkinson, MPICH-G2 Complete implementation of MPI Can use existing MPI programs on a grid without change Uses Globus to start tasks, etc. Version 2 a complete redesign from MPICH-G for Globus 2.2 or later.

Grid Computing, B. Wilkinson, Compiling Application Program As with regular MPI programs, compile on each machine you intend to use (unless binary compatible and have common file system). For C programs: /bin/mpicc

Grid Computing, B. Wilkinson, Running an MPICH-G2 Program mpirun mpirun submits a Globus RSL script (Resource Specification Language Script) to launch application RSL script can be created by mpirun or you can write your own. RSL script gives powerful mechanism to specify different executables etc., but low level.

Grid Computing, B. Wilkinson, mpirun (with it constructing RSL script) Use if want to launch a single executable on binary compatible machines with a shared file system. Requires a “machines” file - a list of computers to be used (and job managers)

Grid Computing, B. Wilkinson, “Machines” file Computers listed by their Globus job manager service followed by optional maximum number of node (tasks) on that machine. If job manager omitted (i.e., just name of computer), will default to Globus job manager.

Grid Computing, B. Wilkinson, Location of “machines” file mpirun command expects the “ machines ” file either in –the directory specified by -machinefile flag –the current directory used to execute the mpirun command, or –in /bin/machines

Grid Computing, B. Wilkinson, Running MPI program Uses the same command line as a regular MPI program: mpirun -np 25 my_prog creates 25 tasks allocated on machines in “machines’ file in around robin fashion

Grid Computing, B. Wilkinson, Example With the machines file containing: ”jupitor.cs.wcu.edu" 4 "venus.cs.wcu.edu" 5 and the command: mpirun -np 10 myProg the first 4 processes (jobs) would run on jupitor, the next 5 on venus, and the remaining one on jupitor.

Grid Computing, B. Wilkinson, mpirun with your own RSL script Necessary if machines not executing same executable. Easiest way to create script is to modify existing one. Use mpirun -dumprsl which causes script printed out. Application program not launched.

Grid Computing, B. Wilkinson, Example mpirun -dumprsl -np 2 myprog will generate appropriate printout of an rsl document according to the details of the job from the command line and machine file.

Grid Computing, B. Wilkinson, Given rsl file, myRSL.rsl, use: mpirun -globusrsl myRSL.rsl to submit modified script.. More details and practice in assignment 5 (once I have finished it!)

Grid Computing, B. Wilkinson, MPICH-G internals Processes allocated a “machine-local” number and a “grid global” number - translated into where process actually resides. Non-local operations uses grid services Local operations do not. globusrun command submits simultaneous job requests

Grid Computing, B. Wilkinson, Limitations “machines” file limits computers to those known - no discovery of resources If machines heterogeneous, need appropriate executables available, and RSL script Speed an issue - original version MPI-G slow.

Grid Computing, B. Wilkinson, More information on MPI-G

Grid Computing, B. Wilkinson, Parallel Programming Techniques Suitable for a Grid

Grid Computing, B. Wilkinson, Message-Passing on a Grid VERY expensive, sending data across network costs millions of cycles Bandwidth shared with other users Links unreliable

Grid Computing, B. Wilkinson, Computational Strategies As a computing platform, a grid favors situations with absolute minimum communication between computers.

Grid Computing, B. Wilkinson, Strategies With no/minimum communication: “Embarrassingly Parallel” Computations –those computations which obviously can be divided into parallel independent parts. Parts executed on separate computers. Separate instance of the same problem executing on each system, each using different data

Grid Computing, B. Wilkinson, Embarrassingly Parallel Computations A computation that can obviously be divided into a number of completely independent parts, each of which can be executed by a separate process(or). No communication or very little communication between processes. Each process can do its tasks without any interaction with other processes

Grid Computing, B. Wilkinson, Monte Carlo Methods An embarrassingly parallel computation. Monte Carlo methods use of random selections.

Grid Computing, B. Wilkinson, Simple Example: To calculate  Circle formed within a square, with radius of 1. Square has sides 2  2.

Grid Computing, B. Wilkinson, Ratio of area of circle to square given by Area of circle =  (1) 2 =  Area of square 2 x 2 4 Points within square chosen randomly. Score kept of how many points happen to lie within circle. Fraction of points within circle will be  /4, given a sufficient number of randomly selected samples.

Grid Computing, B. Wilkinson, Method actually computes an integral. One quadrant of the construction can be described by integral

Grid Computing, B. Wilkinson, So can use method to compute any integral! Monte Carlo method very useful if the function cannot be integrated numerically (maybe having a large number of variables).

Grid Computing, B. Wilkinson, Alternative (better) “Monte Carlo” Method Use random values of x to compute f (x) and sum values of f (x) where x r are randomly generated values of x between x 1 and x 2. Areaf(x)x d x 1 x 2  1 N ---- N  limf( x r ) r i1= N  == (x 2 – x 1 ) X1X1 X2X2

Grid Computing, B. Wilkinson, Example Computing the integral Sequential Code sum = 0; for (i = 0; i < N; i++) { /* N random samples */ xr = rand_v(x1, x2);/* next random value */ sum = sum + xr * xr - 3 * xr/* compute f(xr)*/ } area = (sum / N) * (x2 - x1); randv(x1, x2) returns pseudorandom number between x1 and x2. x 1 x 2  (x 2 – 3x) dx

Grid Computing, B. Wilkinson, For parallelizing Monte Carlo code, must address best way to generate random numbers in parallel. Can use SPRNG (Scalable Pseudo- random Number Generator) -- supposed to be a good parallel random number generator.

Grid Computing, B. Wilkinson, Executing separate problem instances In some application areas, same program executed repeatedly - ideal if with different parameters (“parameter sweep”) Nimrod/G -- a grid broker project that targets parameter sweep problems.

Grid Computing, B. Wilkinson, Techniques to reduce effects of network communication Latency hiding with communication/computation overlap Better to have fewer larger messages than many smaller ones

Grid Computing, B. Wilkinson, Synchronous Algorithms Many tradition parallel algorithms require the parallel processes to synchronize at regular and frequent intervals to exchange data and continue from known points This is bad for grid computations!! All traditional parallel algorithms books have to be thrown away for grid computing.

Grid Computing, B. Wilkinson, Techniques to reduce actual synchronization communications Asynchronous algorithms –Algorithms that do not use synchronization at all Partially synchronous algorithms –those that limit the synchronization, for example only synchronize on every n iterations –Actually such algorithms known for many years but not popularized.

Grid Computing, B. Wilkinson, Big Problems “Grand challenge” problems Most of the high profile projects on the grid involve problems that are so big usually in number of data items that they cannot be solved otherwise

Grid Computing, B. Wilkinson, Examples High-energy physics Bioinformatics Medical databases Combinatorial chemistry Astrophysics

Grid Computing, B. Wilkinson, Workflow Technique Use functional decomposition - dividing problem into separate functions which take results from other functions units and pass on results to functional units - interconnection patterns depends upon the problem. Workflow - describes the flow of information between the units.

Grid Computing, B. Wilkinson, Example Climate Modeling

Grid Computing, B. Wilkinson, Next topic UNC-Wilmington’s GUI workflow editor. Next week Professor Clayton Ferner