1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 3 Distributed Memory Programming with MPI An Introduction to Parallel Programming Peter Pacheco.

Slides:

Advertisements

Similar presentations

The University of Adelaide, School of Computer Science

Advertisements

MPI Basics Introduction to Parallel Programming and Cluster Computing University of Washington/Idaho State University MPI Basics Charlie Peck Earlham College.

Chapter 3. MPI MPI = Message Passing Interface Specification of message passing libraries for developers and users –Not a library by itself, but specifies.

Tutorial on MPI Experimental Environment for ECE5610/CSC

High Performance Computing

MPI Fundamentals—A Quick Overview Shantanu Dutt ECE Dept., UIC.

MPI Program Structure Self Test with solution. Self Test 1.How would you modify "Hello World" so that only even-numbered processors print the greeting.

Introduction MPI Mengxia Zhu Fall An Introduction to MPI Parallel Programming with the Message Passing Interface.

Message-Passing Programming and MPI CS 524 – High-Performance Computing.

Distributed Memory Programming with MPI. What is MPI? Message Passing Interface (MPI) is an industry standard message passing system designed to be both.

1 Parallel Computing—Introduction to Message Passing Interface (MPI)

S an D IEGO S UPERCOMPUTER C ENTER N ATIONAL P ARTNERSHIP FOR A DVANCED C OMPUTATIONAL I NFRASTRUCTURE Message Passing Interface (MPI) Part I NPACI Parallel.

12b.1 Introduction to Message-passing with MPI UNC-Wilmington, C. Ferner, 2008 Nov 4, 2008.

1 Tuesday, October 10, 2006 To err is human, and to blame it on a computer is even more so. -Robert Orben.

1 TRAPEZOIDAL RULE IN MPI Copyright © 2010, Elsevier Inc. All rights Reserved.

ICOM 5995: Performance Instrumentation and Visualization for High Performance Computer Systems Lecture 9 October 30, 2002 Nayda G. Santiago.

1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 5 Shared Memory Programming with OpenMP An Introduction to Parallel Programming Peter Pacheco.

CS6235 L15: Design Review, Midterm Review and 6- Function MPI.

Paul Gray, University of Northern Iowa Henry Neeman, University of Oklahoma Charlie Peck, Earlham College Tuesday October University of Oklahoma.

1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 3 Distributed Memory Programming with MPI An Introduction to Parallel Programming Peter Pacheco.

CS6235 L17: Design Review and 6-Function MPI. L17: DRs and MPI 2 CS6235 Administrative Organick Lecture: TONIGHT -David Shaw, “Watching Proteins Dance:

Parallel & Cluster Computing MPI Basics Paul Gray, University of Northern Iowa David Joiner, Shodor Education Foundation Tom Murphy, Contra Costa College.

Parallel Processing1 Parallel Processing (CS 676) Lecture 7: Message Passing using MPI * Jeremy R. Johnson *Parts of this lecture was derived from chapters.

Lecture 5 MPI Introduction Topics …Readings January 19, 2012 CSCE 713 Advanced Computer Architecture.

The University of Adelaide, School of Computer Science

1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 3 Distributed Memory Programming with MPI An Introduction to Parallel Programming Peter Pacheco.

2.1 Message-Passing Computing ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, Jan 17, 2012.

Specialized Sending and Receiving David Monismith CS599 Based upon notes from Chapter 3 of the MPI 3.0 Standard

CS 240A Models of parallel programming: Distributed memory and MPI.

Parallel Programming with MPI Prof. Sivarama Dandamudi School of Computer Science Carleton University.

Message Passing Programming with MPI Introduction to MPI Basic MPI functions Most of the MPI materials are obtained from William Gropp and Rusty Lusk’s.

CS 838: Pervasive Parallelism Introduction to MPI Copyright 2005 Mark D. Hill University of Wisconsin-Madison Slides are derived from an online tutorial.

Message Passing Programming Model AMANO, Hideharu Textbook pp. １４０－１４７.

Message Passing Interface (MPI) 1 Amit Majumdar Scientific Computing Applications Group San Diego Supercomputer Center Tim Kaiser (now at Colorado School.

MPI Introduction to MPI Commands. Basics – Send and Receive MPI is a message passing environment. The processors’ method of sharing information is NOT.

1 The Message-Passing Model l A process is (traditionally) a program counter and address space. l Processes may have multiple threads (program counters.

CS 420 – Design of Algorithms MPI Data Types Basic Message Passing - sends/receives.

Parallel Programming with MPI By, Santosh K Jena..

MA471Fall 2002 Lecture5. More Point To Point Communications in MPI Note: so far we have covered –MPI_Init, MPI_Finalize –MPI_Comm_size, MPI_Comm_rank.

Lecture 5 Barriers and MPI Introduction Topics Barriers Uses implementations MPI Introduction Readings – Semaphore handout dropboxed January 24, 2012 CSCE.

Chapter 4 Message-Passing Programming. The Message-Passing Model.

Oct. 23, 2002Parallel Processing1 Parallel Processing (CS 730) Lecture 6: Message Passing using MPI * Jeremy R. Johnson *Parts of this lecture was derived.

CS4230 CS4230 Parallel Programming Lecture 13: Introduction to Message Passing Mary Hall October 23, /23/2012.

Introduction to MPI CDP 1. Shared Memory vs. Message Passing Shared Memory Implicit communication via memory operations (load/store/lock) Global address.

Programming distributed memory systems: Message Passing Interface (MPI) Distributed memory systems: multiple processing units working on one task (e.g.

1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 3 Distributed Memory Programming with MPI An Introduction to Parallel Programming Peter Pacheco.

An Introduction to MPI (message passing interface)

Introduction to Parallel Programming at MCSR Message Passing Computing –Processes coordinate and communicate results via calls to message passing library.

Project18 Communication Design + Parallelization Camilo A Silva BIOinformatics Summer 2008.

-1.1- MPI Lectured by: Nguyễn Đức Thái Prepared by: Thoại Nam.

Lecture 5 CSS314 Parallel Computing Book: “An Introduction to Parallel Programming” by Peter Pacheco

Message Passing Interface Using resources from

Lecture 6 MPI Introduction Topics …Readings January 19, 2012 CSCE 713 Advanced Computer Architecture.

MPI-Message Passing Interface. What is MPI?  MPI is a specification for the developers and users of message passing libraries. By itself, it is NOT a.

1 Programming distributed memory systems Clusters Distributed computers ITCS 4/5145 Parallel Computing, UNC-Charlotte, B. Wilkinson, Jan 6, 2015.

MPI Message Passing Interface

MPI: The Message-Passing Interface

CS4961 Parallel Programming Lecture 16: Introduction to Message Passing Mary Hall November 3, /03/2011 CS4961.

MPI-Message Passing Interface

Quiz Questions ITCS 4145/5145 Parallel Programming MPI

Introduction to parallelism and the Message Passing Interface

Hardware Environment VIA cluster - 8 nodes Blade Server – 5 nodes

Copyright © 2013 Elsevier Inc. All rights reserved.

Hello, world in MPI #include <stdio.h> #include "mpi.h"

Distributed Memory Programming with Message-Passing

Hello, world in MPI #include <stdio.h> #include "mpi.h"

MPI Message Passing Interface

Some codes for analysis and preparation for programming

Presentation transcript:

1 Copyright © 2010, Elsevier Inc. All rights Reserved Chapter 3 Distributed Memory Programming with MPI An Introduction to Parallel Programming Peter Pacheco

2 Copyright © 2010, Elsevier Inc. All rights Reserved Roadmap Writing your first MPI program. Using the common MPI functions. # Chapter Subtitle

3 A distributed memory system Copyright © 2010, Elsevier Inc. All rights Reserved

4 A shared memory system Copyright © 2010, Elsevier Inc. All rights Reserved

5 Hello World! Copyright © 2010, Elsevier Inc. All rights Reserved (a classic)

6 Identifying MPI processes Common practice to identify processes by nonnegative integer ranks. p processes are numbered 0, 1, 2,.. p-1 Copyright © 2010, Elsevier Inc. All rights Reserved

7 Compilation Copyright © 2010, Elsevier Inc. All rights Reserved mpicc -g -Wall -o mpi_hello mpi_hello.c wrapper script to compile turns on all warnings source file create this executable file name (as opposed to default a.out) produce debugging information

8 Execution Copyright © 2010, Elsevier Inc. All rights Reserved mpiexec -n mpiexec -n 1./mpi_hello mpiexec -n 4./mpi_hello run with 1 process run with 4 processes

9 Execution Copyright © 2010, Elsevier Inc. All rights Reserved mpiexec -n 1./mpi_hello mpiexec -n 4./mpi_hello Greetings from process 0 of 1 ! Greetings from process 0 of 4 ! Greetings from process 1 of 4 ! Greetings from process 2 of 4 ! Greetings from process 3 of 4 !

10 MPI Programs Written in C. Has main. Uses stdio.h, string.h, etc. Need to add mpi.h header file. Identifiers defined by MPI start with “MPI_”. First letter following underscore is uppercase. For function names and MPI-defined types. Helps to avoid confusion. Copyright © 2010, Elsevier Inc. All rights Reserved

11 MPI Components MPI_Init Tells MPI to do all the necessary setup. MPI_Finalize Tells MPI we’re done, so clean up anything allocated for this program. Copyright © 2010, Elsevier Inc. All rights Reserved

12 Basic Outline Copyright © 2010, Elsevier Inc. All rights Reserved

13 Communicators A collection of processes that can send messages to each other. MPI_Init defines a communicator that consists of all the processes created when the program is started. Called MPI_COMM_WORLD. Copyright © 2010, Elsevier Inc. All rights Reserved

14 SPMD Single-Program Multiple-Data We compile one program. Process 0 does something different. Receives messages and prints them while the other processes do the work. The if-else construct makes our program SPMD. Copyright © 2010, Elsevier Inc. All rights Reserved

15 Communication Copyright © 2010, Elsevier Inc. All rights Reserved

16 Data types Copyright © 2010, Elsevier Inc. All rights Reserved

17 Communication Copyright © 2010, Elsevier Inc. All rights Reserved

18 Message matching Copyright © 2010, Elsevier Inc. All rights Reserved MPI_Send src = q MPI_Recv dest = r r q

19 Receiving messages A receiver can get a message without knowing: the amount of data in the message, the sender of the message, or the tag of the message. Copyright © 2010, Elsevier Inc. All rights Reserved

20 status_p argument Copyright © 2010, Elsevier Inc. All rights Reserved MPI_SOURCE MPI_TAG MPI_ERROR MPI_Status* MPI_Status* status; status.MPI_SOURCE status.MPI_TAG

21 How much data am I receiving? Copyright © 2010, Elsevier Inc. All rights Reserved

22 Issues with send and receive Exact behavior is determined by the MPI implementation. MPI_Send may behave differently with regard to buffer size, cutoffs and blocking. MPI_Recv always blocks until a matching message is received. Copyright © 2010, Elsevier Inc. All rights Reserved