Presentation is loading. Please wait.

Presentation is loading. Please wait.

FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20131 FIT5174 Distributed & Parallel Systems Lecture 5 Message Passing and MPI.

Published byTiffany Tucker Modified over 9 years ago

Similar presentations

Presentation on theme: "FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20131 FIT5174 Distributed & Parallel Systems Lecture 5 Message Passing and MPI."— Presentation transcript:

1 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20131 FIT5174 Distributed & Parallel Systems Lecture 5 Message Passing and MPI

2 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20132 Acknowledgement These slides are based on slides and material by: C Evangelinos, Scott Baden, Zhiliang Xu

3 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20133 Shared Memory Approach

4 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20134 Message Passing Model

5 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20135 Distributed Computing using Message Passing

6 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20136 Single Program / Multiple Program - Multiple Data

7 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20137 Programming with message passing

8 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20138 Message passing concept

9 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20139 Send and Receive concept

10 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201310 Message completion

11 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201311 Buffering concept

12 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201312 Causality concept

13 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201313 Asynchronous versus Synchronous

14 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201314 Overlapping operations

15 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201315 What is MPI?

16 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201316 MPI websites

17 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201317 MPI: Message Passing Interface

18 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201318 MPI history

19 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201319 MPI in context

20 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201320 MPI fundamentals

21 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201321 Minimal MPI subset

22 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201322 Initializing MPI

23 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201323 Getting started with MPI

24 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201324 MPI program details

25 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201325 MPI program more details

26 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201326 Simple MPI C Program

27 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201327 MPI Communicators and handles

28 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201328 MPI Communicator size

29 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201329 MPI Communicator size

30 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201330 MPI Process rank

31 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201331 MPI Process rank

32 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201332 MPI exiting

33 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201333 MPI communications

34 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201334 MPI basic datatypes for C

35 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201335 MPI C data types

36 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201336 MPI messages

37 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201337 MPI point-to-point communications

38 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201338 MPI point-to-point messages

39 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201339 MPI point-to-point messages requirements

40 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201340 Synchronous versus Asynchronous communications

41 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201341 MPI blocking standard Send

42 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201342 Send and Receive code fragment

43 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201343 Other MPI Send options

44 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201344 Considerations for MPI Send

45 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201345 Performance of MPI Send

46 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201346 MPI blocking Receive

47 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201347 MPI Receive considerations

48 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201348 MPI message passing restrictions

49 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201349 Simple MPI ‘ping-pong’

50 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201350 MPI deadlock scenario

51 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201351 MPI correctness and fairness

52 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201352 MPI bidirectional communication

53 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201353 MPI non-blocking communication

54 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201354 MPI non-blocking standard send

55 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201355 MPI non-blocking receive

56 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201356 Avoiding Deadlocks

57 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201357 MPI non-blocking sends

58 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201358 MPI non-blocking Synchronous Send

59 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201359 MPI testing instead of waiting

60 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201360 MPI with many outstanding calls

61 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201361 Other MPI communications calls

62 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201362 MPI persistent communication

63 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201363 MPI persistent communications

64 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201364 Useful MPI wildcards and constants

65 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201365 MPI collective communications

66 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201366 MPI synchronization

67 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201367 MPI broadcast

68 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201368 MPI gather

69 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201369 MPI scatter

70 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201370 MPI gather to all

71 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201371 Using vectors

72 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201372 Using vectors non-contiguously

73 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201373 Using binary trees

74 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201374 Using other kinds of trees

75 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201375 All-to-All personalized communication

76 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201376 MPI program structure summary

77 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201377 MPI program summary

78 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201378 MPI Simple C program

79 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201379 MPI Safe C program

80 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201380 MPI Deadlocking C program

81 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201381 MPI buffering dependent C program

82 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201382 Summarizing Distributed versus Shared Memory

83 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201383 Hybrid programming

84 FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 201384 Why use hybrid programming?

Download ppt "FIT5174 Parallel & Distributed Systems Dr. Ronald Pose Lecture 5 - 20131 FIT5174 Distributed & Parallel Systems Lecture 5 Message Passing and MPI."

Similar presentations

1 Tuning for MPI Protocols l Aggressive Eager l Rendezvous with sender push l Rendezvous with receiver pull l Rendezvous blocking (push or pull)

1 Tuning for MPI Protocols l Aggressive Eager l Rendezvous with sender push l Rendezvous with receiver pull l Rendezvous blocking (push or pull)
1 What is message passing? l Data transfer plus synchronization l Requires cooperation of sender and receiver l Cooperation not always apparent in code.

1 What is message passing? l Data transfer plus synchronization l Requires cooperation of sender and receiver l Cooperation not always apparent in code.
MPI Message Passing Interface

MPI Message Passing Interface
1 Introduction to Collective Operations in MPI l Collective operations are called by all processes in a communicator. MPI_BCAST distributes data from one.

1 Introduction to Collective Operations in MPI l Collective operations are called by all processes in a communicator. MPI_BCAST distributes data from one.
Winter, 2004CSS490 MPI1 CSS490 Group Communication and MPI Textbook Ch3 Instructor: Munehiro Fukuda These slides were compiled from the course textbook,

Winter, 2004CSS490 MPI1 CSS490 Group Communication and MPI Textbook Ch3 Instructor: Munehiro Fukuda These slides were compiled from the course textbook,
Parallel Processing1 Parallel Processing (CS 667) Lecture 9: Advanced Point to Point Communication Jeremy R. Johnson *Parts of this lecture was derived.

Parallel Processing1 Parallel Processing (CS 667) Lecture 9: Advanced Point to Point Communication Jeremy R. Johnson *Parts of this lecture was derived.
1 Collective Operations Dr. Stephen Tse 908-872-2108 Lesson 12.

1 Collective Operations Dr. Stephen Tse Lesson 12.
Getting Started with MPI Self Test with solution.

Getting Started with MPI Self Test with solution.
Concurrency CS 510: Programming Languages David Walker.

Concurrency CS 510: Programming Languages David Walker.
1 Parallel Computing—Introduction to Message Passing Interface (MPI)

1 Parallel Computing—Introduction to Message Passing Interface (MPI)
1 Message protocols l Message consists of “envelope” and data »Envelope contains tag, communicator, length, source information, plus impl. private data.

1 Message protocols l Message consists of “envelope” and data »Envelope contains tag, communicator, length, source information, plus impl. private data.
Computer Science Lecture 2, page 1 CS677: Distributed OS Last Class: Introduction Distributed Systems – A collection of independent computers that appears.

Computer Science Lecture 2, page 1 CS677: Distributed OS Last Class: Introduction Distributed Systems – A collection of independent computers that appears.
MPI Point-to-Point Communication CS 524 – High-Performance Computing.

MPI Point-to-Point Communication CS 524 – High-Performance Computing.
Distributed Systems CS 15-440 Programming Models- Part II Lecture 17, Nov 2, 2011 Majd F. Sakr, Mohammad Hammoud andVinay Kolar 1.

Distributed Systems CS Programming Models- Part II Lecture 17, Nov 2, 2011 Majd F. Sakr, Mohammad Hammoud andVinay Kolar 1.
1 Tuesday, October 10, 2006 To err is human, and to blame it on a computer is even more so. -Robert Orben.

1 Tuesday, October 10, 2006 To err is human, and to blame it on a computer is even more so. -Robert Orben.
Today Objectives Chapter 6 of Quinn Creating 2-D arrays Thinking about “grain size” Introducing point-to-point communications Reading and printing 2-D.

Today Objectives Chapter 6 of Quinn Creating 2-D arrays Thinking about “grain size” Introducing point-to-point communications Reading and printing 2-D.
1 What is message passing? l Data transfer plus synchronization l Requires cooperation of sender and receiver l Cooperation not always apparent in code.

1 What is message passing? l Data transfer plus synchronization l Requires cooperation of sender and receiver l Cooperation not always apparent in code.
The hybird approach to programming clusters of multi-core architetures.

The hybird approach to programming clusters of multi-core architetures.
Parallel Programming with Java

Parallel Programming with Java
CS 179: GPU Programming Lecture 20: Cross-system communication.

CS 179: GPU Programming Lecture 20: Cross-system communication.

Similar presentations

Ads by Google