1. CS 584

2. A Parallel Programming Model We need abstractions to make it simple. The programming model needs to fit our parallel machine model. Abstractions we will use: Task Channel

3. Tasks & Channels A Task is represented by a circle A Channel is represented by a directed edge

4. Tasks A task encapsulates a sequential program and local memory. A parallel computation consists of one or more tasks. Tasks execute concurrently. The number of tasks can vary during program execution.

5. Channels Channels are communication paths between two tasks. A task may: Send a message on a channel  Asynchronous Receive a message on a channel  Synchronous Channels are dynamic.

6. Additional Details Tasks can be mapped to physical processors in various ways. One task per processor Multiple tasks per processor Combinations Tasks can dynamically create new tasks. A Task-Channel diagram is a slice of time in the execution of an algorithm.

7. Bridge Construction Example F FB B Need More! (a) (b)

8. Model Properties Is this model appropriate? In order to decide, we should examine: Performance Mapping Independence Modularity Determinism

9. Performance Sequential abstractions such as procedures, and data structures are effective because they map directly to von Neumann machine. Can we map this model to multi-computer? Task -- Sequential code on a processor Channel -- Communication  Interprocessor communication  Shared memory

10. Mapping Independence Regardless of the Task-Channel Model, can we map the problem to various machines? Tasks always communicate via channels. No need to worry about processor count. Mapping more than one task per processor is a straight-forward way to achieve scalability.

11. Modularity Can we develop various components of the algorithm as independent modules? The Task is a natural fit to modular design. Task encapsulates both data and code Input and Output ports correspond to the module interface.

12. Determinism Does execution of the algorithm always yield the same otuput? Remember: Nondeterminism is useful. Our model should support both Task-Channel model makes determinism relatively easy to guarantee. Single sender and single receiver Receive is blocking.

13. Other Programming Models Message Passing Task-Channel model maps well Data Parallelism Perform operation on all data Shared Memory Can still use the Task-Channel model Channels can be semaphore locks etc.

14. Embarrassingly-Parallel I WWWW O

15. Pipeline T4T3T2T1