1. Computer Science and Engineering Parallel and Distributed Processing CSE 8380 March 08, 2005 Session 16

2. Computer Science and Engineering Scheduling Sami’s Presentation Communication Models (review) Optimal Algorithms with communication Midterm review & Q/A

3. Computer Science and Engineering Communication Models Completion Time Execution time Communication time Completion Time as 2 Components Completion Time from the Gantt Chart

4. Computer Science and Engineering Completion Time as 2 Components Completion Time = Execution Time + Total Communication Delay Total Communication Delay = Number of communication messages * delay per message Execution time  maximum finishing time of any task Number of communication messages  model A Model B

5. Computer Science and Engineering Number of communication Messages Given a task graph G = (V,E) and its allocation on m processors f, we use proc(v) to refer to the processor to which task v is assigned. Model A: number of messages =the number of node pairs (u,v) such that (u,v)belongs to E and proc(u) ≠ proc(v). Model B: number of messages =the number of processor- task pairs (P,v) such that processor P does not compute task v but computes at least one direct successor of v.

6. Computer Science and Engineering Completion Time from the Gantt Chart (Model C) Completion Time = Schedule Length This model assumes the existence of an I/O processor with every processor in the system Communication delay between two tasks allocated to the same processor is negligible. Communication delay is counted only between two tasks assigned to different processors

7. Computer Science and Engineering Example A 1 D 1 E 1 B 1 C 1 Assume a system with 2 processors

8. Computer Science and Engineering Models A and B Assume tasks A, B, and D are assigned to P1 and tasks C and E are assigned to p2 A B D P1 C E P2 Model A Number of messages = 2 Completion time = 3 + 2 Model B Number of messages = 1 Completion time = 3 + 1

9. Computer Science and Engineering Model C A B CD E Communication Delay P1P2 0 1 2 3 4

10. Computer Science and Engineering Optimal Algorithms with Communication In-forests / out-forests on two processors Interval orders on n processors

11. Computer Science and Engineering In-forests / out-forests on two processors with Communication (model –C) Basic idea  Compensate e for communication cost using precedence arcs to find schedule length A 1 B 1 C 1 A 1 B 1 C 1 With communication Without communication 

12. Computer Science and Engineering Algorithm Main Steps Add new arcs to compensate for communication Schedule augmented graph using Hu’s algorithm Make assignment corrections Task Graph Augmented task Graph Schedule (correct length) Schedule (correct)

13. Computer Science and Engineering The depth of a node is defined as the length of the longest path from any node with depth zero to that node. A node with no predecessors has a depth of zero. In other words, depth(u) = 1+ max{depth(v)},  v  predecessors(u) ; and depth(u) = 0  u, predecessors(u) = . Given a schedule f, we define the operation Swapall(f,x,y), where x and y are two tasks in f scheduled to start at time t on processors i and j, respectively. The effect of this operation is to swap all the task pairs scheduled on processors i and j in the schedule f at time t 1,  t 1, t 1 ≥ t Node Depth Swapall Operation Definitions

14. Computer Science and Engineering Given an in-forest G = (V,A), identify the sets of siblings: S1, S2,..., Sk, where Si is the set of all nodes in V with a common child child(Si). A1 ← A 1.For every set Si i. Pick node u  Si with the maximum depth ii. A1 ← A1 - (v, child(Si)) "  v  Si and v ≠ u iii. A1 ← A1 U (v,u) "  v  Si and v ≠ u Obtain the schedule f by applying Hu’s Algorithm on the augmented in- forest F = (V,A1) Algorithm

15. Computer Science and Engineering For every set Si in the original in-forest G if node u (with the maximum depth) is scheduled in f in the time slot immediately before child(Si) but on a different processor, then apply the operation swapall(f,child(Si),x), where x is the task scheduled in the time slot immediately after u on the same processor. Algorithm (cont.)

16. Computer Science and Engineering