1. 1 Tuesday, September 26, 2006 Wisdom consists of knowing when to avoid perfection. -Horowitz

2. 2 §Quiz 2 §Assignment 1

3. 3 Hypercube: log p dimensions with two nodes in each dimension 0-D hypercube

4. 4 Hypercube: log p dimensions with two nodes in each dimension 1-D hypercube 0-D hypercube

5. 5 Hypercube: log p dimensions with two nodes in each dimension 2-D hypercube 1-D hypercube

6. 6 Hypercube: log p dimensions with two nodes in each dimension 3-D hypercube 2-D hypercube

7. 7 Hypercube: log p dimensions with two nodes in each dimension 3-D hypercube 4-D hypercube Each node is connected to d=log p other nodes

8. 8 Numbering Minimum distance between nodes

9. 9 §Diameter: Maximum distance between any two processing nodes in the network l Ring l 2-D Mesh l Hypercube

10. 10 §Diameter: Maximum distance between any two processing nodes in the network l Ring └p/2┘ l 2-D Mesh 2(√p -1) no-wraparound 2 └(√p /2) ┘ wraparound l Hypercube log p

11. 11 §Connectivity: Multiplicity of paths l Minimum arcs that need to be removed to disconnect the network into two §Ring 2 l 2-D Mesh 2 no-wraparound 4 wraparound l Hypercube d=log p

12. 12 §Bisection width: l Minimum arcs that need to be removed to partition the network into two equal halves §Ring 2 l 2-D Mesh √p no-wraparound 2√p wraparound l Hypercube p/2

13. 13

14. 14 Domain Decomposition §In this type of partitioning, the data associated with a problem is decomposed. Each parallel task then works on a portion of the data.

15. 15 Domain Decomposition

16. 16 Functional Decomposition

17. 17 Signal processing

18. 18 Climate modeling.

19. 19 Examples of decomposition and task dependencies

20. 20 Examples of decomposition and task dependencies.

21. 21 Examples of decomposition and task dependencies.

22. 22 Granularity §Fine vs. Coarse l Decomposition in large number of small tasks vs. small number of large tasks. §Maximum degree of concurrency §Average degree of concurrency §Concurrency vs. Granularity?

23. 23 Granularity

24. 24 Granularity §Critical Path length: l Longest directed path between any pair of start and finish nodes is critical path §Average degree of concurrency: l Ratio of total amount of work to the critical path length

25. 25 Granularity Another example

26. 26 Granularity Measure of the ratio of computation to communication. §Fine-grain Parallelism: l Facilitates load balancing l Implies high communication overhead and less opportunity for performance enhancement §Coarse-grain Parallelism: l High computation to communication ratio l Implies more opportunity for performance increase l Harder to load balance efficiently

27. 27 Granularity §Example: l Domain decompositions for a problem involving a three-dimensional grid.