1. CS 584 Lecture 5
Assignment. Due NOW!!

2. Review Partition Communication Avoid centralized patterns
Avoid sequential patterns
Divide and Conquer

3. Agglomeration Partition and Communication steps were abstract
Agglomeration moves to concrete.
Combine tasks to execute efficiently on some parallel computer.
Consider replication.

4. Agglomeration Goals Reduce communication costs by
increasing computation
decreasing/increasing granularity
Retain flexibility for mapping and scaling.
Reduce software engineering costs.

5. Changing Granularity
A large number of tasks does not necessarily produce an efficient algorithm.
We must consider the communication costs.
Reduce communication by
having fewer tasks
sending less messages (batching)

6. Surface to Volume Effects
Communication is proportional to the surface of the subdomain.
Computation is proportional to the volume of the subdomain.
Increasing computation will often decrease communication.

7. How many messages total?
How much data is sent?

8. How many messages total?
How much data is sent?

9. Replicating Computation
Trade-off replicated computation for reduce communication.
Replication will often reduce execution time as well.

10. Summation of N Integers
s = sum
b = broadcast
How many steps?

11. Using Replication (Butterfly)

12. Using Replication
Butterfly to Hypercube

13. Avoid Communication
Look for tasks that cannot execute concurrently because of communication requirements.
Replication can help accomplish two tasks at the same time, like:
Summation
Broadcast

14. Preserve Flexibility Create more tasks than processors.
Overlap communication and computation.
Don't incorporate unnecessary limits on the number of tasks.

15. Agglomeration Checklist
Reduce communication costs by increasing locality.
Do benefits of replication outweigh costs?
Does replication compromise scalability?
Does the number of tasks still scale with problem size?
Is there still sufficient concurrency?

16. Assignment Exercises 2.1, 2.2, 2.4, and 2.12
When asked to design an algorithm, go through the PCAM design stages.