1. Computer Science 320 Parallel Computing Design Patterns

2. Problem Solving: How to Start? See if your problem fits into a class of problems that have already been solved Look for a suggestion for your solution in that class of solutions

3. Design Patterns A design pattern provides a template for suggested solutions to a class of siliarly structured problems Identify a design pattern that best matches your problem

4. Parallel Design Patterns Three patterns in 1989 paper by Carriero and Gelernter: –Result parallelism –Agenda parallelism –Specialist parallelism

5. Result Parallelism Good for processing each element in a data structure, such as the pixels in an image or the frames in a movie Ideally, the results of the computations are independent of each other

6. Result Parallelism Bottleneck: sequential dependencies, where one result must await the computation of another Positions of multiple stars in a time sequence Spreadsheet recalculations

7. Result Parallelism with Dependencies

8. Agenda Parallelism Good for computing one result from a large number of inputs See if any DNA sequences match a query sequence May also run into sequential dependencies, where tasks must wait

9. Agenda Parallelism: BLAST Basic Local Alignment Search Tool Unlike result parallelism, only interested in some results or combination thereof

10. Agenda Parallelism with Reduction Compute in parallel and then apply a reduction operator

11. Specialist Parallelism Each processor performs a specialized task on a series of data items (also known as pipelining)

12. Specialist Parallelism For each star Calculate position Render image Store in PNG file

13. What if There Aren’t Enough Processors? Large problems have billions of results to compute or tasks to perform, but we don’t yet have billions of processors The specialist pattern usually requires fewer processors

14. Result Pattern: Clumping/Slicing Clumping: lump many conceptual processors into one real processor Slicing: partition a data structure into pieces and dedicate a process to each piece

15. Agenda Pattern: Clumping/Slicing Clumping: lump many conceptual processors into one real processor Slicing: partition a data structure into pieces and dedicate a process to each piece

16. Agenda Pattern: Master-Worker A conceptual design usually for clusters The master processor manages the agenda of tasks, and delegates these to the worker processors The master receives the results and combines them

17. For Next Time Introduction to parallel Java, and a first parallel program!