Scaling Conway’s Game of Life

Why do parallelism? Speedup – solve a problem faster. Accuracy – solve a problem better. Scaling – solve a bigger problem.

Problem 1: communication overhead Parallel algorithms that employ distributed memory have a certain amount of communication and a certain amount of computation. As more processes are added, the communication overhead increases. At some number of processes, communication overhead will cause the algorithm to take longer in parallel than serially.

Problem 2: Amdahl’s law Speedup is limited by the algorithm’s serial regions – the parts of the algorithm that cannot be parallelized. An algorithm will never be faster than its parallel regions. Speedup = where P = the proportion of the program that can be made parallel 1 – P = the proportion of the program that cannot be made parallel N = the number of processors

Strong scaling Strong scaling – increasing the number of processes but keeping the problem size constant.

The solution Don’t solve a problem faster, instead solve a bigger problem.

Gustafson’s law A bigger problem solved with more processors can be solved in the same amount of time it takes to solve a smaller problem with fewer processors.

Weak scaling Weak scaling – increasing the problem size as the number of processes increases.