Sieve of Eratosthenes

Sieve of Eratosthenes The Sieve of Eratosthenes is an algorithm to find the prime numbers between 2 and n Start with an array of booleans from 2 to n initially set to all true For each known prime starting with 2, mark all the multiples (composites) of that prime Stop when the next prime > √n What is left unmark are the primes

Sieve of Eratosthenes Next Prime = 2 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Next Prime = 2 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

Sieve of Eratosthenes Next Prime = 3 Next Prime = 5 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Next Prime = 5 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

Sieve of Eratosthenes Next Prime = 7 Next Prime = 11 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 Next Prime = 11 112=121 > 65 so stop

Sequential Sieve of Eratosthenes Algorithm Sieve_Eratosthenes() { boolean marked [n] = { true, ... }; prime = 2; while (prime * prime < n) { // or prime < sqrt(n) composite = prime * prime; // start with prime^2 while (composite < n) { marked[composite] = false; composite = composite + prime; // multiples of prime } do { // find next prime prime++; } while (marked[prime]);

Sequential Complexity The outermost loop will iterate at most √n times The 1st inner loop could iterate up to n/2 times The 2nd loop will iterate √n times total (for all iterations of the outermost loop) Complexity =

Parallelizing Sieve of Eratosthenes An obvious approach is to parallelize the loop marking multiples of prime composite = prime * prime; while (composite < n) { marked[composite] = false; composite = composite + prime; } We can rewrite this as a for loop: for (composite=prime*prime; composite < n; composite += prime) {

Parallelizing Sieve of Eratosthenes The difficulty is in selecting the next prime We have to a reduction and broadcast of the marked array so all processors can continue updating the array

Parallelizing Sieve of Eratosthenes while (prime * prime < n) { // Parallelize this loop for (composite=prime*prime; composite < n; composite += prime) { marked[composite] = false; MPI_Reduce (marked, ... , MPI_LAND, ...); MPI_Bcast (marked, ...); do { prime++; } while (marked[prime]); }

There is a fair bit of communication within each loop Furthermore, the size of the array can be large (n)

Parallelizing Sieve of Eratosthenes A better approach is to partition the array instead of the loop With p processors, each processor is responsible for elements of the array It is the master process’ responsibility to determine the next prime This need to be broadcast, but we can eliminate the reduction Also, only a single integer needs to be broadcast instead of an array

Sieve of Eratosthenes √n primes will be used. These need to be within the master process’ region. 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 p 0 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 p 1 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 p 2 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 p 3

Parallelizing Sieve of Eratosthenes First we need to make sure the master process will compute √n values The master process needs to be able to determine all values used for prime proc0_size = (n-1)/p; if (2 + proc0_size < (int) sqrt((double) n)) { if (rank == 0) printf (“Too many processors\n”); MPI_Finalize(); return 1; }

Parallelizing Sieve of Eratosthenes low_value = (rank*n-1)/p; high_value = ((rank+1)*n-1)/p; prime = 2; do { if (low_value % prime == 0) first = low_value; else first = low_value + prime – (low_value%prime); for (i=first + prime; i < high_value; i += prime) marked[i] = false; Partition the array: each processor is responsible for values from low to high Everyone starts at 2 Compute the first multiple of prime greater to or equal to low_value Mark multiple of prime in each processor’s range

Parallelizing Sieve of Eratosthenes if (rank == 0) { do { prime++; } while (marked[prime]); } MPI_Bcast (prime, ...); } while (prime * prime < n); MPI_Gather(marked, ...); Master determines next prime and broadcasts it. No reduction is needed until we finish the computation

Parallel Complexity The outermost loop will iterate √n times The 1st inner loop could iterate up to n/2p times with p processors The 2nd loop will iterate √n times total (for all iterations of the outermost loop) The broadcast take log(p) time Complexity = Communication Computation

Improving the Parallel Algorithm Still we have a broadcast within the outermost loop How can we eliminate that? Can we have all processors determine what the next prime is?

Improving the Parallel Algorithm The number of primes used in the Sieve of Eratosthenes is √n All processors compute the primes from 2 … √n Now all processors have their own private copy of the primes used We can eliminate the broadcast As well as the requirement that the master process’ section be at least √n

Complexity The complexity is essentially the same for most processors, except: Communication is eliminated until the end There is added complexity to compute the first √n primes sequentially Complexity to compute the first √n primes:

Complexity Final Complexity:

Questions

Discussion Question Question: Would it be better to implement this algorithm in shared-memory (using OpenMP) than distributed-memory (using MPI)?