1. Embarrassingly Parallel Computations
Chapter 3

2. Embarrassingly Parallel Computations
A computation that can obviously be divided into a number of completely independent parts, each of which can be executed by a separate process.
No communication or very little communication between processes
Each process can do its tasks without any interaction with others

3. Practical embarrassingly parallel computation
(static process creation / master-slave approach)

4. Embarrassingly Parallel Computations
Examples
Low level image processing
Many of such operations only involve local data with very limited if any communication between areas of interest.
Mandelbrot set
Monte Carlo Calculations

5. Some geometrical operations
Shifting
Object shifted by Dx in the x-dimension and Dy in the y-dimension:
x¢ = x + Dx
y¢ = y + Dy
where x and y are the original and x¢ and y¢ are the new coordinates.
Scaling
Object scaled by a factor Sx in x-direction and Sy in y-direction:
x¢ = xSx
y¢ = ySy
Rotation
Object rotated through an angle q about the origin of the coordinate system:
x¢ = x cosq + y sinq
y¢ = -x sinq + y cosq

6. Partitioning into regions for individual processes
Square region for each process (can also use strips)

7. Mandelbrot Set What is the Mandelbrot set?
Set of all complex numbers c for which sequence defined by the following iteration remains bounded:
z(0) = c, z(n+1) = z(n)*z(n) + c, n=0,1,2, ...
This means that there is a number B such that the absolute value of all iterates z(n) never gets larger than B.

8. computing value of one point
Sequential routine
computing value of one point
structure complex {
float real;
float imag; };
int cal_pixel(complex c)
{ int count, max;
complex z;
float temp, lengthsq;
max = 256;
z.real = 0; z.imag = 0;
count = 0; /* number of iterations */
do {
temp = z.real * z.real - z.imag * z.imag + c.real;
z.imag = 2 * z.real * z.imag + c.imag;
z.real = temp;
lengthsq = z.real * z.real + z.imag * z.imag;
count++;
} while ((lengthsq < 4.0) && (count < max));
return count; }

9. Static Task Assignment
Parallelizing Mandelbrot Set Computation
Static Task Assignment
Simply divide the region into fixed number of parts, each computed by a separate processor.
Disadvantage:
Different regions may require different numbers of iterations and time.
Dynamic
Task Assignment

10. Monte Carlo Methods Another embarrassingly parallel computation
Example:
calculate π using the ratio:
Randomly choose points within the square
Count the points that lie within the circle
Given a sufficient number of randomly selected samples
fraction of points within the circle will be: π /4

11. Example of Monte Carlo Method
Area = D2

12. Computing an Integral
Use random values of x to compute f(x) and sum values of f(x):
where xi are randomly generated values of x between x1 and x2.
Monte Carlo method is very useful if the function cannot be integrated numerically (maybe having a large number of variables)

13. Example: Sequential Code sum = 0;
for (i = 0; i < N; i++) { /* N random samples */
xr = rand_v(x1, x2); /* generate next random value */
sum = sum + xr * xr - 3 * xr; /* compute f(xr) */ }
area = (sum / N) * (x2 - x1);
Routine rand_v(x1, x2) returns a pseudorandom number between x1 and x2

14. Values for a "good" generator:
a=16807, m=231-1 (a prime number), c=0
This generates a repeating sequence of (231- 2) different numbers.

15. A Parallel Formulation
xi+1 = (axi + c) mod m
xi+k = (Axi+ C) mod m
A and C above can be derived by computing
xi+1=f(xi) , xi+2=f(f(xi)), xi+k=f(f(f(…f(xi))))
and using the following properties:
(A+B) mod M = [(A mod M) + (B mod M)] mod M
[ X(A mod M) ] mod M = (X.A mod M)
X(A + B) mod M = (X.A + X.B) mod M
= [(X.A mod M) + (X.B mod M)] mod M
[ X( (A+B) mod M) ] mod M = (X.A + X.B) mod M