4. Adding numbers n data items, p processors ts = O(n)
tp = O(n/p) if data on each proc => S=ts/tp=O(p)
tp = O(n + n/p) if data needs broadcasting => S=ts/tp=o(1)

5. Sequential Recursion

7. Parallel Recursion tcomm = O(n/2 +n/4 + ..+ n/p) = O(n) S=o(1)
tcomp = O(n/2 +n/ n/p) = O(n)

8. tcomm = O(1 +1 + ..+ 1) = O(log p)
S=O(n / log p)
tcomp = O( ) = O(log p)

9. Parallel Bucket Sort

10. Sequential
m buckets , n numbers
ts = O(n + m((n/m) log (n/m))) = O(n log(n/m))

11. m buckets , n numbers,
p=m processors
tp = O(n + (n/p) log (n/p))

13. tp = O(n/p + (n/p) log (n/p)) = O( (n/p) log (n/p)) => S=O(p)

15. Det. Sample Sort

16. Det. Sample Sort
sort locally and create p-sample

17. Det. Sample Sort
send all p-samples to processor 1

18. Det. Sample Sort
proc.1: sort all received samples and compute global p-sample

19. Det. Sample Sort broadcast global p-sample
bucket locally according to global p-sample
send bucket i to proc.i
resort locally

20. Det. Sample Sort Lemma: Each proc. receives at most 2 n/p data items
global sample
global sample

21. Det. Sample Sort Post-Processing: “Array Balancing” 2 Rounds:
n/p
n/p
n/p
n/p
n/p
n/p
n/p
n/p 1 2 3 4 5 6 7 8 1 2 3 4 5 6 7 8
Post-Processing: “Array Balancing”
2 Rounds:
Each proc. sends rec. data size to all other proc.
Move data to right location via one h-relation

22. Det. Sample Sort 5 MPI_AlltoAllv for n/p > p2
O(n/p log n) local comp.
Goodrich (FOCS'98):
O(1) rounds
for n/p > pe

23. Performance: Det. Sample Sort

24. Numerical Integration

25. static assignment of processors
to segments of [a,b]
area = d (f(p)+f(q))/2

26. Problem: precision depends on curve’s shape

27. Adaptive Quadrature Terminate when C is sufficiently small
Problem: different parts of the curve need different resolution

28. segment 1
segment 3
segment 4
segment 2
segment 5

29. Gravitational N-Body Problem

32. ts = O(n2) per time step

39. for each time step:
for each object:
traverse tree to
determine its forces
Problem: traversals have
different lengths

40. object 1
object 3
object 5
object 2
object 4