CS 584

Sorting n One of the most common operations n Definition: –Arrange an unordered collection of elements into a monotonically increasing or decreasing order. n Two categories of sorting –internal (fits in memory) –external (uses auxiliary storage)

Sorting Algorithms n Comparison based –compare-exchange –O(n log n) n Noncomparison based –Uses known properties of the elements –O(n) - bucket sort etc.

Parallel Sorting Issues n Input and Output sequence storage –Where? –Local to one processor or distributed n Comparisons –How compare elements on different nodes n # of elements per processor –One (compare-exchange --> comm.) –Multiple (compare-split --> comm.)

Compare-Exchange

Compare-Split

Sorting Networks n Specialized hardware for sorting –based on comparator xyxyxyxy max{x,y} min{x,y} max{x,y}

Sorting Network

Parallel Sorting Algorithms n Merge Sort n Quick Sort n Bitonic Sort n Others …

Merge Sort n Simplest parallel sorting algorithm? n Steps –Distribute the elements –Everybody sort their own sequence –Merge the lists n Problem –How to merge the lists

Bitonic Sort n Key operation: –rearrange a bitonic sequence to ordered n Bitonic Sequence –sequence of elements –sequence of elements n There exists i such that is monotonically increasing and is monotonically decreasing or n There exists a cyclic shift of indicies such that the above is satisfied.

Bitonic Sequences n n –First it increases then decreases – i = 3 n n –Consider a cyclic shift –i will equal 3

Rearranging a Bitonic Sequence n Let s = n Let s = –a n/2 is the beginning of the decreasing seq. n Let s 1 = n Let s 1 = n Let s 2 = n Let s 2 = n In sequence s 1 there is an element b i = min{a i, a n/2+i } –all elements before b i are from increasing –all elements after b i are from decreasing n Sequence s 2 has a similar point n Sequences s 1 and s 2 are bitonic

Rearranging a Bitonic Sequence n Every element of s 1 is smaller than every element of s 2 n Thus, we have reduced the problem of rearranging a bitonic sequence of size n to rearranging two bitonic sequences of size n/2 then concatenating the sequences.

Rearranging a Bitonic Sequence

Bitonic Merging Network

What about unordered lists? n To use the bitonic merge for n items, we must first have a bitonic sequence of n items. n Two elements form a bitonic sequence n Any unsorted sequence is a concatenation of bitonic sequences of size 2 n Merge those into larger bitonic sequences until we end up with a bitonic sequence of size n

Mapping onto a hypercube n One element per processor n Start with the sorting network maps n Each wire represents a processor n Map processors to wires to minimize the distance traveled during exchange

Bitonic Merging Network

Bitonic Merge on Hypercube

Bitonic Sort Procedure BitonicSort for i = 0 to d -1 for j = i downto 0 if (i + 1) st bit of iproc <> j th bit of iproc comp_exchange_max(j, item) else comp_exchange_min(j, item) endif endfor comp_exchange_max and comp_exchange_min compare and exchange the item with the neighbor on the j th dimension

Bitonic Sort Stages

Assignment n Pick 16 random integers n Draw the Bitonic Sort network n Step through the Bitonic sort network to produce a sorted list of integers. n Explain how the if statement in the Bitonic sort algorithm works.