Fall 2008Array Manipulation Algorithms1

Fall 2008Array Manipulation Algorithms2 Searching Let A = (a 1, a 2, …, a n ) be a sorted array of data such that a 1 < a 2 < …< a n. Given an element x, we are interested in finding out the index k, such that a k  x < a k+1. It works in O(n) time sequentially. The binary search method solves the search problem in O(log n) time.

Fall 2008Array Manipulation Algorithms3 Binary Search Algorithm Binary_Search Input: Array of data a 1 < a 2 < …< a n and an element x. Output: Index k such that a k  x < a k+1. BEGIN a 0 = －  ; a n+1 = +  ; left = 1; right = n; While (left  right) mid =  (left+right)/2  ; Case: x < a mid : right = mid – 1; x = a mid : Return(mid); x > a mid : left = mid + 1; End-case End-while Return(right); END.

Fall 2008Array Manipulation Algorithms4 Example of Binary Search Given array A = (20, 24, 25, 29, 32, 35, 39, 85) and x = 22. Initially: left = 1, right = 8, and mid = 4. x = 22 < a 4 = 29, set right = mid – 1 = 3. Iteration 1: left = 1, right = 3, and mid = 2. x = 22 < a 2 = 24, set right = mid – 1 = 1. Iteration 2: left = 1, right = 1, and mid = 1. x = 22 > a 1 = 20, set left = mid + 1 = 2. Now, left > right, return the value of right = 1 (= k).

Fall 2008Array Manipulation Algorithms5 Parallel Search Algorithm Parallel_Search Input: Array of data a 1 < a 2 < …< a n and an element x. Output: Index k such that a k  x < a k+1. BEGIN a 0 = －  ; a n+1 = +  ; For k = 0 to n do in parallel If a k  x and x < a k+1 then RESULT = k End-if End-parallel END. O(1) time, O(n) PEs in CREW PRAM model

Fall 2008Array Manipulation Algorithms6 Parallel Search with More Data When the length of the array is more than the number of processors the algorithm may not be implemented in O(1) time. Let p be the number of processors and n = p m. Array can be divided into p segments, and each segment will have r (= n/p) entries.

Fall 2008Array Manipulation Algorithms7 Set R i = 0 if x = a i  r = -1 if a i  r < x = 1 if x < a i  r If R i = 0, then k = i  r. If R i  0, then choose the segment j where R j-1 = － 1 and R j = 1. The value x falls in the segment j. Repeat the process in segment j until k is found. a 1, a 2, …, a r a r+1, a r+2, …, a 2r …a (p-1)r+1, …, a pr Segment 1Segment 2…Segment p

Fall 2008Array Manipulation Algorithms8 Algorithm for Search More Data Algorithm Parallel SearchMoreData Input: Array of data a 1 p. Output: Index k such that a k  x < a k+1. BEGIN If x < a 1 then Return (0); Else if x > a n then Return(n); End-if r = n / p; For i = 1 to p do in parallel Case: a i  r = x : R i = 0; a i  r < x : R i = － 1; a i  r > x : R i = 1; End-case End-parallel

Fall 2008Array Manipulation Algorithms9 For i = 1 to p do in parallel If R i = 0 then Return(i  r); End-if End-parallel R 0 = -1; For j = 1 to p do in parallel If R j = 1 and R j-1 = － 1 then If a j  r-1 = a (j-1)  r then Return(j  r-1) End-if Return( (j-1)  r + SearchMoreData(Sub-array in Segment j, x) ); End-if End-parallel END. O(log p n) time, O(p) PEs in CREW model

Fall 2008Array Manipulation Algorithms10 Searching in Unsorted Array Let A = (a 1, a 2, …, a n ) be an array of unsorted entries and x be an given value. We want to find the index k such that a k = x. If no such k exists, then the value returned is 0.

Fall 2008Array Manipulation Algorithms11 Algorithm Parallel_Search Input: Array A = (a 1, a 2, …, a n ) and an element x. Output: Index k such that a k = x, if not found, k = 0. (Assuming that there is only one k found.) BEGIN k = 0; For i = 1 to n do in parallel if a i = x then k = i; End-if End-parallel END. O(1) time, O(n) PEs in EREW PRAM model

Fall 2008Array Manipulation Algorithms12 Merging By Ranking Let A = (a 1, a 2, …, a m ) and B = (b 1, b 2, …, b n ) be two sorted arrays. Merging A and B means forming a new sorted array with the (m+n) elements of A and B. For example, if A = (2, 4, 11, 12, 14, 35, 95, 99) and B = (6, 7, 9, 25, 26, 31, 42, 85, 87, 102, 105), then the array got by merging A and B is C = (2, 4, 6, 7, 9, 11, 12, 14, 25, 26, 31, 35, 42, 85, 87, 95, 99, 102, 105). The sequential algorithm works in O(m+n) computing time.

Fall 2008Array Manipulation Algorithms13 Sequential Merging Algorithm Sequential Merge Input: Sorted arrays A and B of size m and n, respectively. Output: Merged sorted array C = (c 1, c 2, …, c m+n ). BEGIN a m+1 = b n+1 = +  ; i = j = k = 1; While k  m+n do If a i < b j then c k = a i ; i = i + 1; Else c k = b j ; j = j + 1; End-if k = k+ 1; End-while END;

Fall 2008Array Manipulation Algorithms14 Paralleling The Merging Let rank(x:A) being the number of entries in A which are less than or equal to x. For example: if A = (2, 4, 11, 12, 14, 35, 95, 99) and B = (6, 7, 9, 25, 26, 31, 42, 85, 87, 102, 105), then rank(6:A) = 2, rank(6:B) = 1, rank(9:A) = 2, rank(9:B) = 3, rank(25:A) = 5, rank(25:B) = 4. We further define that rank(B:A) is an array (r 1, r 2, …, r m ) where r i = rank(b i :A). rank(B:A) = (2, 2, 2, 5, 5, 5, 6, 6, 6, 8, 8) rank(B:B) = (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) rank(A:B) = (0, 0, 3, 3, 3, 6, 9, 9) rank(A:A) = (1, 2, 3, 4, 5, 6, 7, 8)

Fall 2008Array Manipulation Algorithms15 rank(x, A  B) = rank(x:A) + rank(x:B), when A and B are disjoint. rank(A:A  B) = (1, 2, 3, 4, 5, 6,,7,8) + (0, 0, 3, 3, 3, 6, 9, 9) = (1, 2, 6, 7, 8, 12, 16, 17) rank(B:A  B) = rank(B:A) + rank(B:B) = (2, 2, 2, 5, 5, 5, 6, 6, 6, 8, 8) + (1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11) = (3, 4, 5, 9, 10, 11, 13, 14, 15, 18, 19) Let RA = rank(A, A  B) and RB = rank(B, A  B), then C(RA i ) = A i and C(RB i ) = B i. For example: C(1) = a 1 = 2, C(6) = a 3 = 11, C(8) = a 5 = 14, C(17) = a 8 = 99, C(3) = b 1 = 6, C(9) = b 4 = 25, C(14) = b 8 = 85, C(19) = b 11 =105.

Fall 2008Array Manipulation Algorithms16 Parallel Merging Algorithm Parallel Merging and Ranking Input: Sorted arrays A and B of size m and n, respectively. Output: Merged sorted array C = (c 1, c 2, …, c m+n ). BEGIN For i  {1, 2, …, m} and j  {1, 2, …, n} do in parallel find rank(a i :A), rank(a i :B), rank(b j :A), rank(b j :B); End-parellel RA = rank(A:A) + rank(A:B); RB = rank(B:A) + rank(B:B); For i = 1 to m do in parallel C(RA i ) = A i ; End-parallel For i = 1 to n do in parallel C(RB i ) = B i ; End-parallel END Do in Parallel O(log n) time, O(m+n) PEs in CREW model n  m

Fall 2008Array Manipulation Algorithms17 Sorting Algorithms Computer manufacturers estimate that over 25 percent of the running time of computer is spent on sorting. Searching a particular entry is made easy only when the file is in a sorted form. Several sequential sorting algorithms can take O(n  log n) computing time.

Fall 2008Array Manipulation Algorithms18 Bubble Sort There is an unsorted array A = (8, 5, 7, 4, 3, 1, 2, 0). Sequential bubble sort: x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 Original Step Step … Step Setp … Comparison made: (n-1) + (n-2) + … + 1 = O(n 2 )

Fall 2008Array Manipulation Algorithms19 Parallel Bubble sort Let us employ four processors and assign the two numbers to each processor. For odd steps, processor P i : (x 2i-1, x 2i ), if x 2i-1 > x 2i, then interchange x 2i-1 and x 2i, 1  i  4. For even steps, processor P i : (x 2i, x 2i+1 ), if x 2i > x 2i+1, then interchange x 2i and x 2i+1, 1  i  3.

Fall 2008Array Manipulation Algorithms20 Example: Parallel Bubble Sort x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 Original Step Step Step Step Step Step Step Step

Fall 2008Array Manipulation Algorithms21 Parallel Algorithm Algorithm Parallel Bubble-Sort Input: Unsorted array A(1:n), n is even. Output: Sorted array A(1:n). BEGIN For k = 1 to n If k is odd then For i = 1 to i = n/2 do in parallel If A 2i-1 > A 2i then interchange them End-if End-parallel Else For i = 1 to i = (n-1)/2 do in parallel If A 2i > A 2i+1 then interchange them End-if End-parallel End-if End-for END. O(n) time, O(n) PEs in EREW PRAM model

Fall 2008Array Manipulation Algorithms22 Parallel Merge Sort There is an array x(1:n). If we consider x i alone as an array, it is sorted already. We can merge x 1 and x 2 to get the sorted array (a 1, a 2 ). Similarly, if (x 1, x 2 ) and (x 3, x 4 ) are sorted arrays, we can merge them to get the sorted array (a 1, a 2, a 3, a 4 ). x1x1 x2x2 x3x3 x4x4 x5x5 x6x6 x7x7 x8x8 Original Step Step Step

Fall 2008Array Manipulation Algorithms23 Parallel Algorithm Algorithm Parallel Merge-Sort Input: Unsorted array A(1:n), n = 2 k. Output: Sorted array A(1:n). BEGIN For i = 1 to n do in parallel T i = A i ; End-parallel p = n / 2; Do For j = 1 to p do in parallel T j = Parallel_Merging_Ranking(T 2j-1, T 2j ); End-parallel While (p = p / 2) > 0; A = T 1 ; END. O(log 2 n) time, O(n) PEs in CREW model

Fall 2008Array Manipulation Algorithms24 Sorting Networks It is possible to use a special purpose hardware to sort a collection of data. There are two preliminary sorting hardware which are capable of sorting two elements.  x y Max(x, y) Min(x, y) Upward Sorting Network  x y Min(x, y) Max(x, y) Downward Sorting Network

Fall 2008Array Manipulation Algorithms25 Sorting Four Elements  x1x1     x2x2 x3x3 x4x4 y1y1 y2y2 y3y3 y4y4