Divide and Conquer Sorting Algorithms CS 105

10/02/05 HeapSort Slide 2 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved A new sorting strategy Divide-and-Conquer Given the collection of n elements to sort: perform the sort in three steps Divide step: split the collection S into two subsets, S1 and S2 Recur step: sort S1 and S2 separately Conquer step: combine the two lists into one sorted list

10/02/05 HeapSort Slide 3 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Quick sort and Merge sort Two algorithms adopt this divide-and-conquer strategy Quick sort Work is carried out in the divide step using a pivot element Conquer step is trivial Merge sort Divide step is trivial – just split the list into two equal parts Work is carried out in the conquer step by merging two sorted lists

10/02/05 HeapSort Slide 4 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Quick sort: divide step In the divide step, select a pivot from the array (say, the last element) Split the list/array S using the pivot: S1 consists of all elements pivot S1 S2

10/02/05 HeapSort Slide 5 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Quick sort: conquer step After sorting S1 and S2, combine the sorted lists so that S1 is on the left, the pivot is in the middle, and S2 is on the right S2-sortedS1-sorted

10/02/05 HeapSort Slide 6 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Quick sort with recur step S2-sortedS1-sorted S1 S2 Divide Conquer Recur

10/02/05 HeapSort Slide 7 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Implementing quick sort It is preferable if we can perform quick-sort in-place; i.e., we sort by swapping elements in the array, perhaps using some temporary variables Plan: algorithm QSort( S, a, b ) sorts the sublist in the array S from index a to index b QSort( L, 0, n-1 ) will sort an array L of length n Within the QSort( S, a, b ) algorithm, there will be recursive calls to QSort on smaller ranges within the range a…b.

10/02/05 HeapSort Slide 8 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Algorithm QSort Algorithm QSort( S, a, b ) if (a < b) { p=S[b]; /* rearrange S so that: S[a]…S[x-1] are elements < p S[x] = p S[x+1]…S[b] are elements > p */ x = partition(S, a, b); QSort( S, a, x-1 ); QSort( S, x+1, b ); } base case: a…b range contains 0 or 1 element

10/02/05 HeapSort Slide 9 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved int partition(S, a, b) p=S[b]; l=a; r=b-1; while(l <= r) { // find an element larger than the pivot while(l <= r && S[l] <= p ) ++l; // find an element smaller than the pivot while(r >= l && S[r] >= p ) --r; if(l < r) swap ( S[l], S[r] ); } swap( S[l], S[b] ) // place pivot in proper place return l;

10/02/05 HeapSort Slide 10 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Time complexity of quick sort First, note that rearranging a sublist takes O( n ) time, where n is the length of the sublist Requires scanning the list from both ends until both l and r pointers meet O( n ) even if loops are nested within a loop Rearranging sublists is all that the quick sort algorithm does Need to find out how often the sort would perform the rearrange operation

10/02/05 HeapSort Slide 11 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Time complexity of quick sort Suppose the pivots always split the lists into two lists of roughly equal size... 1 list of length n 2 lists of length n/2 4 lists of length n/4 n lists of length 1

10/02/05 HeapSort Slide 12 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Time complexity of quick sort Each level takes O( n ) time for sublist rearranging Assuming an even split caused by each pivot, there will be around log n levels Therefore, quick sort takes O( n log n ) time But…

10/02/05 HeapSort Slide 13 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Time complexity of quick sort In the worst case, the pivot might split the lists such that there is only 1 element in one partition (not an even split) There will be n levels Each level requires O( n ) time for sublist rearranging Quick sort takes O( n 2 ) time in the worst case

10/02/05 HeapSort Slide 14 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Merge sort Another sorting algorithm using the divide-and- conquer paradigm This time, the hard work is carried out in the conquer phase instead of the divide phase Divide: split the list S[0..n-1] by taking the middle index m ( = (0 + n-1) / 2 ) Recur: recursively sort S[0..m] and S[m+1..n-1] Conquer: merge the two sorted lists (how?)

10/02/05 HeapSort Slide 15 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Merge sort S2-sortedS1-sorted S1 S2 Divide Conquer Recur

10/02/05 HeapSort Slide 16 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Merging two sorted lists Requires inspecting the “head elements” of both lists Whichever element is smaller, that elements goes first, and the head element for the “winning” list is updated so that it refers to the next element in that list Repeat the process until all the elements in both lists have been processed

10/02/05 HeapSort Slide 17 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Merge sort time complexity Divide step ensures that the sublist split is done evenly O( log n ) levels Conquer/merge step takes O( n ) time per level Time complexity is O( n log n ), guaranteed Disadvantage: hard to carry out the merge step in-place; temporary array/list is necessary if we want a simple implementation

10/02/05 HeapSort Slide 18 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Merge Two Sorted Sublists merge(Item[]a, Item[]aux, int l, int m, int r) { // subarray a[l:m] is sorted; copy to aux[l:m] for(i=l; i<=m; ++i) aux[i] = a[i]; // subarray a[m+1:r] is sorted; copy to aux[r:m+1] for(i=m+1, j=r; j>m; ++i, --j) aux[j] = a[i]; // merge aux[l:m] and aux[m+1:r] into a[l:r] for(i=l, j=r, k=l; k<=r; ++k) { if(aux[i] < aux[j]) a[k] = aux[i++]; else a[k] = aux[j--]; }}

10/02/05 HeapSort Slide 19 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Top-Down Recursive Mergesort mergesort(Item[]a, Item[]aux, int l, int r) { if(l >= r) return; m = (l+r)/2; mergesort(a, aux, l, m); mergesort(a, aux, m+1, r); merge(a, aux, l, m, r); }

10/02/05 HeapSort Slide 20 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Buttom-Up Non-recursive Mergesort mergesort(Item[]a, Item[]aux, int l, int r) { if(l >= r) return; for(m=1; m<=r-l; m=mm) { mm=m+m; for(i=l; i<=r-m; i+=mm) { x=i+mm-1; last=((x<r)?x:r); merge(a, aux, i, i+m-1, last); }}}

10/02/05 HeapSort Slide 21 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Time complexity summary AlgorithmBest case Worst case Heap sortO( n log n ) Quick sortO( n log n )O(n 2 ) Merge sortO( n log n )

10/02/05 HeapSort Slide 22 Copyright 2005, by the authors of these slides, and Ateneo de Manila University. All rights reserved Summary and final points Quick sort has a bad worst case but works very well in practice; O( n log n ) on the average Merge sort is difficult to implement in-place but O( n log n ) complexity is guaranteed But it doesn’t perform well in practice Heap sort is a guaranteed O( n log n ) algorithm that is not that difficult to implement Reasonable alternative to quick sort