Sorting CS-212 Dick Steflik

Exchange Sorting Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1 compares) O(n 2 )

Exchange Sorting int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = 0 ; j < (numel - 1) ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; } Basic Algorithm ( no improvements)

Exchange Sort int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = i ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; } Improvement #1 : don’t revist places that are already in the correct place

Exchange Sort int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, swap,moves = 0; swap = 1; for ( i = 0 ; i < (numel - 1) ; i++ ) if ( swap != 0 ) { swap = 0; for ( j = 0 ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; swap = 1; } return moves; } // Improvement # 2: // if a pass through the inner loop occurs // that doesn’t product a swap, you’re done. // both of the improvements improve the // overall performance but to not improve // O(n 2 )

4 Exchange Sort compare 3 and 7 ; 7 is > 3 so advance compare 7 and 5, 7 > 5 so swap them 3527 compare 7 and 2, 7 >4 so swap them compare 7 and 4, 7 >4 so swap them End of pass 1; notice that 7 is in the right place

Exchange Sort - pass compare 3 and 5 ; 3 is >5 so advance compare 5 and 2, 5 < 2 so swap them compare 5 and 4, 5 > 4 so swap them compare 5 and 7, 5 < 7 so pass 2 done End of pass 2; notice that 5 and 7 are in the right places 35247

Exchange Sort - pass compare 3 and 2 ; 3 is >2 so swap them compare 3 and 4, 3 <42 so advance compare 4 and 5, 4 < 5 so advance compare 5 and 7, 5 < 7 so pass 3 done End of pass 3; notice that 4, 5 and 7 are in the right places 32457

Exchange Sort - pass n-1 (4) compare 2 and 3 ; 2 is < 3 so advance compare 3 and 4, 3 <4 so advance compare 4 and 5, 4 < 5 so advance compare 5 and 7, 5 < 7 so pass n-1 done End of pass 4; notice that everything is where it should be

Exchange Sort Improvements on each successive pass, do one less compare, because the last item from that pass is in place if you ever make a pass in which no swap occurs, the sort is complete These will both improve performance but Big O will remain O(n 2 )

Insertion Sort Strategy: divide the collection into two lists, one listed with one element (sorted) and the other with the remaining elements. On successive passes take an item from the unsorted list and insert it into the sorted list so the the sorted list is always sorted Do this until the unsorted list is empty

Insertion Sort sortedunsorted take an item from the unsorted list (7) and insert into the sorted list sortedunsorted take next item from the unsorted list (5) and insert into the sorted list sortedunsorted sortedunsorted sorted unsorted take next item from the unsorted list (2) and insert into the sorted list take next item from the unsorted list (4) and insert into the sorted list

Insertion Sort Void InsertionSort ( int A[ ], int n ) { int i, j; int temp; for ( i = i < n, i++ ) { // scan down list looking for correct place to put new element j = i ; temp = A[ i ]; while ( j < 0 && temp < A[ j-1 ]) { // shift the list 1 element to the right to make room for new element A[ j ] = A[ j+1 ]; j--; } A [ j ] = temp; }

Insertion Sort Note that each insertion could be O(n-1) and there are n-1 insertions being done therefore Big O is O(n 2 ) This is very much like building an ordered linked list except there is more data movement

Selection Sort Strategy: make a pass across the data looking for the largest item, swap the largest with the last item in the array. On successive passes (n-1) assume the array is one smaller (the last item is in the correct place) and repeat previous step

Selection Sort int SelectionSort ( int num [ ], int numelem ) { int i, j, min minidx, temp, moves = 0; for ( i = 0 ; i < numelem ; i++) { min = num [ i ] ; minidx = i ; for ( j = i + 1 ; j < numelem ; j++ ) { min = num[ j ] ; minidx = j ; } if ( min < num[ i ] ) { temp = num[ i ] ; num[ i ] = min ; num[ minidx ]; moves++; }

Selection Sort biggest last biggestlast biggestlast

Selection Sort Notice that in selection sort, there is the least possible data movement There are still n-1 compares on sublists that become one item smaller on each pass so, Big O is still O(n 2 ) This method has the best overall performance of the O(n 2 ) algorithms because of the limited amount of data movement

Heap Sort A heap is a tree structure in which the key at the root is max(keys) and the key of every parent is greater than the key of either of its children Visualize your array as a complete binary tree Arrange the tree into a heap by recursively reapplying the heap definition

Heap Sort repeat the following n-1 times –swap the root with the last element in the tree –starting at the root reinforce the heap definition –decrement the index of the last element

A Sample Heap Notice: largest key is at root Notice: Keys of children are < key of parent

Heap Sort void HeapSort( int A[ ], int n ) { // this constructor turns the array into a max heap Heap H(A,n); int elt; for (int i = n-1 ; i >= 1 ; i--) { // delete smallest element from heap and place it in A[ i ] elt = H.Hdelete ( ); // Hdelete deletes the largest element and reenforces the heap property A[ I ] = elt ; }

Visualize array to sort as tree

Make into a heap STEP 1STEP 2STEP 3 Start at root, compare root to the children, swap root with largest child Original ArrayRepeat on preorder traversal path. At this point we have a heap Note: Larger arrays will take more steps

Now the sort starts Swap the root and the last node Notice what is left is no longer a heap Re heapize it by swaping the root with its largets child

Sorting Swap the root and the last node Notice what is left is no longer a heap Reenforce the heap property on the remaining tree 0

Still Sorting Swap the root with the last element Reenforce the heap property if necessary Swap the root with the last element 0

And what we have left is

The Merge Principle Assume there exist two sorted lists (with queue behavior) Compare the items at the front of the list and move the smaller to the back of a third list Do it again

The Merge Principle And again, and again, and again…until And again We have a list whose length is the sum of the lengths and contains all of the elements of both lists, and this list is also ordered

Merge Sort void MergeSort ( int data[ ], int n ) { int n1, n2 ; // size of first subarray and second subarrays respectively if (n > 1 ) { n1 = n / 2 ; n2 = n - n1; MergeSort ( data, n1); // sort from data[0] to data[n1-1] MergeSort ((data + n1), n2); // sort from data[n1] to end // merge the two sorted halves Merge (data, n1, n2 ); }

Merge Sort Picture the given list as a collection of n, 1 element sorted lists (i.e. 75 is a sorted 1 element list, as is 17. Now merge the adjacent lists of length 1 into lists of length 2… now merge the lists of length 2 into lists of length now merge the lists of length 4 into lists of length now merge the lists of length 8 into lists of length …and there you have it merge sort

Quick Sort This sorting method by far outshines all of the others for flat out speed Average run time is O(nlog 2 n) there are problems, worst case performance is O(n 2 ) when data is already in sorted order or is almost in sorted order (we’ll analyze this separately) and there are solutions to the problems and there is an improvement to make it faster still

Quick Sort Note : 23 is now in the right place and everything to its left is 23 Pick the leftmost element as the pivot (23). Now, start two cursors (one at either end) going towards the middle and swap values that are > pivot (found with left cursor) with values < pivot (found with right cursor) swap Finally, swap the pivot and the value where the cursors passed each other

Quick Sort Now, repeat the process for the right partition swap swap swap swap Note: the 11 is now in the right place, and the left partition is all pivot

Quick Sort Now, repeat the process with the right partition 5483 Notice that there is nothing to swap, so swap the pivot and the 4, now the 8 is on the right place Repeat the process on the leftmost partition again The 4 is now in the right place and the left and right partitions are both of size one so they must also be in the right place

Quick Sort Now that we’ve exhausted the left partitions, back up and do the right partition swap swap Since the left partition is just two items, just compare and swap if necessary Now back up and do the remaining right partition

Quick Sort swap swap All done

Quick Sort (worst case) If the data is already sorted watch what happens to the partitions There is nothing to swap Again, nothing to swap….. The partitions are always the maximum size and the performance degrades to O(n 2 )

Quick Sort (worst case solution) The problem comes from the way we picked the pivot Pick the pivot a different way –median-of-three instead of always picking the leftmost value of the partition use the median of the first, the middle and the last value in the partition.

Quick Sort Improvement Since Quick sort is a recursive algorithm, a lot of time gets eaten up in the recursion involved with sorting the small partitions Solution - use a different algorithm for the small partitions –remember for small collections of data the n 2 algorithms perform better than the log 2 n algorithms –this is one place where exchange sort excels if the partition size is 15 or less use exchange (or selection sort)

Radix Sort