1. Sorting CS-212 Dick Steflik

2. Exchange Sorting Method : make n-1 passes across the data, on each pass compare adjacent items, swapping as necessary (n-1 compares) O(n 2 )

3. Exchange Sorting int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = 0 ; j < (numel - 1) ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; } Basic Algorithm ( no improvements)

4. Exchange Sort int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, moves = 0; for ( i = 0 ; i < (numel - 1) ; i++ ) for ( j = i ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; } return moves; } Improvement #1 : don’t revist places that are already in the correct place

5. Exchange Sort int ExchangeSort ( int & num[ ], const int & numel ) { int i,j,k, swap,moves = 0; swap = 1; for ( i = 0 ; i < (numel - 1) ; i++ ) if ( swap != 0 ) { swap = 0; for ( j = 0 ; j < numel ; j++ ) if ( num [ j ] < num [ j-1 ]) { temp = num [ j ]; num [ j ] = num [ j - 1 ]; num [ j - 1] = temp; moves++; swap = 1; } return moves; } // Improvement # 2: // if a pass through the inner loop occurs // that doesn’t product a swap, you’re done. // both of the improvements improve the // overall performance but to not improve // O(n 2 )

6. 4 Exchange Sort 37524 compare 3 and 7 ; 7 is > 3 so advance 35724 compare 7 and 5, 7 > 5 so swap them 3527 compare 7 and 2, 7 >4 so swap them 35247 compare 7 and 4, 7 >4 so swap them End of pass 1; notice that 7 is in the right place

7. Exchange Sort - pass 2 35247 compare 3 and 5 ; 3 is >5 so advance 32547 compare 5 and 2, 5 < 2 so swap them 32457 compare 5 and 4, 5 > 4 so swap them 32457 compare 5 and 7, 5 < 7 so pass 2 done End of pass 2; notice that 5 and 7 are in the right places 35247

8. Exchange Sort - pass 3 23457 compare 3 and 2 ; 3 is >2 so swap them 23457 compare 3 and 4, 3 <42 so advance 23457 compare 4 and 5, 4 < 5 so advance 23457 compare 5 and 7, 5 < 7 so pass 3 done End of pass 3; notice that 4, 5 and 7 are in the right places 32457

9. Exchange Sort - pass n-1 (4) 23457 compare 2 and 3 ; 2 is < 3 so advance 23457 compare 3 and 4, 3 <4 so advance 23457 compare 4 and 5, 4 < 5 so advance 23457 compare 5 and 7, 5 < 7 so pass n-1 done End of pass 4; notice that everything is where it should be. 23457

10. Exchange Sort Improvements on each successive pass, do one less compare, because the last item from that pass is in place if you ever make a pass in which no swap occurs, the sort is complete These will both improve performance but Big O will remain O(n 2 )

11. Insertion Sort Strategy: divide the collection into two lists, one listed with one element (sorted) and the other with the remaining elements. On successive passes take an item from the unsorted list and insert it into the sorted list so the the sorted list is always sorted Do this until the unsorted list is empty

12. Insertion Sort 37524 sortedunsorted take an item from the unsorted list (7) and insert into the sorted list 37 524 sortedunsorted take next item from the unsorted list (5) and insert into the sorted list 357 24 sortedunsorted 2357 4 sortedunsorted 23457 sorted unsorted take next item from the unsorted list (2) and insert into the sorted list take next item from the unsorted list (4) and insert into the sorted list

13. Insertion Sort Void InsertionSort ( int A[ ], int n ) { int i, j; int temp; for ( i = i < n, i++ ) { // scan down list looking for correct place to put new element j = i ; temp = A[ i ]; while ( j < 0 && temp < A[ j-1 ]) { // shift the list 1 element to the right to make room for new element A[ j ] = A[ j+1 ]; j--; } A [ j ] = temp; }

14. Insertion Sort Note that each insertion could be O(n-1) and there are n-1 insertions being done therefore Big O is O(n 2 ) This is very much like building an ordered linked list except there is more data movement

15. Selection Sort Strategy: make a pass across the data looking for the largest item, swap the largest with the last item in the array. On successive passes (n-1) assume the array is one smaller (the last item is in the correct place) and repeat previous step

16. Selection Sort int SelectionSort ( int num [ ], int numelem ) { int i, j, min minidx, temp, moves = 0; for ( i = 0 ; i < numelem ; i++) { min = num [ i ] ; minidx = i ; for ( j = i + 1 ; j < numelem ; j++ ) { min = num[ j ] ; minidx = j ; } if ( min < num[ i ] ) { temp = num[ i ] ; num[ i ] = min ; num[ minidx ]; moves++; }

17. Selection Sort 37524 biggest last 34527 3452 biggestlast 3425 7 3425 7 7 biggestlast 324 32 23 57 574 574

18. Selection Sort Notice that in selection sort, there is the least possible data movement There are still n-1 compares on sublists that become one item smaller on each pass so, Big O is still O(n 2 ) This method has the best overall performance of the O(n 2 ) algorithms because of the limited amount of data movement

19. Heap Sort A heap is a tree structure in which the key at the root is max(keys) and the key of every parent is greater than the key of either of its children Visualize your array as a complete binary tree Arrange the tree into a heap by recursively reapplying the heap definition

20. Heap Sort repeat the following n-1 times –swap the root with the last element in the tree –starting at the root reinforce the heap definition –decrement the index of the last element

21. A Sample Heap 8 57 32 Notice: largest key is at root Notice: Keys of children are < key of parent

22. Heap Sort void HeapSort( int A[ ], int n ) { // this constructor turns the array into a max heap Heap H(A,n); int elt; for (int i = n-1 ; i >= 1 ; i--) { // delete smallest element from heap and place it in A[ i ] elt = H.Hdelete ( ); // Hdelete deletes the largest element and reenforces the heap property A[ I ] = elt ; }

23. Visualize array to sort as tree 37524 3 75 24 0 1 2 3 4 0 1 2 34

24. Make into a heap 3 75 24 0 1 34 7 35 24 0 1 34 7 45 23 0 1 34 STEP 1STEP 2STEP 3 Start at root, compare root to the children, swap root with largest child Original ArrayRepeat on preorder traversal path. At this point we have a heap Note: Larger arrays will take more steps

25. Now the sort starts 7 45 23 0 1 34 3 45 27 0 1 34 Swap the root and the last node Notice what is left is no longer a heap 5 43 27 0 1 34 Re heapize it by swaping the root with its largets child 2 3 2 2

26. Sorting... 2 43 57 0 1 34 2 Swap the root and the last node Notice what is left is no longer a heap 4 23 57 1 34 2 Reenforce the heap property on the remaining tree 0

27. Still Sorting... 3 24 57 1 34 2 Swap the root with the last element 0 3 24 57 1 34 2 Reenforce the heap property if necessary 0 2 34 57 1 34 2 Swap the root with the last element 0

28. And what we have left is... 23457 0 1 2 3 4 2 34 57 1 3 4 2 0

29. The Merge Principle Assume there exist two sorted lists (with queue behavior) 7531 8642 753 1 8642 Compare the items at the front of the list and move the smaller to the back of a third list Do it again 753 1 864 2

30. The Merge Principle 75 31 864 2 And again, and again, and again…until And again 75318642 We have a list whose length is the sum of the lengths and contains all of the elements of both lists, and this list is also ordered

31. Merge Sort void MergeSort ( int data[ ], int n ) { int n1, n2 ; // size of first subarray and second subarrays respectively if (n > 1 ) { n1 = n / 2 ; n2 = n - n1; MergeSort ( data, n1); // sort from data[0] to data[n1-1] MergeSort ((data + n1), n2); // sort from data[n1] to end // merge the two sorted halves Merge (data, n1, n2 ); }

32. Merge Sort 75175121924427823143331134 43 Picture the given list as a collection of n, 1 element sorted lists (i.e. 75 is a sorted 1 element list, as is 17. Now merge the adjacent lists of length 1 into lists of length 2…. 1775512192442782314333341143...now merge the lists of length 2 into lists of length 4 5171275419242723148343311343...now merge the lists of length 4 into lists of length 8 4512171924433433272314118375...now merge the lists of length 8 into lists of length 16 3458111214754334332723 1917 …and there you have it merge sort

33. Quick Sort This sorting method by far outshines all of the others for flat out speed Average run time is O(nlog 2 n) there are problems, worst case performance is O(n 2 ) when data is already in sorted order or is almost in sorted order (we’ll analyze this separately) and there are solutions to the problems and there is an improvement to make it faster still

34. Quick Sort 23175121924427826143331134 43 23175121924427826143331134 43 23175121924427826 14 3331134 43 23175121924427826 14 3331134 43 23175121924427826 14 3331134 43 Note : 23 is now in the right place and everything to its left is 23 Pick the leftmost element as the pivot (23). Now, start two cursors (one at either end) going towards the middle and swap values that are > pivot (found with left cursor) with values < pivot (found with right cursor) swap Finally, swap the pivot and the value where the cursors passed each other

35. Quick Sort 23175121924427826 14 3331134 43 Now, repeat the process for the right partition 175121948 14 311 swap 17512194814311 swap 17512194814311 swap 17512194814311 swap Note: the 11 is now in the right place, and the left partition is all pivot

36. Quick Sort 17512194814311 Now, repeat the process with the right partition 5483 Notice that there is nothing to swap, so swap the pivot and the 4, now the 8 is on the right place Repeat the process on the leftmost partition again 543 5483 543 The 4 is now in the right place and the left and right partitions are both of size one so they must also be in the right place

37. Quick Sort 1751219481431123 Now that we’ve exhausted the left partitions, back up and do the right partition 17121914 swap 171214 swap Since the left partition is just two items, just compare and swap if necessary 1214 1751219481431123 Now back up and do the remaining right partition

38. Quick Sort 17512194814311232427263334 43 242726333443 swap 242726333443 24272633 swap 272633 24272633 1751219481431123242726333443 All done

39. Quick Sort (worst case) If the data is already sorted watch what happens to the partitions 1751219481431123242726333443 There is nothing to swap 175121948141123242726333443 Again, nothing to swap….. The partitions are always the maximum size and the performance degrades to O(n 2 )

40. Quick Sort (worst case solution) The problem comes from the way we picked the pivot Pick the pivot a different way –median-of-three instead of always picking the leftmost value of the partition use the median of the first, the middle and the last value in the partition.

41. Quick Sort Improvement Since Quick sort is a recursive algorithm, a lot of time gets eaten up in the recursion involved with sorting the small partitions Solution - use a different algorithm for the small partitions –remember for small collections of data the n 2 algorithms perform better than the log 2 n algorithms –this is one place where exchange sort excels if the partition size is 15 or less use exchange (or selection sort)

42. Radix Sort