1. Kruse/Ryba ch081 Object Oriented Data Structures Sorting Insertion Sort Selection Sort Index Sort Shell Sort Divide and Conquer Sort Merge Sort Quick Sort Heaps and Heap Sort

2. Kruse/Ryba ch082 Sorting There is part of me that wants to write, part that wants to theorize, a part that wants to sculpt, a part that wants to teach.... To force myself into a single role, to decide to be just one thing in life, would kill off large parts of me. My career will form behind me. All I can do is let this day come to me in peace. All I can do is take the step before me now, and not fear repeating an effort or making a new one. Hugh Prather Notes to Myself

3. Kruse/Ryba ch083 Sortable Lists template class Sortable_list:public List { public: // Add prototypes for sorting methods here. private: // Add prototypes for auxiliary functions here. };

4. Kruse/Ryba ch084 Insertion Sort - contiguous "Cow" "Dog" "Horse" "Jackass" "Yak" "Ilama"

5. Kruse/Ryba ch085 Insertion Sort - contiguous "Cow" "Dog" "Horse" "Jackass" "Yak" "Iquana"

6. Kruse/Ryba ch086 template void Sortable_list ::insertion_sort() { int i; // first unsorted entry int j; // searches sorted part Record current; //entry temporarily removed for (i = 1; i 0 && entry[j - 1] > current); entry[j] = current; } { // for loop }

7. Kruse/Ryba ch087 Insertion Sort - linked "Cow""Dog""Horse" "Jackass""Yak""Llama"

8. Kruse/Ryba ch088 Insertion Sort - linked "Cow""Dog""Horse" "Jackass""Yak""Llama" It goes here!!!!

9. Kruse/Ryba ch089 Insertion Sort - linked "Cow""Dog""Horse" "Jackass""Yak""Llama"

10. Kruse/Ryba ch0810 { Node *first_unsorted, // the first unsorted *last_sorted, // tail of the sorted sublist *current, // used to traverse the sorted sublist *trailing; // one position behind current

11. Kruse/Ryba ch0811 if (head != NULL) { last_sorted = head; while (last_sorted->next != NULL) { first_unsorted = last_sorted->next; if (first_unsorted->entry entry) { last_sorted->next = first_unsorted->next; first_unsorted->next = head; head = first_unsorted; } else { trailing = head; current = trailing->next; while (first_unsorted->entry > current->entry) { trailing = current; current = trailing->next; } if (first_unsorted == current) last_sorted = first_unsorted; else { last_sorted->next = first_unsorted->next; first_unsorted->next = current; trailing->next = first_unsorted; }

12. Kruse/Ryba ch0812 Analysis Verifies that a list is correctly sorted as quickly as can be done. Good whenever a list is nearly in the correct order and a few entries are many position away from correct locations.

13. Kruse/Ryba ch0813 Selection Sort Still relatively easy to prove correct. Performs only O ( n ) swaps. Still performs O ( n 2 ) comparisons.

14. Kruse/Ryba ch0814 Selection Sort table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 Create a variable largest initialize to zero 0

15. Kruse/Ryba ch0815 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 0 create a variable j and initialize it to 1 j Create a variable largest Selection Sort

16. Kruse/Ryba ch0816 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 0 create a variable j and initialize it to 1 j Create a variable largest Selection Sort

17. Kruse/Ryba ch0817 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 0 compare the contents of the cells specified by largest and j. If the data in largest is less than the data in position j, copy j to largest j Create a variable largest Selection Sort

18. Kruse/Ryba ch0818 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 0 advance j by 1 and compare again. j Create a variable largest Selection Sort

19. Kruse/Ryba ch0819 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 3 advance j and compare again. This time there is a change in largest j Create a variable largest Selection Sort

20. Kruse/Ryba ch0820 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 3 the last comparison takes place when j reaches top. There is no change and largest contains the location of the largest element. j Create a variable largest Selection Sort

21. Kruse/Ryba ch0821 table[0] table[1] table[2] table[3] table[4] 45 31 23 76 20 3 If the element found at largest is greater than the element found at top, swap them j Create a variable largest Selection Sort

22. Kruse/Ryba ch0822 table[0] table[1] table[2] table[3] table[4] 45 31 23 20 76 0 decrement top by one and start over top Create a variable largest Selection (Index) Sort

23. Kruse/Ryba ch0823 template void Sortable_list ::selection_sort() /* Post: The entries of the Sortable_list have been rearranged so that the keys in all the entries are sorted into nondecreasing order. Uses: max_key, swap. */ { for (int position = count - 1; position > 0; position--) { int max = max_key(0, position); swap(max, position); } }

24. Kruse/Ryba ch0824 template int Sortable_list ::max_key(int low, int high) /*Pre: low and high are valid positions in the Sortable_list and low <= high. Post: The position of the entry between low and high with the largest key is returned.*/ { int largest, current; largest = low; for (current = low + 1; current <= high; current++) if (entry[largest] < entry[current]) largest = current; return largest; }

25. Kruse/Ryba ch0825 Shell Sort Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan

26. Kruse/Ryba ch0826 Shell Sort Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan

27. Kruse/Ryba ch0827 Shell Sort Jim Dot Eva Roy Tom Kim Guy Amy Jon Ann Tim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Amy Roy Tom Kim Guy Eva Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Tom Kim Guy Amy Jon Ann Jim Kay Ron Jan Tim Dot Eva Roy Ann Kim Guy Amy Jon Tom Jim Kay Ron Jan

28. Kruse/Ryba ch0828 Shell Sort Jim Dot Amy Jan Ann Kim Guy Eva Jon Tom Tim Kay Ron Roy

29. Kruse/Ryba ch0829 Shell Sort Jim Dot Amy Jan Ann Kim Guy Eva Jon Tom Tim Kay Ron Roy Jim Dot Amy Jan Ann Kim Guy Eva Jon Tom Tim Kay Ron Roy Jim Dot Amy Jan Ann Kim Guy Eva Jon Tom Tim Kay Ron Roy

30. Kruse/Ryba ch0830 Shell Sort Guy Dot Amy Jan Ann Kim Jim Eva Jon Ron Tim Kay Tom Roy Guy Ann Amy Jan Dot Kim Jim Eva Jon Ron Roy Kay Tom Tim Guy Ann Amy Jan Dot Jon Jim Eva Kay Ron Roy Kim Tom Roy

31. Kruse/Ryba ch0831 Shell Sort Guy Ann Amy Jan Dot Jon Jim Eva Kay Ron Roy Kim Tom Tim Amy Ann Dot Eva Guy Jan Jim Jon Kay Kim Ron Roy Tim Tom

32. Kruse/Ryba ch0832 template void Sortable_list ::shell_sort() { int increment, // spacing of entries start; // starting point increment = count; do { increment = increment / 3 + 1; for (start = 0; start < increment; start++) sort_interval(start, increment); // modified insertion sort } while (increment > 1); }

33. Kruse/Ryba ch0833 How Fast? Any algorithm that sorts a list of n entries by use of key comparisons must, in its worst case, perform at least comparisons of keys, and in the average case, it must perform at least comparison of keys.

34. Kruse/Ryba ch0834 Divide and Conquer void Sortable_list::sort() { if the list has length greater than 1 { partition the list into lowlist, highlist; lowlist.sort(); highlist.sort(); combine(lowlist, highlist); } }

35. Kruse/Ryba ch0835 Mergesort Chop into two sublists of near equal size Sort each sublist separately Merge two lists into one sorted list

36. Kruse/Ryba ch0836 template void sortable_list :: recursive_merge_sort(Node *&sub_list) { if (sub_list != NULL && sub_list->next != NULL) { Node *second_half = divide_from(sub_list); recursive_merge_sort(sub_list); recursive_merge_sort(second_half); sub_list = merge(sub_list, second_half); }

37. Kruse/Ryba ch0837 divide_from(Node *sub_list) { Node *position,*midpoint,*second_half; if ((midpoint = sub_list) == NULL) return NULL; // List is empty. position = midpoint->next; while (position != NULL) { position = position->next; if (position != NULL) { midpoint = midpoint->next; position = position->next; } } second_half = midpoint->next; midpoint->next = NULL; return second_half; }

38. Kruse/Ryba ch0838 Quicksort Choose one key from the list that will (hopefully) partition the list into two approximately equal sublists. Repeat for each of the sublists until the sublists are very small Sort the reduced sublists Put the sublists together

39. Kruse/Ryba ch08 Partitioning A key insight is that one can very quickly partition a vector into those elements that are larger than some value and those elements that are smaller. 297 173712461055804239

40. Kruse/Ryba ch08 Partitioning 297 173712461055804239 Choose an element, say element 37. Call this the pivot element. Divide the vector into those portions smaller than 37 and those larger than 37.

41. Kruse/Ryba ch08 Partitioning 297 173712461055804239 Divide the vector into those portions smaller than 37 and those larger than 37. 297 173712 46 10 55804239 pivot > pivot <= pivot

42. Kruse/Ryba ch08 Vizualization of Partitioning Algorithm The algorithm proceeds by first moving the pivot to the bottom of the vector, then moving low and high index counters towards each other. The area between the two index values is unknown. 297 173712461055804239 2 97 17 37 12461055804239 2 97 17 37 12461055804239 2 97 17 37 12461055804239

43. Kruse/Ryba ch08 2 97 17 37 12461055804239 2 97 17 37 12461055804239 2 97 17 37 12461055804239 2 97 17 37 12461055804239 2 10 17 37 12469755804239 2 10 17 37 12469755804239

44. Kruse/Ryba ch08 2 10 17 37 12469755804239 check 17 2 10 17 37 12469755804239 check 2 2 10 17 37 12469755804239 check 12 2 10 17 37 12469755804239 check 46 2 10 17 37 12974655804239 SWAP 2 10 17 37 12974655804239 check 46

45. Kruse/Ryba ch08 When the two indexes cross, the vector is divided into two parts, but the pivot is out of place. 2 10 17 37 12469755804239 A final swap is then used to put the pivot between the two sections. 2 10 17 12 37 469755804239 2 10 17 37 12469755804239 2 10 17 37 12469755804239 check 46

46. Kruse/Ryba ch0846 Quicksort, the Algorithm template void Sortable_list ::quick_sort() /* Post: The entries of the Sortable_list have been rearranged so that their keys are sorted into nondecreasing order. Uses: The contiguous List implementation of Chapter 6, recursive_quick_sort. */ { recursive_quick_sort(0, count - 1); }

47. Kruse/Ryba ch0847 template void sortable_list :: recursive_quick_sort(int low, int high) { int pivot; if (low < high) { // Otherwise, no sorting needed. pivot = partition(low, high); recursive_quick_sort(low, pivot - 1); recursive_quick_sort(pivot + 1, high); } }

48. Kruse/Ryba ch08 Selecting Good Pivots Execution can be improved if we try to select pivot elements in an intelligent fashion. Algorithms that have been proposed:

49. Kruse/Ryba ch08 Selecting Good Pivots Simply use the first element as the pivot value. This avoids the need for the initial swap, but is a relatively poor choice if the vector is somewhat, but not completely, ordered. Select the pivot at random from the range of values. Select the pivot as the value in the midpoint range of elements (this is the option used in our version of the algorithm) Select three values from the range, and compute the mean (middle) value from these three.

50. Kruse/Ryba ch08 Asymptotic Execution of Quicksort Best case occurs if we always select a pivot that divides vector in half. n n/2 n/4 n/8

51. Kruse/Ryba ch08 Asymptotic Execution of Quicksort In the worst case, the pivot gives one very long subvector and one very short one. n n-1 n-2 n-3 3 2 1 Average case is somewhere between, but usually very good.

52. Kruse/Ryba ch08 Timings for Quick sort 200 180 160 140 120 100 80 60 40 20 0 0 1000 2000 3000 4000 5000 seconds elements in input vector list insertion sort tree sort quick sort

53. Kruse/Ryba ch08 Heaps: An alternative data structure An alternative structure can be made into a more efficient priority queue. A minimum heap is a complete binary tree with the property that each node is smaller than (or equal to) all nodes in either left or right child subtree. A maximum heap is a complete binary tree with the property that each node is larger than (or equal to) all nodes in either left or right child subtree.

54. Kruse/Ryba ch08 2 35 91078 14121116 minimum heap

55. Kruse/Ryba ch08 2 35 91078 14121116 minimum heap 45 3642 26342830 15232210 maximum heap

56. Kruse/Ryba ch08 Vector Encoding 0456789 123 23591078141211 2 35 9 1078 141211 When vector encoding is applied to heaps, the smallest element is always held in the first (index zero) position.

57. Kruse/Ryba ch08 Adding an Element To add a new element, put it at the end of the array. But then the heap property may be destroyed. To restore the heap property, move elements down until the new value finds it's proper location.

58. Kruse/Ryba ch08 Adding an Element 2 35 91078 14121116 Insert a 4. It would be at the end of the active array. position

59. Kruse/Ryba ch08 Adding an Element 2 35 91078 14121116 If the number to be inserted is less than the parent (position/2) then move the element in (position/2) to position. position parent

60. Kruse/Ryba ch08 Adding an Element 2 35 910 7 8 14121116 If the number to be inserted is less than the parent (position/2) then move the element in (position/2) to position. position parent

61. Kruse/Ryba ch08 Adding an Element 2 3 5 910 7 8 14121116 If the number to be inserted is greater than or equal to the parent, you found the location to insert the new element. position parent

62. Kruse/Ryba ch08 Adding an Element 2 3 5 910 7 8 14121116 If the number to be inserted is greater than or equal to the parent, you found the location to insert the new element. position parent 4

63. Kruse/Ryba ch08 Removal Removing the smallest element is easy. What is hard is making it back into a complete binary tree. Solution: move the last element (bottom rightmost) into the tree at the right location. Again, take the " hole " at the top and move it around until you find where the new element should go.

64. Kruse/Ryba ch08 2 3 5 910 7 8 14121116 4 Remove the minimum value, leaving a hole.

65. Kruse/Ryba ch08 2 3 5 910 7 8 14121116 4 3 5 910 7 8 14121116 4 Remove the minimum value, leaving a hole.

66. Kruse/Ryba ch08 3 5 910 7 8 14121116 4

67. Kruse/Ryba ch08 3 5 910 7 8 14121116 4 3 5 9108 14121116 4 Fill the hole with the last element in the heap 7

68. Kruse/Ryba ch08 3 5 9108 14121116 4 7 Find the right place for the new element by exchanging it with it's smallest child until both children are bigger

69. Kruse/Ryba ch08 3 5 9108 14121116 4 7 Find the right place for the new element by exchanging it with it's smallest child until both children are bigger 3 5 9108 14121116 4 7 Wallah!! A heap.

70. Kruse/Ryba ch08 Application - Heap Sort Surprisingly fast Does not require any additional structures (above and beyond the input/output vector). Advantages

71. Kruse/Ryba ch0871 template void Sortable_list ::heap_sort() { Record current; int last_unsorted; build_heap();. for (last_unsorted = count - 1; last_unsorted > 0; last_unsorted--) { current = entry[last_unsorted]; entry[last_unsorted] = entry[0]; insert_heap(current, 0, last_unsorted - 1); }

72. Kruse/Ryba ch0872 template void Sortable_list ::build_heap() { int low; for (low = count / 2 - 1; low >= 0; low--) { Record current = entry[low]; insert_heap(current, low, count - 1); }

73. Kruse/Ryba ch0873 Comparison of Sorts Shell Selection O(n)O(n 2 ) AssignCompare InsertionO(n 2 ) HeapO(nlog n) QuickO(nlog n) RadixO(nk) MergeO(nlog n) O(n 1.25 -1.6 1.25 )

74. Kruse/Ryba ch08 Definition of Priority Queue Actually, not a queue at all. Instead, permits the removal of the smallest element contained in the collection.

75. Kruse/Ryba ch0875 template void Sortable_list :: insert_heap(const Record &current, int low, int high) { int large; large = 2 * low + 1; while (large = entry[large]) break; else { entry[low] = entry[large]; low = large; large = 2 * low + 1; } } entry[low] = current; }

76. Kruse/Ryba ch0876 Chapter 8 Sinks Into the Sea