1. 1 TCSS 342, Winter 2005 Lecture Notes Sorting Weiss Ch. 8, pp. 283-321

2. 2 Sorting Sorting = putting objects in order in array/list one of the fundamental problems in computer science comparison-based sorting: must determine order through comparison operations on the input data:, compareTo, … 152811710125 01234567

3. 3 1. Bogo sort Bogo sort orders a list of values by repetitively shuffling them and checking if they are sorted More specifically: scan the list, seeing if it is sorted if not, shuffle the values in the list and repeat This sorting algorithm has terrible performance! Can we deduce its runtime?

4. 4 Bogo sort code public static void bogoSort(int[] a) { while (!isSorted(a)) shuffle(a); } // Returns true if array a's elements // are in sorted order. public static boolean isSorted(int[] a) { for (int i = 0; i < a.length - 1; i++) if (a[i] > a[i+1]) return false; return true; }

5. 5 Bogo sort code, helpers // Shuffles an array of ints by randomly swapping each // element with an element ahead of it in the array. public static void shuffle(int[] a) { for (int i = 0; i < a.length - 1; i++) { // pick random number in [i+1, a.length-1] inclusive int range = a.length-1 - (i+1) + 1; int j = (int)(Math.random()*range + (i+1)); swap(a, i, j); } // Swaps a[i] with a[j]. private static void swap(int[] a, int i, int j) { if (i == j) return; int temp = a[i]; a[i] = a[j]; a[j] = temp; }

6. 6 Bogo sort runtime How long should we expect bogo sort to take? related to probability of shuffling into sorted order assuming shuffling code is fair, probability equals 1 / (number of permutations of n elements) bogo sort takes roughly factorial time to run note that if array is initially sorted, bogo finishes quickly! it should be clear that this is not satisfactory...

7. 7 2. Bubble sort Bubble sort orders a list of values by repetitively comparing neighboring elements and swapping their positions if necessary More specifically: scan the list, exchanging adjacent elements if they are not in relative order; this bubbles the highest value to the top scan the list again, bubbling up the second highest value repeat until all elements have been placed in their proper order

8. 8 Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 5 12 3542 77 101 1 2 3 4 5 6 Swap 4277 "Bubbling" largest element

9. 9 Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 5 12 3577 42 101 1 2 3 4 5 6 Swap 3577

10. 10 "Bubbling" largest element Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 5 12 7735 42 101 1 2 3 4 5 6 Swap 1277

11. 11 "Bubbling" largest element Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 No need to swap

12. 12 "Bubbling" largest element Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 Swap 5101

13. 13 "Bubbling" largest element Traverse a collection of elements Move from the front to the end "Bubble" the largest value to the end using pair- wise comparisons and swapping 77 1235 42 5 1 2 3 4 5 6 101 Largest value correctly placed

14. 14 Bubble sort code public static void bubbleSort(int[] a) { for (int i = 0; i < a.length; i++) for (int j = 1; j < a.length - i; j++) if (a[j-1] > a[j]) swap(a, j-1, j); } private static void swap(int[] a, int i, int j) { if (i == j) return; int temp = a[i]; a[i] = a[j]; a[j] = temp; }

15. 15 Bubble sort runtime Running time (# comparisons) for input size n: number of actual swaps performed depends on the data; out-of-order data performs many swaps

16. 16 3. Selection sort Selection sort orders a list of values by repetitively putting a particular value into its final position More specifically: find the smallest value in the list switch it with the value in the first position find the next smallest value in the list switch it with the value in the second position repeat until all values are in their proper places

17. 17 Selection sort example

18. 18 Index 01234567 Value 27631726458149 1 st pass 16327726458149 2 nd pass 19277264581463 3 rd pass 19147264582763 … Selection sort example 2

19. 19 Selection sort code public static void selectionSort(int[] a) { for (int i = 0; i < a.length; i++) { // find index of smallest element int min = i; for (int j = i + 1; j < a.length; j++) if (a[j] < a[min]) min = j; // O(1) // swap smallest element with a[i] swap(a, i, min); // O(1) }

20. 20 Selection sort runtime Running time for input size n: in practice, a bit faster than bubble sort. Why?

21. 21 4. Insertion sort Insertion sort orders a list of values by repetitively inserting a particular value into a sorted subset of the list More specifically: consider the first item to be a sorted sublist of length 1 insert the second item into the sorted sublist, shifting the first item if needed insert the third item into the sorted sublist, shifting the other items as needed repeat until all values have been inserted into their proper positions

22. 22 Insertion sort Simple sorting algorithm. n-1 passes over the array At the end of pass i, the elements that occupied A[0] … A[i] originally are still in those spots and in sorted order. 281511710125 01234567 128151710125 01234567 after pass 2 after pass 3

23. 23 Insertion sort example

24. 24 Names: Fred, Alice, David, Bill, and Carol Insertion sort example 2

25. 25 Insertion sort example 2, cont.

26. 26 Insertion sort example 2, cont.

27. 27 Worst-case for insertion sort

28. 28 Insertion sort code public static void insertionSort(int[] a) { for (int i = 1; i < a.length; i++) { int temp = a[i]; // slide elements down to make room for a[i] int j; for (j = i; j > 0 && a[j - 1] > temp; j--) a[j] = a[j - 1]; a[j] = temp; }

29. 29 Insertion sort runtime worst case: reverse-ordered elements in array. best case: array is in sorted ascending order. average case: each element is about halfway in order.

30. 30 Comparing sorts We've seen "simple" sorting algos. so far, such as: selection sort insertion sort They all use nested loops and perform approximately n 2 comparisons They are relatively inefficient comparisonsswaps selectionn 2 /2n insertion worst: n 2 /2 best: n worst: n 2 /2 best: n

31. 31 Average case analysis Given an array A of elements, an inversion is an ordered pair (i, j) such that i A[j].(out of order elements) Assume no duplicate elements. Theorem: The average number of inversions in an array of n distinct elements is n (n - 1) / 4. Corollary: Any algorithm that sorts by exchanging adjacent elements requires O(n 2 ) time on average.

32. 32 Merge sort Merge sort orders a list of values by recursively dividing the list in half until each sub-list has one element, then recombining More specifically: divide the list into two roughly equal parts recursively divide each part in half, continuing until a part contains only one element merge the two parts into one sorted list continue to merge parts as the recursion unfolds a "divide and conquer" algorithm

33. 33 Merge sort idea: Divide the array into two halves. Recursively sort the two halves (using merge sort). Use merge to combine the two arrays. sort merge(0, n/2, n-1) mergeSort(0, n/2-1)mergeSort(n/2, n-1) Merge sort

34. 34 674523146339842

35. 35 674523146339842 674523146339842

36. 36 674523146339842 674523146339842 45231498

37. 37 674523146339842 674523146339842 45231498 2398

38. 38 674523146339842 674523146339842 45231498 2398 Merge

39. 39 674523146339842 674523146339842 45231498 2398 23 Merge

40. 40 674523146339842 674523146339842 45231498 2398 2398 Merge

41. 41 674523146339842 674523146339842 45231498 23984514 2398

42. 42 674523146339842 674523146339842 45231498 23984514 Merge 2398

43. 43 674523146339842 674523146339842 45231498 23984514 Merge 2398

44. 44 674523146339842 674523146339842 45231498 23984514 45 Merge 239814

45. 45 674523146339842 674523146339842 45231498 23984514 Merge 98451423

46. 46 674523146339842 674523146339842 45231498 23984514 Merge 9814 2345

47. 47 674523146339842 674523146339842 45231498 23984514 Merge 2314 23 9845

48. 48 674523146339842 674523146339842 45231498 23984514 Merge 23984514 2345

49. 49 674523146339842 674523146339842 45231498 23984514 Merge 23984514 234598

50. 50 674523146339842 674523146339842 45231498 23984514 6763342 23984514 234598

51. 51 674523146339842 674523146339842 45231498 23984514 6763342 676 23984514 234598

52. 52 674523146339842 674523146339842 45231498 23984514 6763342 676 Merge 23984514 234598

53. 53 674523146339842 674523146339842 45231498 23984514 6763342 676 6 Merge 23984514 234598

54. 54 674523146339842 674523146339842 45231498 23984514 6763342 676 Merge 239845146 234598

55. 55 674523146339842 674523146339842 45231498 23984514 6763342 6763342 23984514676 14234598

56. 56 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 23984514676 14234598

57. 57 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 3323984514676 14234598

58. 58 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 422398451467633 14234598

59. 59 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 14234598

60. 60 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 2398451464233 14234598 6 67

61. 61 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 23984514633 14234598 633 6742

62. 62 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 2398451464233 14234598 63342 67

63. 63 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 14234598 6334267

64. 64 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 234598 334267 14 6

65. 65 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 234598 64267 6 14 33

66. 66 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 144598 64267 614 23 33

67. 67 4523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 142398 64267 61423 45 33

68. 68 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 142398 63367 6142333 45 42

69. 69 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 142398 63342 614233342 45 67

70. 70 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 142345 63342 61423334245 98 67

71. 71 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 14234598 6334267 6142333424567

72. 72 674523146339842 674523146339842 45231498 23984514 6763342 6763342 Merge 239845146764233 14234598 6334267 614233342456798

73. 73 674523146339842 674523146339842 45231498 23984514 6763342 6763342 239845146764233 14234598 6334267 614233342456798

74. 74 674523146339842 614233342456798

75. 75 136211894820 07 468913182021 07 1362118 03 94820 47 6131821 03 48920 47 136 01 2118 23 94 45 820 67 613 01 1821 23 49 45 820 67 13 0 6 1 21 2 18 3 9 4 4 5 8 6 20 7 Merge sort illustrated

76. 76 merge operation: Given two sorted arrays, merge operation produces a sorted array with all the elements of the two arrays A6131821 B48920 C468913182021 Running time of merge: O(n), where n is the number of elements in the merged array. Merging two sorted arrays

77. 77 Merge sort code public static void mergeSort(int[] a) { int[] temp = new int[a.length]; mergeSort(a, temp, 0, a.length - 1); } private static void mergeSort(int[] a, int[] tmp, int left, int right) { if (left >= right) return; // sort the two halves int mid = (left + right) / 2; mergeSort(a, tmp, left, mid); mergeSort(a, tmp, mid + 1, right); // merge the sorted halves into a sorted whole merge(a, tmp, left, mid, right); }

78. 78 Merge code private static void merge(int[] a, int[] temp, int left, int leftEnd, int rightEnd) { int count = rightEnd - left + 1; int right = leftEnd + 1; // main loop to copy the halves into the temp array for (int i = 0, l = left, r = right; i < count; i++) if (r > rightEnd) temp[i] = a[l++]; else if (l > leftEnd) temp[i] = a[r++]; else if (a[l] < a[r]) temp[i] = a[l++]; else temp[i] = a[r++]; // copy sorted temp array back into main array for (int i = 0; i < count; i++) a[left + i] = temp[i]; }

79. 79 Merge sort runtime let T(n) be runtime of merge sort on n items T(0) = 1 T(1) = 2*T(0) + 1 T(2) = 2*T(1) + 2 T(4) = 2*T(2) + 4 T(8) = 2*T(4) + 8... T(n/2) = 2*T(n/4) + n/2 T(n) = 2*T(n/2) + n

80. 80 Merge sort runtime T(n) = 2*T(n/2) + n T(n/2) = 2*T(n/4) + n/2 T(n) = 2*(2*T(n/4) + n/2) + n T(n) = 4*T(n/4) + 2n T(n) = 8*T(n/8) + 3n... T(n) = 2 k T(n/2 k ) + kn To get to a base case, let's set k = log 2 n. T(n) = 2 log n T(n/2 log n ) + (log n) n T(n) = n * T(n/n) + n log n T(n) = n * T(1) + n log n T(n) = n * 1 + n log n T(n) = n + n log n T(n) = O(n log n)

81. 81 Quick sort Quick sort orders a list of values by partitioning the list around one element called a pivot, then sorting each partition More specifically: choose one element in the list to be the pivot (= partition element) organize the elements so that all elements less than the pivot are to the left and all greater are to the right apply the quick sort algorithm (recursively) to both partitions

82. 82 Quick sort, continued For correctness, can choose any pivot. For efficiency, one of following is best case, the other worst case: pivot partitions the list roughly in half pivot is greatest or least element in list Which case above is best? We will divide the work into two methods: quickSort – performs the recursive algorithm partition – rearranges the elements into two partitions

83. 83 Quick sort pseudo-code Let S be the input set. 1. If |S| = 0 or |S| = 1, then return. 2. Pick an element v in S. Call v the pivot. 3. Partition S – {v} into two disjoint groups: S 1 = {x  S – {v} | x  v} S 2 = {x  S – {v} | x  v} 4. Return { quicksort(S 1 ), v, quicksort(S 2 ) }

84. 84 40 10 18 32 2 35 37 17 6 12 pick a pivot 6 10 12 2 17 18 40 37 32 35 partition quicksort 1810121762 403735 32 combine 181012176240373532 Quick sort illustrated

85. 85 How to choose a pivot first element bad if input is sorted or in reverse sorted order bad if input is nearly sorted variation: particular element (e.g. middle element) random element even a malicious agent cannot arrange a bad input median of three elements choose the median of the left, right, and center elements

86. 86 Partitioning algorithm Basic idea: 1. Move the pivot to the rightmost position. 2. Starting from the left, find an element  pivot. Call the position i. 3. Starting from the right, find an element  pivot. Call the position j. 4. Swap S[i] and S[j]. 8149035276 09

87. 87 Quick sort code private static void quickSort(int[] a, int min, int max) { if (min >= max) // base case; no need to sort return; // choose pivot (we'll use the middle element) int mid = (min + max) / 2; int pivot = a[mid]; swap(a, mid, max); // move pivot to end // partition the two sides of the array int i = partition(a, min, max - 1, pivot); // restore the pivot to its proper location swap(a, max, i); // recursively sort the left and right partitions quickSort(a, min, i - 1); quickSort(a, i + 1, max); }

88. 88 Quick sort code, cont'd. // partitions a with elements < pivot on left and // elements > pivot on right private static int partition(int[] a, int i, int j, int pivot) { i--; j++; // kludge because the loops pre-increment while (true) { // move index markers i,j toward center // until we find a pair of mis-partitioned elements do { i++; } while (i < j && a[i] < pivot); do { j--; } while (i pivot); if (i >= j) break; swap(a, i, j); } return i; }

89. 89 91731287211 07 11731287219 07 17312817219 07 17381217219 07 17389172112 07 pick pivot i i j j ji swap S[i] with S[right] "Median of three" pivot

90. 90 Special cases What happens when the array contains many duplicate elements? What happens when the array is already sorted (or nearly sorted) to begin with? Small arrays Quicksort is slower than insertion sort when is N is small (say, N  20). Optimization: Make |A|  20 the base case and call insertion sort.

91. 91 Quick sort runtime Worst case: pivot is the smallest (or largest) element all the time. T(n) = T(n-1) + cn T(n-1) = T(n-2) + c(n-1) T(n-2) = T(n-3) + c(n-2) … T(2) = T(1) + 2c Best case: pivot is the median T(n) = 2 T(n/2) + cn T(n) = cn log n + n = O(n log n)

92. 92 Quick sort runtime, cont'd. Assume each of the sizes for S 1 are equally likely. 0  |S 1 |  N-1.

93. 93 …  log e (N+1) – 3/2 divide equation by N(N+1) Quick sort runtime, cont'd.

94. 94 Quick sort runtime summary O(n log n) on average. O(n 2 ) worst case. comparisons mergeO(n log n) quick average: O(n log n) worst: O(n 2 )

95. 95 Selection problem, quickSelect Recall: the Selection problem Find the kth smallest element in an array a quickSelect(a, k): 1. If a.length = 1, then k=1 and return the element. 2. Pick a pivot v  a. 3. Partition a – {v} into a 1 (left side) and a 2 (right side). if k  a 1.length, then the kth smallest element must be in a 1. So return quickSelect(a 1, k). else if k = 1 + a 1.length, return the pivot v. Otherwise, the kth smallest element is in a 2. Return quickSelect(a 2, k - a 1.length - 1).

96. 96 Quick select runtime … = O(N) average case: worst case: same as Quick sort, O(N 2 )