1. Sorting Algorithms Bubble Sort Merge Sort Quick Sort Randomized Quick Sort

2. 2 Overview Bubble Sort Divide and Conquer Merge Sort Quick Sort Randomization

3. 3 Sorting Sorting takes an unordered collection and makes it an ordered one. 5 12 3542 77 101 1 2 3 4 5 6 5 12 35 42 77101 1 2 3 4 5 6

4. 4 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 3542 77 101 1 2 3 4 5 6

5. 5 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 3542 77 101 1 2 3 4 5 6 Swap 4277

6. 6 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 3577 42 101 1 2 3 4 5 6 Swap 3577

7. 7 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 12 7735 42 101 1 2 3 4 5 6 Swap 1277

8. 8 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 No need to swap

9. 9 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 5 77 1235 42 101 1 2 3 4 5 6 Swap 5101

10. 10 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping 77 1235 42 5 1 2 3 4 5 6 101 Largest value correctly placed

11. 11 The “Bubble Up” Algorithm index <- 1 last_compare_at <- n – 1 loop exitif(index > last_compare_at) if(A[index] > A[index + 1]) then Swap(A[index], A[index + 1]) endif index <- index + 1 endloop

12. 12 No, Swap isn’t built in. Procedure Swap(a, b isoftype in/out Num) t isoftype Num t <- a a <- b b <- t endprocedure // Swap

13. 13 Items of Interest Notice that only the largest value is correctly placed All other values are still out of order So we need to repeat this process 77 1235 42 5 1 2 3 4 5 6 101 Largest value correctly placed

14. 14 Repeat “Bubble Up” How Many Times? If we have N elements… And if each time we bubble an element, we place it in its correct location… Then we repeat the “bubble up” process N – 1 times. This guarantees we’ll correctly place all N elements.

15. 15 “Bubbling” All the Elements 77 1235 42 5 1 2 3 4 5 6 101 5 4212 35 77 1 2 3 4 5 6 10142 5 35 12 77 1 2 3 4 5 6 10142 35 5 12 77 1 2 3 4 5 6 10142 35 12 5 77 1 2 3 4 5 6 101 N - 1

16. 16 Reducing the Number of Comparisons 12 35 42 77 101 1 2 3 4 5 6 577 1235 42 5 1 2 3 4 5 6 101 5 4212 35 77 1 2 3 4 5 6 10142 5 35 12 77 1 2 3 4 5 6 10142 35 5 12 77 1 2 3 4 5 6 101

17. 17 Reducing the Number of Comparisons On the N th “bubble up”, we only need to do MAX-N comparisons. For example: This is the 4 th “bubble up” MAX is 6 Thus we have 2 comparisons to do 42 5 35 12 77 1 2 3 4 5 6 101

18. 18 N is … // Size of Array Arr_Type definesa Array[1..N] of Num Procedure Swap(n1, n2 isoftype Num) temp isoftype Num temp <- n1 n1 <- n2 n2 <- temp endprocedure // Swap Putting It All Together

19. 19 The Truth NOBODY EVER uses Bubble Sort (except for Ezra) NOBODY NOT EVER Because it is EXTREMELY INEFFICIENT

20. 20 This is how it goes…

21. 21 Comparison (semi-log y axis)

22. 22 Let’s forget about Bubble Sort

23. 23 Divide and Conquer 1.Base Case, solve the problem directly if it is small enough 1.Divide the problem into two or more similar and smaller subproblems 1.Recursively solve the subproblems 1.Combine solutions to the subproblems

24. 24 Divide and Conquer - Sort Problem: Input: A[left..right] – unsorted array of integers Output: A[left..right] – sorted in non-decreasing order

25. 25 Divide and Conquer - Sort 1. Base case at most one element (left ≥ right), return 2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems: sort the FirstPart sort the SecondPart 3. Recursively sort FirstPart sort SecondPart 4. Combine sorted FirstPart and sorted SecondPart

26. 26 This reminds me of… That’s easy. Mariah Carey. As a singing star, Ms. Carey has perfected the “wax-on” wave motion--a clockwise sweep of her hand used to emphasize lyrics. The Maria “Wax-on” Angle:  ( ,t) The Siren of Subquadratic Sorts

27. 27 How To Remember Merge Sort? Just as Mariah recursively moves her hands into smaller circles, so too does merge sort recursively split an array into smaller segments.  ( ,t) We need two such recursions, one for each half of the split array.

28. 28 Overview Divide and Conquer Merge Sort Quick Sort Randomization

29. 29 Merge Sort: Idea Merge Recursively sort Divide into two halves FirstPart SecondPart FirstPart SecondPart A A is sorted!

30. 30 Merge Sort: Algorithm Merge-Sort (A, left, right) if left ≥ right return else middle ← b (left+right)/2  Merge-Sort(A, left, middle) Merge-Sort(A, middle+1, right) Merge(A, left, middle, right) Recursive Call

31. 31 A[middle] A[left] Sorted FirstPart Sorted SecondPart Merge-Sort: Merge A[right] merge A: A: Sorted

32. 32 6101422 351528 L:R: Temporary Arrays 515283061014 5 Merge-Sort: Merge Example 23781456A:

33. 33 Merge-Sort: Merge Example 3515283061014L: A: 3152830 6101422R: i=0 j=0 k=0 2378 1456 1

34. 34 Merge-Sort: Merge Example 1515283061014L: A: 351528 6101422R: k=1 2378 1456 2 i=0 j=1

35. 35 Merge-Sort: Merge Example 1215283061014L: A: 6101422R: i=1 k=2 2378 1456 3 j=1

36. 36 Merge-Sort: Merge Example 12361014L: A: 6101422R: i=2 j=1 k=3 2378 1456 4

37. 37 Merge-Sort: Merge Example 123461014L: A: 6101422R: j=2 k=4 2378 1456 i=2 5

38. 38 Merge-Sort: Merge Example 1234561014L: A: 6101422R: i=2 j=3 k=5 2378 1456 6

39. 39 Merge-Sort: Merge Example 12345614L: A: 6101422R: k=6 2378 1456 7 i=2 j=4

40. 40 Merge-Sort: Merge Example 123456714L: A: 351528 6101422R: 2378 1456 8 i=3 j=4 k=7

41. 41 Merge-Sort: Merge Example 12345678L: A: 351528 6101422R: 2378 1456 i=4 j=4 k=8

42. 42 Merge(A, left, middle, right) 1. n 1 ← middle – left + 1 2. n 2 ← right – middle 3. create array L[n 1 ], R[n 2 ] 4. for i ← 0 to n 1 -1 do L[i] ← A[left +i] 5. for j ← 0 to n 2 -1 do R[j] ← A[middle+j] 6. k ← i ← j ← 0 7. while i < n 1 & j < n 2 8. if L[i] < R[j] 9. A[k++] ← L[i++] 10. else 11. A[k++] ← R[j++] 12. while i < n 1 13. A[k++] ← L[i++] 14. while j < n 2 15. A[k++] ← R[j++] n = n 1 +n 2 Space: n Time : cn for some constant c

43. 43 6 2 8 4 3 7 5 1 6 2 8 4 3 7 5 1 Merge-Sort(A, 0, 7) Divide A:

44. 44 6 2 8 4 3 7 5 1 6 2 8484 Merge-Sort(A, 0, 3), divide A: Merge-Sort(A, 0, 7)

45. 45 3 7 5 1 8484 6 262 Merge-Sort(A, 0, 1), divide A: Merge-Sort(A, 0, 7)

46. 46 3 7 5 1 8484 62 Merge-Sort(A, 0, 0), base case A: Merge-Sort(A, 0, 7)

47. 47 3 7 5 1 8484 62 Merge-Sort(A, 0, 0), return A: Merge-Sort(A, 0, 7)

48. 48 3 7 5 1 8484 62 Merge-Sort(A, 1, 1), base case A: Merge-Sort(A, 0, 7)

49. 49 3 7 5 1 8484 62 Merge-Sort(A, 1, 1), return A: Merge-Sort(A, 0, 7)

50. 50 3 7 5 1 8484 2626 Merge(A, 0, 0, 1) A: Merge-Sort(A, 0, 7)

51. 51 3 7 5 1 8484 2626 Merge-Sort(A, 0, 1), return A: Merge-Sort(A, 0, 7)

52. 52 3 7 5 1 8484 2626 Merge-Sort(A, 2, 3) 48, divide A: Merge-Sort(A, 0, 7)

53. 53 3 7 5 1 4 2626 8 Merge-Sort(A, 2, 2), base case A: Merge-Sort(A, 0, 7)

54. 54 3 7 5 1 4 2626 8 Merge-Sort(A, 2, 2), return A: Merge-Sort(A, 0, 7)

55. 55 4 2626 8 Merge-Sort(A, 3, 3), base case A: Merge-Sort(A, 0, 7)

56. 56 3 7 5 1 4 2626 8 Merge-Sort(A, 3, 3), return A: Merge-Sort(A, 0, 7)

57. 57 3 7 5 1 2626 4 8 Merge(A, 2, 2, 3) A: Merge-Sort(A, 0, 7)

58. 58 3 7 5 1 2626 4 8 Merge-Sort(A, 2, 3), return A: Merge-Sort(A, 0, 7)

59. 59 3 7 5 1 2 4 6 8 Merge(A, 0, 1, 3) A: Merge-Sort(A, 0, 7)

60. 60 3 7 5 1 2 4 6 8 Merge-Sort(A, 0, 3), return A: Merge-Sort(A, 0, 7)

61. 61 3 7 5 1 2 4 6 8 Merge-Sort(A, 4, 7) A: Merge-Sort(A, 0, 7)

62. 62 1 3 5 7 2 4 6 8 A: Merge (A, 4, 5, 7) Merge-Sort(A, 0, 7)

63. 63 1 3 5 7 2 4 6 8 Merge-Sort(A, 4, 7), return A: Merge-Sort(A, 0, 7)

64. 64 1 2 3 4 5 6 7 8 Merge(A, 0, 3, 7) A: Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 7), done!

65. 65 Merge-Sort Analysis cn 2 × cn/2 = cn 4 × cn/4 = cn n/2 × 2c = cn log n levels Total running time:  (nlogn) Total Space:  (n) Total: cn log n n n/2 n/4 2 2 2

66. 66 Merge-Sort Summary Approach: divide and conquer Time Most of the work is in the merging Total time:  (n log n) Space:  (n), more space than other sorts.

67. 67 Overview Divide and Conquer Merge Sort Quick Sort

68. 68 Quick Sort Divide: Pick any element p as the pivot, e.g, the first element Partition the remaining elements into FirstPart, which contains all elements < p SecondPart, which contains all elements ≥ p Recursively sort the FirstPart and SecondPart Combine: no work is necessary since sorting is done in place

69. 69 Quick Sort x < p p p ≤ x Partition FirstPart SecondPart p pivot A: Recursive call x < p p p ≤ x Sorted FirstPart Sorted SecondPart Sorted

70. 70 Quick Sort Quick-Sort(A, left, right) if left ≥ right return else middle ← Partition(A, left, right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right) end if

71. 71 Partition p p x < pp ≤ x p x < p A: A: A: p

72. 72 Partition ExampleA: 48635172

73. 73 Partition ExampleA: 48635172i=0 j=1

74. 74 Partition ExampleA: j=1 48635172i=0 8

75. 75 Partition ExampleA: 486351726i=0 j=2

76. 76 Partition ExampleA: 4 8 635172i=0 3 83j=3i=1

77. 77 Partition ExampleA: 43685172 i=1 5j=4

78. 78 Partition ExampleA: 43685172 i=1 1j=5

79. 79 Partition ExampleA: 43685172 i=2 16j=5

80. 80 Partition ExampleA: 438572 i=2 16 7j=6

81. 81 Partition ExampleA: 438572 i=2 16 2 2 8 i=3j=7

82. 82 Partition ExampleA: 432678 i=3 15j=8

83. 83 Partition ExampleA: 41678 i=3 25 4 2 3

84. 84 A: 3 678 1 5 42 x < 4 4 ≤ x pivot in correct position Partition Example

85. 85 Partition(A, left, right) 1. x ← A[left] 2. i ← left 3. for j ← left+1 to right 4. if A[j] < x then 5. i ← i + 1 6. swap(A[i], A[j]) 7. end if 8. end for j 9. swap(A[i], A[left]) 10. return i n = right – left +1 Time: cn for some constant c Space: constant

86. 86 4 8 6 3 5 1 7 22 3 1 5 6 7 8 4 Quick-Sort(A, 0, 7) Partition A:

87. 87 2 3 1 5 6 7 8 4 2 13 Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2) A:, partition

88. 88 2 5 6 7 8 4113 Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 0), base case, return

89. 89 2 5 6 7 8 41 3 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 1, 1), base case

90. 90 5 6 7 8 4 2 1 3 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 0, 2), return

91. 91 4 2 1 3 5 6 7 8 6 7 8 5 Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 4, 7), partition

92. 92 45 6 7 8 7878 66 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), partition

93. 93 456 7878 8 7 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), partition

94. 94 4567 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 7, 7) 8, return, base case 8

95. 95 456 8 7 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), return

96. 96 45 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), return 6 8 7

97. 97 4 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 4, 7), return 5 6 8 7

98. 98 4 2 1 3 Quick-Sort(A, 0, 7), done! 5 6 8 7

99. 99 Quick-Sort: Best Case Even Partition Total time:  (nlogn) cn 2 × cn/2 = cn 4 × c/4 = cn n/3 × 3c = cn log n levels n n/2 n/4 3 3 3

100. 100 cn c(n-1) 3c 2c n n-1 n-2 3 2 c(n-2) Happens only if input is sortd input is reversely sorted Quick-Sort: Worst Case Unbalanced Partition Total time:  (n 2 )

101. 101 Randomized Quick Sort

102. 102 Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10 Quick-Sort: an Average Case cn ≤cn n 0.1n 0.9n 0.01n 0.09n Total time:  (nlogn) 0.81n 2 2 log 10 n log 10/9 n ≤cn

103. 103 Quick-Sort Summary Time Most of the work done in partitioning. Average case takes  (n log(n)) time. Worst case takes  (n 2 ) time Space Sorts in-place, i.e., does not require additional space

104. 104 Summary Divide and Conquer Merge-Sort Most of the work done in Merging  (n log(n)) time  (n) space Quick-Sort Most of the work done in partitioning Average case takes  (n log(n)) time Worst case takes  (n 2 ) time  (1) space

105. 105 Quiz 6 (Show output) public static void main(String[ ] args) { int[ ] array = {4,18,16,3,5,21,7}; recursiveQuickSort(array, 0, array.length-1); System.out.println("The following array should be sorted: "); printList(array); System.exit(0); } public static void recursiveQuickSort(int[ ] list, int first, int last) { if(first < last) { int p = partition(list, first, last); printList(list); recursiveQuickSort(list, first, p-1); recursiveQuickSort(list, p+1, last); } Assume that printlist(list) prints the list separated by commas.