Sorting Algorithms Bubble Sort Merge Sort Quick Sort Randomized Quick Sort

2 Overview Bubble Sort Divide and Conquer Merge Sort Quick Sort Randomization

3 Sorting Sorting takes an unordered collection and makes it an ordered one

4 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping

5 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 4277

6 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 3577

7 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 1277

8 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping No need to swap

9 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping Swap 5101

10 "Bubbling Up" the Largest Element Traverse a collection of elements Move from the front to the end “Bubble” the largest value to the end using pair-wise comparisons and swapping Largest value correctly placed

11 The “Bubble Up” Algorithm index <- 1 last_compare_at <- n – 1 loop exitif(index > last_compare_at) if(A[index] > A[index + 1]) then Swap(A[index], A[index + 1]) endif index <- index + 1 endloop

12 No, Swap isn’t built in. Procedure Swap(a, b isoftype in/out Num) t isoftype Num t <- a a <- b b <- t endprocedure // Swap

13 Items of Interest Notice that only the largest value is correctly placed All other values are still out of order So we need to repeat this process Largest value correctly placed

14 Repeat “Bubble Up” How Many Times? If we have N elements… And if each time we bubble an element, we place it in its correct location… Then we repeat the “bubble up” process N – 1 times. This guarantees we’ll correctly place all N elements.

15 “Bubbling” All the Elements N - 1

16 Reducing the Number of Comparisons

17 Reducing the Number of Comparisons On the N th “bubble up”, we only need to do MAX-N comparisons. For example: This is the 4 th “bubble up” MAX is 6 Thus we have 2 comparisons to do

18 N is … // Size of Array Arr_Type definesa Array[1..N] of Num Procedure Swap(n1, n2 isoftype Num) temp isoftype Num temp <- n1 n1 <- n2 n2 <- temp endprocedure // Swap Putting It All Together

19 The Truth NOBODY EVER uses Bubble Sort (except for Ezra) NOBODY NOT EVER Because it is EXTREMELY INEFFICIENT

20 This is how it goes…

21 Comparison (semi-log y axis)

22 Let’s forget about Bubble Sort

23 Divide and Conquer 1.Base Case, solve the problem directly if it is small enough 1.Divide the problem into two or more similar and smaller subproblems 1.Recursively solve the subproblems 1.Combine solutions to the subproblems

24 Divide and Conquer - Sort Problem: Input: A[left..right] – unsorted array of integers Output: A[left..right] – sorted in non-decreasing order

25 Divide and Conquer - Sort 1. Base case at most one element (left ≥ right), return 2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems: sort the FirstPart sort the SecondPart 3. Recursively sort FirstPart sort SecondPart 4. Combine sorted FirstPart and sorted SecondPart

26 This reminds me of… That’s easy. Mariah Carey. As a singing star, Ms. Carey has perfected the “wax-on” wave motion--a clockwise sweep of her hand used to emphasize lyrics. The Maria “Wax-on” Angle:  ( ,t) The Siren of Subquadratic Sorts

27 How To Remember Merge Sort? Just as Mariah recursively moves her hands into smaller circles, so too does merge sort recursively split an array into smaller segments.  ( ,t) We need two such recursions, one for each half of the split array.

28 Overview Divide and Conquer Merge Sort Quick Sort Randomization

29 Merge Sort: Idea Merge Recursively sort Divide into two halves FirstPart SecondPart FirstPart SecondPart A A is sorted!

30 Merge Sort: Algorithm Merge-Sort (A, left, right) if left ≥ right return else middle ← b (left+right)/2  Merge-Sort(A, left, middle) Merge-Sort(A, middle+1, right) Merge(A, left, middle, right) Recursive Call

31 A[middle] A[left] Sorted FirstPart Sorted SecondPart Merge-Sort: Merge A[right] merge A: A: Sorted

L:R: Temporary Arrays Merge-Sort: Merge Example A:

33 Merge-Sort: Merge Example L: A: R: i=0 j=0 k=

34 Merge-Sort: Merge Example L: A: R: k= i=0 j=1

35 Merge-Sort: Merge Example L: A: R: i=1 k= j=1

36 Merge-Sort: Merge Example L: A: R: i=2 j=1 k=

37 Merge-Sort: Merge Example L: A: R: j=2 k= i=2 5

38 Merge-Sort: Merge Example L: A: R: i=2 j=3 k=

39 Merge-Sort: Merge Example L: A: R: k= i=2 j=4

40 Merge-Sort: Merge Example L: A: R: i=3 j=4 k=7

41 Merge-Sort: Merge Example L: A: R: i=4 j=4 k=8

42 Merge(A, left, middle, right) 1. n 1 ← middle – left n 2 ← right – middle 3. create array L[n 1 ], R[n 2 ] 4. for i ← 0 to n 1 -1 do L[i] ← A[left +i] 5. for j ← 0 to n 2 -1 do R[j] ← A[middle+j] 6. k ← i ← j ← 0 7. while i < n 1 & j < n 2 8. if L[i] < R[j] 9. A[k++] ← L[i++] 10. else 11. A[k++] ← R[j++] 12. while i < n A[k++] ← L[i++] 14. while j < n A[k++] ← R[j++] n = n 1 +n 2 Space: n Time : cn for some constant c

Merge-Sort(A, 0, 7) Divide A:

Merge-Sort(A, 0, 3), divide A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 1), divide A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 0), base case A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 0), return A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 1, 1), base case A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 1, 1), return A: Merge-Sort(A, 0, 7)

Merge(A, 0, 0, 1) A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 1), return A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 2, 3) 48, divide A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 2, 2), base case A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 2, 2), return A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 3, 3), base case A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 3, 3), return A: Merge-Sort(A, 0, 7)

Merge(A, 2, 2, 3) A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 2, 3), return A: Merge-Sort(A, 0, 7)

Merge(A, 0, 1, 3) A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 0, 3), return A: Merge-Sort(A, 0, 7)

Merge-Sort(A, 4, 7) A: Merge-Sort(A, 0, 7)

A: Merge (A, 4, 5, 7) Merge-Sort(A, 0, 7)

Merge-Sort(A, 4, 7), return A: Merge-Sort(A, 0, 7)

Merge(A, 0, 3, 7) A: Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 7), done!

65 Merge-Sort Analysis cn 2 × cn/2 = cn 4 × cn/4 = cn n/2 × 2c = cn log n levels Total running time:  (nlogn) Total Space:  (n) Total: cn log n n n/2 n/

66 Merge-Sort Summary Approach: divide and conquer Time Most of the work is in the merging Total time:  (n log n) Space:  (n), more space than other sorts.

67 Overview Divide and Conquer Merge Sort Quick Sort

68 Quick Sort Divide: Pick any element p as the pivot, e.g, the first element Partition the remaining elements into FirstPart, which contains all elements < p SecondPart, which contains all elements ≥ p Recursively sort the FirstPart and SecondPart Combine: no work is necessary since sorting is done in place

69 Quick Sort x < p p p ≤ x Partition FirstPart SecondPart p pivot A: Recursive call x < p p p ≤ x Sorted FirstPart Sorted SecondPart Sorted

70 Quick Sort Quick-Sort(A, left, right) if left ≥ right return else middle ← Partition(A, left, right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right) end if

71 Partition p p x < pp ≤ x p x < p A: A: A: p

72 Partition ExampleA:

73 Partition ExampleA: i=0 j=1

74 Partition ExampleA: j= i=0 8

75 Partition ExampleA: i=0 j=2

76 Partition ExampleA: i=0 3 83j=3i=1

77 Partition ExampleA: i=1 5j=4

78 Partition ExampleA: i=1 1j=5

79 Partition ExampleA: i=2 16j=5

80 Partition ExampleA: i=2 16 7j=6

81 Partition ExampleA: i= i=3j=7

82 Partition ExampleA: i=3 15j=8

83 Partition ExampleA: i=

84 A: x < 4 4 ≤ x pivot in correct position Partition Example

85 Partition(A, left, right) 1. x ← A[left] 2. i ← left 3. for j ← left+1 to right 4. if A[j] < x then 5. i ← i swap(A[i], A[j]) 7. end if 8. end for j 9. swap(A[i], A[left]) 10. return i n = right – left +1 Time: cn for some constant c Space: constant

Quick-Sort(A, 0, 7) Partition A:

Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2) A:, partition

Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 0), base case, return

Quick-Sort(A, 0, 7) Quick-Sort(A, 1, 1), base case

Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 0, 2), return

Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 4, 7), partition

Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), partition

Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), partition

Quick-Sort(A, 0, 7) Quick-Sort(A, 7, 7) 8, return, base case 8

Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), return

Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), return 6 8 7

Quick-Sort(A, 0, 7) Quick-Sort(A, 4, 7), return

Quick-Sort(A, 0, 7), done!

99 Quick-Sort: Best Case Even Partition Total time:  (nlogn) cn 2 × cn/2 = cn 4 × c/4 = cn n/3 × 3c = cn log n levels n n/2 n/

100 cn c(n-1) 3c 2c n n-1 n c(n-2) Happens only if input is sortd input is reversely sorted Quick-Sort: Worst Case Unbalanced Partition Total time:  (n 2 )

101 Randomized Quick Sort

102 Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10 Quick-Sort: an Average Case cn ≤cn n 0.1n 0.9n 0.01n 0.09n Total time:  (nlogn) 0.81n 2 2 log 10 n log 10/9 n ≤cn

103 Quick-Sort Summary Time Most of the work done in partitioning. Average case takes  (n log(n)) time. Worst case takes  (n 2 ) time Space Sorts in-place, i.e., does not require additional space

104 Summary Divide and Conquer Merge-Sort Most of the work done in Merging  (n log(n)) time  (n) space Quick-Sort Most of the work done in partitioning Average case takes  (n log(n)) time Worst case takes  (n 2 ) time  (1) space

105 Quiz 6 (Show output) public static void main(String[ ] args) { int[ ] array = {4,18,16,3,5,21,7}; recursiveQuickSort(array, 0, array.length-1); System.out.println("The following array should be sorted: "); printList(array); System.exit(0); } public static void recursiveQuickSort(int[ ] list, int first, int last) { if(first < last) { int p = partition(list, first, last); printList(list); recursiveQuickSort(list, first, p-1); recursiveQuickSort(list, p+1, last); } Assume that printlist(list) prints the list separated by commas.