1. Sorting Algorithms Merge Sort Quick Sort Hairong Zhao http://web.njit.edu/~hz2 New Jersey Institute of Technology

2. 2 Overview Divide and Conquer Merge Sort Quick Sort

3. 3 Divide and Conquer 1.Base Case, solve the problem directly if it is small enough 2.Divide the problem into two or more similar and smaller subproblems 3.Recursively solve the subproblems 4.Combine solutions to the subproblems

4. 4 Divide and Conquer - Sort Problem: Input: A[left..right] – unsorted array of integers Output: A[left..right] – sorted in non-decreasing order

5. 5 Divide and Conquer - Sort 1. Base case at most one element (left ≥ right), return 2. Divide A into two subarrays: FirstPart, SecondPart Two Subproblems: sort the FirstPart sort the SecondPart 3. Recursively sort FirstPart sort SecondPart 4. Combine sorted FirstPart and sorted SecondPart

6. 6 Overview Divide and Conquer Merge Sort Quick Sort

7. 7 Merge Sort: Idea Merge Recursively sort Divide into two halves FirstPart SecondPart FirstPart SecondPart A A is sorted!

8. 8 Merge Sort: Algorithm Merge-Sort (A, left, right) if left ≥ right return else middle ← b (left+right)/2  Merge-Sort(A, left, middle) Merge-Sort(A, middle+1, right) Merge(A, left, middle, right) Recursive Call

9. 9 A[middle] A[left] Sorted FirstPart Sorted SecondPart Merge-Sort: Merge A[right] merge A: A: Sorted

10. 10 6 1422 351528 L:R: Temporary Arrays 515283061014 5 Merge-Sort: Merge Example 23781456 A:

11. 11 Merge-Sort: Merge Example 3515283061014 L: A: 3152830 6101422 R: i=0 j=0 k=0 2378 1456 1

12. 12 Merge-Sort: Merge Example 1515283061014 L: A: 351528 6101422 R: k=1 2378 1456 2 i=0 j=1

13. 13 Merge-Sort: Merge Example 1215283061014 L: A: 6101422 R: i=1 k=2 2378 1456 3 j=1

14. 14 Merge-Sort: Merge Example 12361014 L: A: 6101422 R: i=2 j=1 k=3 2378 1456 4

15. 15 Merge-Sort: Merge Example 123461014 L: A: 6101422 R: j=2 k=4 2378 1456 i=2 5

16. 16 Merge-Sort: Merge Example 1234561014 L: A: 6101422 R: i=2 j=3 k=5 2378 1456 6

17. 17 Merge-Sort: Merge Example 12345614 L: A: 6101422 R: k=6 2378 1456 7 i=2 j=4

18. 18 Merge-Sort: Merge Example 123456714 L: A: 351528 6101422 R: 2378 1456 8 i=3 j=4 k=7

19. 19 Merge-Sort: Merge Example 12345678 L: A: 351528 6101422 R: 2378 1456 i=4 j=4 k=8

20. 20 Merge(A, left, middle, right) 1. n 1 ← middle – left + 1 2. n 2 ← right – middle 3. create array L[n 1 ], R[n 2 ] 4. for i ← 0 to n 1 -1 do L[i] ← A[left +i] 5. for j ← 0 to n 2 -1 do R[j] ← A[middle+j] 6. k ← i ← j ← 0 7. while i < n 1 & j < n 2 8. if L[i] < R[j] 9. A[k++] ← L[i++] 10. else 11. A[k++] ← R[j++] 12. while i < n 1 13. A[k++] ← L[i++] 14. while j < n 2 15. A[k++] ← R[j++] n = n 1 +n 2 Space: n Time : cn for some constant c

21. 21 6 2 8 4 3 7 5 1 6 2 8 4 3 7 5 1 Merge-Sort(A, 0, 7) Divide A:

22. 22 6 2 8 4 3 7 5 1 6 2 8 4 Merge-Sort(A, 0, 3), divide A: Merge-Sort(A, 0, 7)

23. 23 3 7 5 1 8 4 6 2 6 2 Merge-Sort(A, 0, 1), divide A: Merge-Sort(A, 0, 7)

24. 24 3 7 5 1 8 4 6 2 Merge-Sort(A, 0, 0), base case A: Merge-Sort(A, 0, 7)

25. 25 3 7 5 1 8 4 6 2 Merge-Sort(A, 0, 0), return A: Merge-Sort(A, 0, 7)

26. 26 3 7 5 1 8 4 6 2 Merge-Sort(A, 1, 1), base case A: Merge-Sort(A, 0, 7)

27. 27 3 7 5 1 8 4 6 2 Merge-Sort(A, 1, 1), return A: Merge-Sort(A, 0, 7)

28. 28 3 7 5 1 8 4 2 6 Merge(A, 0, 0, 1) A: Merge-Sort(A, 0, 7)

29. 29 3 7 5 1 8 4 2 6 Merge-Sort(A, 0, 1), return A: Merge-Sort(A, 0, 7)

30. 30 3 7 5 1 8 4 2 6 Merge-Sort(A, 2, 3) 4 8, divide A: Merge-Sort(A, 0, 7)

31. 31 3 7 5 1 4 2 6 8 Merge-Sort(A, 2, 2), base case A: Merge-Sort(A, 0, 7)

32. 32 3 7 5 1 4 2 6 8 Merge-Sort(A, 2, 2), return A: Merge-Sort(A, 0, 7)

33. 33 4 2 6 8 Merge-Sort(A, 3, 3), base case A: Merge-Sort(A, 0, 7)

34. 34 3 7 5 1 4 2 6 8 Merge-Sort(A, 3, 3), return A: Merge-Sort(A, 0, 7)

35. 35 3 7 5 1 2 6 4 8 Merge(A, 2, 2, 3) A: Merge-Sort(A, 0, 7)

36. 36 3 7 5 1 2 6 4 8 Merge-Sort(A, 2, 3), return A: Merge-Sort(A, 0, 7)

37. 37 3 7 5 1 2 4 6 8 Merge(A, 0, 1, 3) A: Merge-Sort(A, 0, 7)

38. 38 3 7 5 1 2 4 6 8 Merge-Sort(A, 0, 3), return A: Merge-Sort(A, 0, 7)

39. 39 3 7 5 1 2 4 6 8 Merge-Sort(A, 4, 7) A: Merge-Sort(A, 0, 7)

40. 40 1 3 5 7 2 4 6 8 A: Merge (A, 4, 5, 7) Merge-Sort(A, 0, 7)

41. 41 1 3 5 7 2 4 6 8 Merge-Sort(A, 4, 7), return A: Merge-Sort(A, 0, 7)

42. 42 1 2 3 4 5 6 7 8 Merge(A, 0, 3, 7) A: Merge-Sort(A, 0, 7) Merge-Sort(A, 0, 7), done!

43. 43 Merge-Sort Analysis cn 2 × cn/2 = cn 4 × cn/4 = cn n/2 × 2c = cn log n levels Total running time:  (nlogn) Total Space:  (n) Total: cn log n n n/2 n/4 2 2 2

44. 44 Merge-Sort Summary Approach: divide and conquer Time Most of the work is in the merging Total time:  (n log n) Space:  (n), more space than other sorts.

45. 45 Overview Divide and Conquer Merge Sort Quick Sort

46. 46 Quick Sort Divide: Pick any element p as the pivot, e.g, the first element Partition the remaining elements into FirstPart, which contains all elements < p SecondPart, which contains all elements ≥ p Recursively sort the FirstPart and SecondPart Combine: no work is necessary since sorting is done in place

47. 47 Quick Sort x < p p p ≤ x Partition FirstPart SecondPart p pivot A: Recursive call x < p p p ≤ x Sorted FirstPart Sorted SecondPart Sorted

48. 48 Quick Sort Quick-Sort(A, left, right) if left ≥ right return else middle ← Partition(A, left, right) Quick-Sort(A, left, middle–1 ) Quick-Sort(A, middle+1, right) end if

49. 49 Partition p p x < pp ≤ x p x < p A: A: A: p

50. 50 Partition ExampleA: 48635172

51. 51 Partition ExampleA: 48635172 i=0 j=1

52. 52 Partition ExampleA: j=1 48635172 i=0 8

53. 53 Partition ExampleA: 486351726 i=0 j=2

54. 54 Partition ExampleA: 4 8 635172 i=0 3 83 j=3i=1

55. 55 Partition ExampleA: 43685172i=1 5 j=4

56. 56 Partition ExampleA: 43685172i=1 1 j=5

57. 57 Partition ExampleA: 43685172i=2 16 j=5

58. 58 Partition ExampleA: 438572i=2 16 7 j=6

59. 59 Partition ExampleA: 438572i=2 16 2 2 8i=3j=7

60. 60 Partition ExampleA: 432678i=3 15 j=8

61. 61 Partition ExampleA: 41678i=3 25 4 2 3

62. 62 A: 3 678 1 5 42 x < 4 4 ≤ x pivot in correct position Partition Example

63. 63 Partition(A, left, right) 1. x ← A[left] 2. i ← left 3. for j ← left+1 to right 4. if A[j] < x then 5. i ← i + 1 6. swap(A[i], A[j]) 7. end if 8. end for j 9. swap(A[i], A[left]) 10. return i n = right – left +1 Time: cn for some constant c Space: constant

64. 64 4 8 6 3 5 1 7 2 2 3 1 5 6 7 8 4 Quick-Sort(A, 0, 7) Partition A:

65. 65 2 3 1 5 6 7 8 4 2 13 Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 2) A:, partition

66. 66 2 5 6 7 8 4 1 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 0, 0), base case, return

67. 67 2 5 6 7 8 4 1 3 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 1, 1), base case

68. 68 5 6 7 8 4 2 1 3 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 0, 2), return

69. 69 4 2 1 3 5 6 7 86 7 8 5 Quick-Sort(A, 0, 7) Quick-Sort(A, 2, 2), return Quick-Sort(A, 4, 7), partition

70. 70 4 5 6 7 8 7 8 6 6 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), partition

71. 71 4 5 6 7 8 8 7 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), partition

72. 72 4 5 6 7 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 7, 7) 8, return, base case 8

73. 73 4 5 6 8 7 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 6, 7), return

74. 74 4 5 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 5, 7), return 6 8 7

75. 75 4 2 1 3 Quick-Sort(A, 0, 7) Quick-Sort(A, 4, 7), return 5 6 8 7

76. 76 4 2 1 3 Quick-Sort(A, 0, 7), done! 5 6 8 7

77. 77 Quick-Sort: Best Case Even Partition Total time:  (nlogn) cn 2 × cn/2 = cn 4 × c/4 = cn n/3 × 3c = cn log n levels n n/2 n/4 3 3 3

78. 78 cn c(n-1) 3c 2c n n-1 n-2 3 2 c(n-2) Happens only if input is sortd input is reversely sorted Quick-Sort: Worst Case Unbalanced Partition Total time:  (n 2 )

79. 79 Quick-Sort: an Average Case Suppose the split is 1/10 : 9/10 Quick-Sort: an Average Case cn ≤cn n 0.1n 0.9n 0.01n 0.09n Total time:  (nlogn) 0.81n 2 2 log 10 n log 10/9 n ≤cn

80. 80 Quick-Sort Summary Time Most of the work done in partitioning. Average case takes  (n log(n)) time. Worst case takes  (n 2 ) time Space Sorts in-place, i.e., does not require additional space

81. 81 Summary Divide and Conquer Merge-Sort Most of the work done in Merging  (n log(n)) time  (n) space Quick-Sort Most of the work done in partitioning Average case takes  (n log(n)) time Worst case takes  (n 2 ) time  (1) space

82. 82 Homework 1. What is the running time of Merge-Sort if the array is already sorted? What is the best case running time of Merge-Sort? 2. Demonstrate the working of Partition on sequence (13, 5, 14, 11, 16, 12, 1, 15). What is the value of i returned at the completion of Partition?