1. Sorting Text Read Shaffer, Chapter 7 Sorting O(N 2 ) sorting algorithms: – Insertion, Selection, Bubble O(N log N) sorting algorithms – HeapSort, MergeSort, QuickSort

2. Assumptions Array of elements Contains only integers Array contained completely in memory

3. O(N 2 ) Sorting Algorithms Insertion Sort Selection Sort Bubble Sort

4. Insertion Sort Pseudo-code Algorithm public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort

5. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : 0 | 1 2 3 4 5 | a : 15 | 4 13 2 21 10 | Insertion Sort Strategy: Start with p=1. In each pass of the outer loop, determine where the p th element should be inserted in the sorted subarray. Make room for it, if necessary, by sliding sorted elements down one. When appropriate slot is found, insert p th element. Increment p and repeat.

6. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p (insert p th element into sorted array)  i : 0 1 2 3 4 5 a : 15 4 13 2 21 10

7. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 a : 15 4 13 2 21 10 tmp=4

8. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  tmp < a[j-1]! a : 15 4 13 2 21 10 tmp=4

9. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  Copy a[j-1] down! a : 15 15 13 2 21 10 tmp=4

10. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  j==0, exit inner loop. a : 15 15 13 2 21 10 tmp=4

11. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  Copy tmp. a : 4 15 13 2 21 10 tmp=4

12. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : 0 1 | 2 3 4 5 | a : 4 15 |13 2 21 10 |

13. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p (insert p th element into sorted array)  i : 0 1 2 3 4 5 a : 4 15 13 2 21 10

14. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  tmp < a[j-1]! a : 4 15 13 2 21 10 tmp=13

15. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  Copy a[j-1] down! a : 4 15 15 2 21 10 tmp=13

16. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  tmp >= a[j-1], exit loop! a : 4 15 15 2 21 10 tmp=13

17. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 j  Copy tmp! a : 4 13 15 2 21 10 tmp=13

18. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : 0 1 2 | 3 4 5 | a : 4 13 15 | 2 21 10 |

19. Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : 0 1 2 3 4 5 Continue … a : 4 13 15 2 21 10

20. Insertion Sort: Analysis public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort Count comparisons Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i ( from j=p downto j>0 ) p: 1 2 3 4 … i … (n-1) max #comparisons: 1 2 3 4 … i … (n-1)  total number of comparisons ≤ 1 + 2 + 3 + … + (n-1) = (n-1)n/2

21. Selection Sort Pseudo-code Algorithm public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort

22. Selection Sort: Step Through public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort | | unsorted  i : | 0 1 2 3 4 5 a : | 15 4 13 2 21 10 | Selection Sort Strategy: In each pass of the outer loop, select smallest value in unsorted subarray (i.e., from p th element on). Swap smallest element with p th element. Increment p and repeat.

23. Selection Sort: Analysis public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Count comparisons. Assume a.length == n In general, for a given p the number of comparisons performed in the inner loop is ( from j=p+1 to j<a.length ) = (n-p-1) p: 0 1 2 … i … (n-3)(n-2) max #comparisons: (n-1)(n-2)(n-3) … (n-i-1) … 2 1  total number of comparisons ≤ (n-1)+(n-2)+ … + 2 + 1 = (n-1)n/2

24. Bubble Sort Pseudo-code Algorithm public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort

25. Bubble Sort: Step Through public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort |  unsorted | i : 0 1 2 3 4 5 | a : 15 4 13 2 21 10 | | Bubble Sort Strategy: Outer loop starts with bottom of array (i.e. p=a.length-1). In each pass of outer loop, “ bubble ” largest element down by swapping adjacent elements (i.e., a[j] and a[j+1]) from the top whenever a[j] is larger. Decrement p and repeat.

26. Bubble Sort: Analysis public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort Count comparisons. Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i ( from j=0 to j<p ) p: (n-1) (n-2) (n-3) … i … 2 1 max #comparisons: (n-1) (n-2) (n-3) … i … 2 1  total number of comparisons ≤ (n-1)+(n-2) + … + 2 + 1 = (n-1)n/2

27. O(N log N) Sorting Algorithms HeapSort MergeSort QuickSort

28. HeapSort Strategy and Back-of-the-Envelope Analysis –Insert N elements into a Heap Each insert takes O(log N) time Inserting N elements takes O(N log N) time –Remove N elements from a Heap Each delete takes O(log N) time Removing N elements takes O(N log N) time

29. MergeSort Pseudo-code Algorithm // Merge two sorted arrays into a single array public static Comparable[] merge (Comparable a[], Comparable b[]) { int i=0; int j=0; int k=0; while (i<a.length && j<b.length) { if (a[i]<b[j]) { c[k] = a[i]; // merge a-value i++; } // a < b else c[k] = b[j]; // merge b-value j++; } // b <= a k++; } // while // continued next slide } // mergeSort

30. MergeSort Pseudo-code Algorithm if (i==a.length) // a-values exhausted, flush b while(j<b.length) { c[k] = b[j]; j++; k++; } // flush b-values else // b-values exhausted, flush a while(i<a.length) { c[k] = a[j]; i++; k++; } // flush a-values return c; // c contains merged values } // mergeSort

31. MergeSort: Step Through Start with two sorted sets of values a: 3 7 8 19 24 25 b: 2 5 6 10 c:

32. MergeSort: Step Through Merge a: 3 7 8 19 24 25 b: _ 5 6 10 c: 2

33. MergeSort: Step Through Merge a: _ 7 8 19 24 25 b: _ 5 6 10 c: 2 3

34. MergeSort: Step Through Merge a: _ 7 8 19 24 25 b: _ _ 6 10 c: 2 3 5

35. MergeSort: Step Through Merge a: _ 7 8 19 24 25 b: _ _ _ 10 c: 2 3 5 6

36. MergeSort: Step Through Merge a: _ _ 8 19 24 25 b: _ _ _ 10 c: 2 3 5 6 7

37. MergeSort: Step Through Merge a: _ _ _ 19 24 25 b: _ _ _ 10 c: 2 3 5 6 7 8

38. MergeSort: Step Through Merge a: _ _ _ 19 24 25 b: _ _ _ _ c: 2 3 5 6 7 8 10 Exit first loop

39. MergeSort: Step Through Merge a: _ _ _ _ 24 25 b: _ _ _ _ c: 2 3 5 6 7 8 10 19 Flush a-values

40. MergeSort: Step Through Merge a: _ _ _ _ _ 25 b: _ _ _ _ c: 2 3 5 6 7 8 10 19 24 Flush a-values

41. MergeSort: Step Through Merge a: _ _ _ _ _ _ b: _ _ _ _ c: 2 3 5 6 7 8 10 19 24 25 Flush a-values

42. MergeSort: Step Through Merge a: _ _ _ _ _ _ b: _ _ _ _ c: 2 3 5 6 7 8 10 19 24 25 Return c-array

43. MergeSort: Text Example Start with array of elements a: 5 9 1 0 12 15 7 8 11 13 16 24 10 4 3 2

44. MergeSort: Text Example Merge 1-element lists  2-element list a: 5 9 1 0 12 15 7 8 11 13 16 24 10 4 3 2  b: 5 9 0 1 12 15 7 8 11 13 16 24 4 10 2 3

45. Merge 2-element lists  4-element list b: 5 9 0 1 12 15 7 8 11 13 16 24 4 10 2 3  a: 0 1 5 9 7 8 12 15 11 13 16 24 2 3 4 10 Note that we move values from b to a in this pass. MergeSort: Text Example

46. Merge 4-element lists  8-element list a: 0 1 5 9 7 8 12 15 11 13 16 24 2 3 4 10  b: 0 1 5 7 8 9 12 15 2 3 4 10 11 13 16 24 Note that we move values from a to b in this pass. MergeSort: Text Example

47. Merge 8-element lists  16-element list b: 0 1 5 7 8 9 12 15 2 3 4 10 11 13 16 24  a: 0 1 2 3 4 5 7 8 9 10 11 12 23 15 16 24 Note that we move values from b to a in this pass. MergeSort: Text Example

48. QuickSort See Weiss, §7.7 Key: Partitioning, Figures 7.13 – 7.14 Example: i: … 20 21 22 23 24 25 26 27 28 29 30 31 32 33 … a: … 19 24 36 9 7 16 20 31 26 17 19 18 23 14 …  quickSort( a, 23, 31);

49. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 20 31 26 17 19 18|23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 Assume CUTOFF=5

50. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 i=23, j=30 After call to median3

51. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 20 19 26 17 18 31|23 14 … | i  j  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 6 of Figure 7.14

52. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 … | i  j  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 8 of Figure 7.14

53. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 17 19 26 20 18 31|23 14 … | j  i  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 Just before statement 10 of Figure 7.14

54. QuickSort: Partitioning | | i: … 20 21 22|23 24 25 26 27 28 29 30 31|32 33 … a: … 19 24 36| 9 7 16 17 18 26 20 19 31|23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 10 of Figure 7.14

55. QuickSort: Analysis N elements in original array  log N height Each level is created by partitioning  O(N) time per pass Total time to create tree = time to perform QuickSort == O(N log N) Assuming tree is balanced  assume good pivots are selected