Sorting Text Read Shaffer, Chapter 7 Sorting O(N 2 ) sorting algorithms: – Insertion, Selection, Bubble O(N log N) sorting algorithms – HeapSort, MergeSort, QuickSort

Assumptions Array of elements Contains only integers Array contained completely in memory

O(N 2 ) Sorting Algorithms Insertion Sort Selection Sort Bubble Sort

Insertion Sort Pseudo-code Algorithm public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : 0 | | a : 15 | | Insertion Sort Strategy: Start with p=1. In each pass of the outer loop, determine where the p th element should be inserted in the sorted subarray. Make room for it, if necessary, by sliding sorted elements down one. When appropriate slot is found, insert p th element. Increment p and repeat.

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p (insert p th element into sorted array)  i : a :

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : a : tmp=4

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  tmp < a[j-1]! a : tmp=4

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  Copy a[j-1] down! a : tmp=4

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  j==0, exit inner loop. a : tmp=4

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  Copy tmp. a : tmp=4

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : 0 1 | | a : 4 15 | |

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p (insert p th element into sorted array)  i : a :

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  tmp < a[j-1]! a : tmp=13

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  Copy a[j-1] down! a : tmp=13

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  tmp >= a[j-1], exit loop! a : tmp=13

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : j  Copy tmp! a : tmp=13

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p |  sorted | unsorted  i : | | a : | |

Insertion Sort: Step Through for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p p  i : Continue … a :

Insertion Sort: Analysis public static void insertionSort(Comparable a[]) { int j; for (int p=1; p<a.length; p++) { Comparable tmp = a[p]; for (j=p; j>0 && tmp.compareTo(a[j-1])<0; j--) a[j] = a[j-1]; a[j]=tmp; } // p } // insertionSort Count comparisons Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i ( from j=p downto j>0 ) p: … i … (n-1) max #comparisons: … i … (n-1)  total number of comparisons ≤ … + (n-1) = (n-1)n/2

Selection Sort Pseudo-code Algorithm public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort

Selection Sort: Step Through public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort | | unsorted  i : | a : | | Selection Sort Strategy: In each pass of the outer loop, select smallest value in unsorted subarray (i.e., from p th element on). Swap smallest element with p th element. Increment p and repeat.

Selection Sort: Analysis public static void selectionSort(Comparable a[]) { for (int p=0; p<a.length-1; p++) { Comparable min = a[p]; int minIndex = p; for (int j=p+1; j<a.length; j++) { if min.compareTo(a[j])>0 { minIndex = j; min = a[j]; } // new min found } // j swap(a,p,minIndex); } // p } // selectionSort Count comparisons. Assume a.length == n In general, for a given p the number of comparisons performed in the inner loop is ( from j=p+1 to j<a.length ) = (n-p-1) p: … i … (n-3)(n-2) max #comparisons: (n-1)(n-2)(n-3) … (n-i-1) … 2 1  total number of comparisons ≤ (n-1)+(n-2)+ … = (n-1)n/2

Bubble Sort Pseudo-code Algorithm public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort

Bubble Sort: Step Through public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort |  unsorted | i : | a : | | Bubble Sort Strategy: Outer loop starts with bottom of array (i.e. p=a.length-1). In each pass of outer loop, “ bubble ” largest element down by swapping adjacent elements (i.e., a[j] and a[j+1]) from the top whenever a[j] is larger. Decrement p and repeat.

Bubble Sort: Analysis public static void bubbleSort(Comparable a[]) { for (int p=a.length-1; p>0; p--) { for (int j=0; j<p; j++) if (a[j].compareTo(a[j+1])>0) swap(a,j,j+1); } // p } // bubbleSort Count comparisons. Assume a.length == n In general, for a given p==i the number of comparisons performed in the inner loop is i ( from j=0 to j<p ) p: (n-1) (n-2) (n-3) … i … 2 1 max #comparisons: (n-1) (n-2) (n-3) … i … 2 1  total number of comparisons ≤ (n-1)+(n-2) + … = (n-1)n/2

O(N log N) Sorting Algorithms HeapSort MergeSort QuickSort

HeapSort Strategy and Back-of-the-Envelope Analysis –Insert N elements into a Heap Each insert takes O(log N) time Inserting N elements takes O(N log N) time –Remove N elements from a Heap Each delete takes O(log N) time Removing N elements takes O(N log N) time

MergeSort Pseudo-code Algorithm // Merge two sorted arrays into a single array public static Comparable[] merge (Comparable a[], Comparable b[]) { int i=0; int j=0; int k=0; while (i<a.length && j<b.length) { if (a[i]<b[j]) { c[k] = a[i]; // merge a-value i++; } // a < b else c[k] = b[j]; // merge b-value j++; } // b <= a k++; } // while // continued next slide } // mergeSort

MergeSort Pseudo-code Algorithm if (i==a.length) // a-values exhausted, flush b while(j<b.length) { c[k] = b[j]; j++; k++; } // flush b-values else // b-values exhausted, flush a while(i<a.length) { c[k] = a[j]; i++; k++; } // flush a-values return c; // c contains merged values } // mergeSort

MergeSort: Step Through Start with two sorted sets of values a: b: c:

MergeSort: Step Through Merge a: b: _ c: 2

MergeSort: Step Through Merge a: _ b: _ c: 2 3

MergeSort: Step Through Merge a: _ b: _ _ 6 10 c: 2 3 5

MergeSort: Step Through Merge a: _ b: _ _ _ 10 c:

MergeSort: Step Through Merge a: _ _ b: _ _ _ 10 c:

MergeSort: Step Through Merge a: _ _ _ b: _ _ _ 10 c:

MergeSort: Step Through Merge a: _ _ _ b: _ _ _ _ c: Exit first loop

MergeSort: Step Through Merge a: _ _ _ _ b: _ _ _ _ c: Flush a-values

MergeSort: Step Through Merge a: _ _ _ _ _ 25 b: _ _ _ _ c: Flush a-values

MergeSort: Step Through Merge a: _ _ _ _ _ _ b: _ _ _ _ c: Flush a-values

MergeSort: Step Through Merge a: _ _ _ _ _ _ b: _ _ _ _ c: Return c-array

MergeSort: Text Example Start with array of elements a:

MergeSort: Text Example Merge 1-element lists  2-element list a:  b:

Merge 2-element lists  4-element list b:  a: Note that we move values from b to a in this pass. MergeSort: Text Example

Merge 4-element lists  8-element list a:  b: Note that we move values from a to b in this pass. MergeSort: Text Example

Merge 8-element lists  16-element list b:  a: Note that we move values from b to a in this pass. MergeSort: Text Example

QuickSort See Weiss, §7.7 Key: Partitioning, Figures 7.13 – 7.14 Example: i: … … a: … …  quickSort( a, 23, 31);

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 Assume CUTOFF=5

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 i=23, j=30 After call to median3

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | i  j  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 6 of Figure 7.14

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | i  j  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 8 of Figure 7.14

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | j  i  |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 Just before statement 10 of Figure 7.14

QuickSort: Partitioning | | i: … | |32 33 … a: … | |23 14 … | |  quickSort( a, 23, 31 ); left = 23 right = 31 pivot = 18 After statement 10 of Figure 7.14

QuickSort: Analysis N elements in original array  log N height Each level is created by partitioning  O(N) time per pass Total time to create tree = time to perform QuickSort == O(N log N) Assuming tree is balanced  assume good pivots are selected