2. Chapter 7: Sorting Insertion Sort Shell Sort CS 340 Page 112 Heap Sort Merge Sort Quick Sort Best Case Sorting Algorithm Analysis

3. CS 340 Page 113 Sorting While sorting an array of elements is itself a significant problem, examining this problem serves several other purposes: A large variety of sorting algorithms have been developed, so we can explore many algorithm design techniques. Formal analysis of these algorithms is rather straightforward, so our intuitive feel for time complexity can be expanded. Worst-case, best-case, and average-case analyses are possible with most of these algorithms, showing us that algorithms that might be advantageous under certain conditions can be quite disadvantageous under other conditions. Memory limitations can affect the ability to apply certain sorting algorithms, so we can also examine the effects of external system limitations upon algorithm design and implementation.

4. CS 340 Page 114 Insertion Sort // Note: The list itself will be the n elements // in A[1] through A[n]. A sentinel value // of INT_MIN will be placed in A[0]. template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // Defined elsewhere as smallest // possible value of type Etype. for (p = 1; p < n; p++) { itemToInsert = A[p]; for (j = p; itemToInsert < A[j-1]; j--) A[j] = A[j-1]; A[j] = itemToInsert; } // Note: The list itself will be the n elements // in A[1] through A[n]. A sentinel value // of INT_MIN will be placed in A[0]. template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // Defined elsewhere as smallest // possible value of type Etype. for (p = 1; p < n; p++) { itemToInsert = A[p]; for (j = p; itemToInsert < A[j-1]; j--) A[j] = A[j-1]; A[j] = itemToInsert; } Sort the first k elements and then insert the next element by sequentially finding its properly location, sliding each element greater than the next element over by one slot.

5. CS 340 Page 115 Insertion Sort Example Catwoman Joker MrFreeze Penguin Riddler Scarecrow Two-Face Clayface Egghead PoisonIvy Joker Penguin MrFreeze Scarecrow Riddler Catwoman Two-Face Clayface Egghead PoisonIvy Joker MrFreeze Penguin Scarecrow Riddler Catwoman Two-Face Clayface Egghead PoisonIvy Joker MrFreeze Penguin Riddler Scarecrow Catwoman Two-Face Clayface Egghead PoisonIvy Catwoman Joker MrFreeze Penguin Riddler Scarecrow Two-Face Clayface Egghead PoisonIvy Catwoman Clayface Joker MrFreeze Penguin Riddler Scarecrow Two-Face Egghead PoisonIvy Catwoman Clayface Egghead Joker MrFreeze Penguin Riddler Scarecrow Two-Face PoisonIvy Catwoman Clayface Egghead Joker MrFreeze Penguin PoisonIvy Riddler Scarecrow Two-Face

6. CS 340 Page 116 Insertion Sort Analysis template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment }

7. CS 340 Page 117 Insertion Sort Analysis (continued) Best case time complexity (i.e., inner loop iterates minimally): 3+  p=1,n- 1 (2+4+6)+3 = 6+12(n-1) = 12n-6 time units, which is O(n). Since there actually is an array requiring this time (i.e., an array with its elements initially in order), the best case is also  (n). Average case time complexity: The average number of inversions in a list is ¼n(n-1) (an inversion is a pair of array slots i and j where I A[j]), and this algorithm only removes one inversion per inner loop iteration. Thus, the average case is  (n 2 ), and, by the worst-case argument above, it is also O(n 2 ). template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment } template void insertionSort(Etype A[], int n) { int j, p; Etype itemToInsert; A[0] = MIN_VAL; // 3 TU: 1 multiplication, 1 addition, and 1 // memory access (i.e., assigning to array slot) for (p = 1; p < n; p++) // n-1 iterations, 2 TU each (1 assignment { // or increment and 1 comparison) itemToInsert = A[p]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, and 1 assignment for (j = p; itemToInsert < A[j-1]; j--) // At most p iterations, 6 TU each // (1 assignment or decrement, // 1 subtraction, 1 multiplication, // 1 addition, 1 memory access, and // 1 comparison) A[j] = A[j-1]; // 7 TU: 1 subtraction, 2 multiplications, 2 additions, // 1 memory access, and 1 assignment A[j] = itemToInsert; // 3 TU: 1 multiplication, 1 addition, 1 assignment }

8. CS 340 Page 118 Shell Sort Sort the list in layered sublists, so when sorting the list as a whole, it’s already reasonably close to being sorted. // Note: The list itself will be the n // elements in A[1] through A[n]. // No meaningful value will be // placed in A[0]. void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) for (i = inc + 1; i <= n; i++) { temp = A[i]; for (j = i; j > inc; j -= inc) { if (temp < A[j - inc]) A[j] = A[j - inc]; else break; } A[j] = temp; } // Note: The list itself will be the n // elements in A[1] through A[n]. // No meaningful value will be // placed in A[0]. void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) for (i = inc + 1; i <= n; i++) { temp = A[i]; for (j = i; j > inc; j -= inc) { if (temp < A[j - inc]) A[j] = A[j - inc]; else break; } A[j] = temp; }

9. CS 340 Page 119 Shell Sort Example Increment: 6 Increment: 3 Increment: 1 Note that for each increment value, the algorithm performs an insertion sort on every subarray of values spaced increment apart. Also note that this implementation uses increment values n/2, n/4, n/8, …, 1, known as Shell’s increments. A more strategic choice of increment values (i.e., all values relatively prime), would yield better overall performance.LeiaGreedoLandoAckbarR2D2C3PODarthYodaLukeJabbaHanObiwanChewie LeiaDarthChewieGreedoYodaLandoLukeAckbarJabbaR2D2HanC3POObiwan ChewieDarthLeiaGreedoYoda LandoLukeAckbarJabbaHanR2D2 C3POObiwan ChewieAckbarDarthJabbaLeia GreedoHanYodaR2D2 LandoC3POLukeObiwan AckbarChewieDarthJabbaLeia GreedoHanR2D2Yoda C3POLandoLukeObiwan AckbarGreedoC3POChewieHanLandoDarthR2D2LukeJabbaYodaObiwanLeia AckbarC3POChewieDarthGreedoHanJabbaLandoLeiaLukeObiwanR2D2Yoda

10. CS 340 Page 120 void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 iterations; 3 TU each // (1 division, 1 assignment, // and 1 comparison) for (i = inc + 1; i <= n; i++) // (n-inc) iterations; 3 TU each // (1 addition, 1 assignment, // and 1 comparison) { temp = A[i]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, 1 assignment for (j = i; j > inc; j -= inc) // (i/inc) iterations; 3 TU each: 1 // subtraction, 1 assignment, and 1 // comparison (with first iteration // having one less subtraction) { if (temp < A[j - inc]) // At most 12 TU: 2 subtractions, 3 A[j] = A[j - inc]; // multiplications, 3 additions, 3 else // memory accesses (including the break; // assignment), and 1 comparison } A[j] = temp; // 3 TU: 1 multiplication, 1 addition, } // and 1 assignment } void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 iterations; 3 TU each // (1 division, 1 assignment, // and 1 comparison) for (i = inc + 1; i <= n; i++) // (n-inc) iterations; 3 TU each // (1 addition, 1 assignment, // and 1 comparison) { temp = A[i]; // 4 TU: 1 multiplication, 1 addition, // 1 memory access, 1 assignment for (j = i; j > inc; j -= inc) // (i/inc) iterations; 3 TU each: 1 // subtraction, 1 assignment, and 1 // comparison (with first iteration // having one less subtraction) { if (temp < A[j - inc]) // At most 12 TU: 2 subtractions, 3 A[j] = A[j - inc]; // multiplications, 3 additions, 3 else // memory accesses (including the break; // assignment), and 1 comparison } A[j] = temp; // 3 TU: 1 multiplication, 1 addition, } // and 1 assignment } Shell Sort Analysis

11. CS 340 Page 121 void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 it.@3TU for (i = inc + 1; i <= n; i++) // (n-inc) it.@3TU { temp = A[i]; // 4TU for (j = i; j > inc; j -= inc) // (i/inc) it.@3TU { if (temp < A[j - inc]) // At A[j] = A[j - inc]; // most else // 12 break; // TU } A[j] = temp; // 3TU } void shellSort(Etype A[], int n) { Etype temp; int inc, i, j; for (inc = n/2; inc > 0; inc /= 2) // log(n)-1 it.@3TU for (i = inc + 1; i <= n; i++) // (n-inc) it.@3TU { temp = A[i]; // 4TU for (j = i; j > inc; j -= inc) // (i/inc) it.@3TU { if (temp < A[j - inc]) // At A[j] = A[j - inc]; // most else // 12 break; // TU } A[j] = temp; // 3TU } Shell Sort Analysis (continued) Best case time complexity (i.e., inner loop iterates once):  p=1,log(n)-1 (3+  i=p+1,n( 3+4+3+5+3)) =  p=1,logn (3+18(n-p)) = 3logn+18nlogn- 9logn(1+logn) time units, which is O(nlogn). Since there actually is an array requiring this time (i.e., an array with its elements initially in order), the best case is also  (nlogn). Note: Some other (relatively prime) Shell sort increments have  (n 1.5 ) time complexity. This is O(n 2 ). There actually is an array requiring this time (i.e., an array with the largest n/2 elements in the even positions and the smallest n/2 elements in the odd positions), so the worst case is also  (n 2 ).

12. CS 340 Page 122 Heap Sort Insert the list into a maximum heap, so removals will involve quick access to the next largest element in the list. template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; for ( ; 2*i <= n; i = childIndex) { childIndex = ((2*i != n) && (A[2*i+1] > A[2*i])) ? (2*i+1) : (2*i); if (temp < A[childIndex]) A[i] = A[childIndex]; else break; } A[i] = temp; } template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j--)// Set list up as percDown(A, j, n);// a max-heap. for (int i = n; i >= 2; i--) { temp = A[1]; A[1] = A[i]; A[i] = temp;// (i.e., deleteMax) percDown(A, 1, i-1); } template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; for ( ; 2*i <= n; i = childIndex) { childIndex = ((2*i != n) && (A[2*i+1] > A[2*i])) ? (2*i+1) : (2*i); if (temp < A[childIndex]) A[i] = A[childIndex]; else break; } A[i] = temp; } template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j--)// Set list up as percDown(A, j, n);// a max-heap. for (int i = n; i >= 2; i--) { temp = A[1]; A[1] = A[i]; A[i] = temp;// (i.e., deleteMax) percDown(A, 1, i-1); }

13. CS 340 Page 123 Heap Sort Example Original List: Homer Marge Maggie Bart Lisa Wiggum Burns Moe Barney Flanders Krusty Milhouse Lovejoy Skinner Original List: Homer Marge Maggie Bart Lisa Wiggum Burns Moe Barney Flanders Krusty Milhouse Lovejoy Skinner After Converting Into Max-Heap: Rest Of Heap Sort: Skinner Moe Milhouse Lisa Krusty Marge Burns Bart Barney Flanders Homer Maggie Lovejoy Wiggum Wiggum Moe Skinner Lisa Krusty Marge Milhouse Bart Barney Flanders Homer Maggie Lovejoy Burns Moe Lovejoy Milhouse Lisa Krusty Marge Burns Bart Barney Flanders Homer Maggie Skinner Wiggum Milhouse Lovejoy Marge Lisa Krusty Maggie Burns Bart Barney Flanders Homer Moe Skinner Wiggum Marge Lovejoy Maggie Lisa Krusty Homer Burns Bart Barney Flanders Milhouse Moe Skinner Wiggum Maggie Lovejoy Homer Lisa Krusty Flanders Burns Bart Barney Marge Milhouse Moe Skinner Wiggum Lovejoy Lisa Homer Bart Krusty Flanders Burns Barney Maggie Marge Milhouse Moe Skinner Wiggum Lisa Krusty Homer Bart Barney Flanders Burns Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Krusty Burns Homer Bart Barney Flanders Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Homer Burns Flanders Bart Barney Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Flanders Burns Barney Bart Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Burns Bart Barney Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Bart Barney Burns Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum Barney Bart Burns Flanders Homer Krusty Lisa Lovejoy Maggie Marge Milhouse Moe Skinner Wiggum

14. CS 340 Page 124 template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; // 4 TU for ( ; 2*i <= n; i = childIndex) // At most log(n-i) iterations; 3 TU each { // except first iteration (1 less assignment) childIndex = ((2*i != n) && (A[2*i+1] > A[2*i])) ? (2*i+1) : (2*i); // At most 16 TU if (temp < A[childIndex]) // At most 11 TU A[i] = A[childIndex]; else break; } A[i] = temp; // 4 TU } // TOTAL FOR PERCDOWN: At most 30log(n-i)+7 TU template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j—-) // n/2 iterations; 3 TU each (except first) percDown(A, j, n); // At most 30log(n-j)+7 TU (by above analysis) for (int i = n; i >= 2; i--) // (n-i iterations; 2 TU each) { temp = A[1]; A[1] = A[i]; A[i] = temp; // 13 TU percDown(A, 1, i-1); // 30log(i-2)+8 TU } template void percDown(Etype A[], int i, int n) { int childIndex; Etype temp = A[i]; // 4 TU for ( ; 2*i <= n; i = childIndex) // At most log(n-i) iterations; 3 TU each { // except first iteration (1 less assignment) childIndex = ((2*i != n) && (A[2*i+1] > A[2*i])) ? (2*i+1) : (2*i); // At most 16 TU if (temp < A[childIndex]) // At most 11 TU A[i] = A[childIndex]; else break; } A[i] = temp; // 4 TU } // TOTAL FOR PERCDOWN: At most 30log(n-i)+7 TU template void heapSort(Etype A[], int n) { Etype temp; for (int j = n/2; j > 0; j—-) // n/2 iterations; 3 TU each (except first) percDown(A, j, n); // At most 30log(n-j)+7 TU (by above analysis) for (int i = n; i >= 2; i--) // (n-i iterations; 2 TU each) { temp = A[1]; A[1] = A[i]; A[i] = temp; // 13 TU percDown(A, 1, i-1); // 30log(i-2)+8 TU } Worst case time complexity:  j=1,n/2 (3+30log(n-j)+7)-1+  i=2,n (2+13+30log(i-2)+8) ≤ (5n + 15nlogn – 1) + (23n – 23 + 30nlogn) = 45nlogn + 28n - 24 time units, which is O(nlogn). Since either changing an array into a max-heap or “re-heapifying” it as maximal elements are removed requires this time, the worst case is also  (nlogn). Heap Sort Analysis

15. CS 340 Page 125 Merge Sort Recursively sort both halves of the list, and then quickly merge the two sorted halves to form the entire sorted list. template void mergeSort(Etype A[], const int n) { Etype Acopy[n+1]; int size; for (int k = 1; k <= n; k++) Acopy[k] = A[k]; order(Acopy, A, 1, size); } template void order(Etype source[], Etype dest[], int lower, int upper) { int middle; if (lower != upper) { middle = (lower + upper) / 2; order(dest, source, lower, middle); order(dest, source, middle + 1, upper); merge(source, dest, lower, middle, upper); } template void merge(Etype source[], Etype dest[], int lower, int middle, int upper) { template void mergeSort(Etype A[], const int n) { Etype Acopy[n+1]; int size; for (int k = 1; k <= n; k++) Acopy[k] = A[k]; order(Acopy, A, 1, size); } template void order(Etype source[], Etype dest[], int lower, int upper) { int middle; if (lower != upper) { middle = (lower + upper) / 2; order(dest, source, lower, middle); order(dest, source, middle + 1, upper); merge(source, dest, lower, middle, upper); } template void merge(Etype source[], Etype dest[], int lower, int middle, int upper) { int s1 = lower; int s2 = middle + 1; int d = lower; do { if (source[s1] < source[s2]) { dest[d] = source[s1]; s1++; } else { dest[d] = source[s2]; s2++; } d++; } while ((s1 <= middle) && (s2 <= upper)); if (s1 > middle) do { dest[d] = source[s2]; s2++; d++; } while (s2 <= upper); else do { dest[d] = source[s1]; s1++; d++; } while (s1 <= middle); } int s1 = lower; int s2 = middle + 1; int d = lower; do { if (source[s1] < source[s2]) { dest[d] = source[s1]; s1++; } else { dest[d] = source[s2]; s2++; } d++; } while ((s1 <= middle) && (s2 <= upper)); if (s1 > middle) do { dest[d] = source[s2]; s2++; d++; } while (s2 <= upper); else do { dest[d] = source[s1]; s1++; d++; } while (s1 <= middle); }

16. CS 340 Page 126 Merge Sort Example Original List: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Original List: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #1: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #1: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #2: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #2: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #3: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #3: MulderScullySkinnerKrycekFrohikeLanglyByersBlevinsMathesonSpender Split #4: MulderScullySkinner Krycek FrohikeLanglyByersBlevins Matheson Spender Split #4: MulderScullySkinner Krycek FrohikeLanglyByersBlevins Matheson Spender Merge #4: MulderScullySkinner FrohikeKrycek LanglyByersBlevins MathesonSpender Merge #4: MulderScullySkinner FrohikeKrycek LanglyByersBlevins MathesonSpender Merge #3: MulderScullyFrohikeKrycekSkinnerByersLanglyBlevinsMathesonSpender Merge #3: MulderScullyFrohikeKrycekSkinnerByersLanglyBlevinsMathesonSpender Merge #2: FrohikeKrycekMulderScullySkinnerBlevinsByersLanglyMathesonSpender Merge #2: FrohikeKrycekMulderScullySkinnerBlevinsByersLanglyMathesonSpender Merge #1: BlevinsByersFrohikeKrycekLanglyMathesonMulderScullySkinnerSpender Merge #1: BlevinsByersFrohikeKrycekLanglyMathesonMulderScullySkinnerSpender

17. CS 340 Page 127 Merge Sort Analysis Worst-case time complexity for applying the merge function to a size-k subarray: M(k) = 18k-7. Time complexity for applying the order function to a size-k subarray: R(k), where R(1)=1 and R(k) = 5+M(k)+2R(k/2) = 18k-2+2R(k/2). This recurrence relation yields R(k) = 18klogk-logk+2. Time complexity for applying the mergesort function to a size-n subarray: T(n) = 8n+1+R(n) = 18nlogn+8n-logn+3. While this O(nlogn) time complexity is favorable, the requirement of a duplicate array is detrimental to the Merge Sort algorithm, possibly making it less popular than certain alternative choices. template void mergeSort(Etype A[], const int n) { Etype Acopy[n+1]; // 1 TU int size; for (int k = 1; k <= n; k++) // n iter. @ 2 TU Acopy[k] = A[k]; // 6 TU order(Acopy, A, 1, size); // R(n) TU } template void mergeSort(Etype A[], const int n) { Etype Acopy[n+1]; // 1 TU int size; for (int k = 1; k <= n; k++) // n iter. @ 2 TU Acopy[k] = A[k]; // 6 TU order(Acopy, A, 1, size); // R(n) TU } template void merge(Etype source[], Etype dest[], int lower, int middle, int upper) { int s1 = lower; int s2 = middle + 1; // 1 TU int d = lower; do { if (source[s1] < source[s2]) // If block: { // 14 TU dest[d] = source[s1]; s1++; } else { dest[d] = source[s2]; s2++; } d++; // 1 TU } while ((s1 <= middle) && // k-m iter. (s2 <= upper)); // @ 3 TU if (s1 > middle) // 1 TU do { dest[d] = source[s2]; // 6 TU s2++; // 1 TU d++; // 1 TU } while (s2 <= upper); // m iter. @ 1 TU else do { dest[d] = source[s1]; s1++; d++; } while (s1 <= middle); } template void merge(Etype source[], Etype dest[], int lower, int middle, int upper) { int s1 = lower; int s2 = middle + 1; // 1 TU int d = lower; do { if (source[s1] < source[s2]) // If block: { // 14 TU dest[d] = source[s1]; s1++; } else { dest[d] = source[s2]; s2++; } d++; // 1 TU } while ((s1 <= middle) && // k-m iter. (s2 <= upper)); // @ 3 TU if (s1 > middle) // 1 TU do { dest[d] = source[s2]; // 6 TU s2++; // 1 TU d++; // 1 TU } while (s2 <= upper); // m iter. @ 1 TU else do { dest[d] = source[s1]; s1++; d++; } while (s1 <= middle); } template void order(Etype source[], Etype dest[], int lower, int upper) { int middle; if (lower != upper) // 1 TU { middle = (lower + upper) / 2; // 3 TU order(dest, source, lower, middle); // R(k/2) TU order(dest, source, middle + 1, upper); // R(k/2)+1 TU merge(source, dest, lower, middle, upper); // M(k) TU } template void order(Etype source[], Etype dest[], int lower, int upper) { int middle; if (lower != upper) // 1 TU { middle = (lower + upper) / 2; // 3 TU order(dest, source, lower, middle); // R(k/2) TU order(dest, source, middle + 1, upper); // R(k/2)+1 TU merge(source, dest, lower, middle, upper); // M(k) TU }

18. CS 340 Page 128 Quick Sort Recursively select a pivot element and quickly place all list elements smaller than the pivot before it in the list and all elements larger than the pivot after it in the list. template void quickSort(Etype A[], int lower, int upper) { int pivot_point; partition(A, lower, upper, pivot_point); if (lower < pivot_point) quick_sort(A, lower, pivot_point - 1); if (upper > pivot_point) quick_sort(A, pivot_point + 1, upper); } template void partition(Etype A[], int lo, int hi, int &pivot_point) { Etype pivot = A[lo]; while (lo < hi) { while ((pivot < A[hi]) && (lo < hi)) hi--; if (hi != lo) { A[lo] = A[hi]; lo++; } while ((pivot > A[lo]) && (lo < hi)) lo++; if (hi != lo) { A[hi] = A[lo]; hi--; } A[hi] = pivot; pivot_point = hi; } template void quickSort(Etype A[], int lower, int upper) { int pivot_point; partition(A, lower, upper, pivot_point); if (lower < pivot_point) quick_sort(A, lower, pivot_point - 1); if (upper > pivot_point) quick_sort(A, pivot_point + 1, upper); } template void partition(Etype A[], int lo, int hi, int &pivot_point) { Etype pivot = A[lo]; while (lo < hi) { while ((pivot < A[hi]) && (lo < hi)) hi--; if (hi != lo) { A[lo] = A[hi]; lo++; } while ((pivot > A[lo]) && (lo < hi)) lo++; if (hi != lo) { A[hi] = A[lo]; hi--; } A[hi] = pivot; pivot_point = hi; }

19. CS 340 Page 129 Quick Sort Example Scooby Itchy Scratchy Huckleberry Tom Jerry Snoopy Sylvester Speedy Goofy Garfield Mickey Mickey Itchy Garfield Huckleberry Goofy Jerry Scooby Sylvester Speedy Snoopy Tom Scratchy Jerry Itchy Garfield Huckleberry Goofy Mickey Scooby Sylvester Speedy Snoopy Tom Scratchy Goofy Itchy Garfield Huckleberry Jerry Mickey Scooby Sylvester Speedy Snoopy Tom Scratchy Garfield Goofy Itchy Huckleberry Jerry Mickey Scooby Sylvester Speedy Snoopy Tom Scratchy Garfield Goofy Huckleberry Itchy Jerry Mickey Scooby Sylvester Speedy Snoopy Tom Scratchy Garfield Goofy Huckleberry Itchy Jerry Mickey Scooby Scratchy Speedy Snoopy Sylvester Tom Garfield Goofy Huckleberry Itchy Jerry Mickey Scooby Scratchy Snoopy Speedy Sylvester Tom Note that a more strategic choice for each new pivot position may be possible.

20. CS 340 Page 130 Quick Sort Analysis Time complexity for applying the quickSort: T(n)=T(i)+T(n-i-1)+P(n)+4, where P(n) is the partition time and i is the proper location of the pivot element. Note that T(1)=2 and that P(n) is O(n) since every non-pivot element is examined in turn and, at worst, repositioned, all of which would take constant time for each element. Average case (i is equally likely to be any location) T(n) is (through a rather elaborate analysis, shown in the Weiss text)  (nlogn). So, the algorithm is optimal except in certain extreme cases.

21. CS 340 Page 131 A Lower Bound for Comparison-Based Sorting a<b? c<d? a<c? b<d? b<c? a<d? Note that n n ≥ n! ≥ (n/2) n/2. Since the logarithm is an increasing function, log(n n ) ≥ log(n!) ≥ log((n/2) n/2 ), i.e., nlogn ≥ log(n!) ≥ (n/2)log(n/2 ) = ½nlogn- ½n, so log(n!) is  (nlogn). Since the logarithm is an increasing function, log(n n ) ≥ log(n!) ≥ log((n/2) n/2 ), i.e., nlogn ≥ log(n!) ≥ (n/2)log(n/2 ) = ½nlogn- ½n, so log(n!) is  (nlogn). We’ve seen sorting algorithms with time complexity O(nlogn), but is it possible to develop one that’s faster that? If a sorting algorithm is based upon comparisons (i.e., decisions are based upon whether one element is less than another), then how many comparisons are necessary to guarantee a complete sorting? Since there are n! ways to order n elements and each comparison will at best split the possibilities in half, the best that could be achieved would be log(n!). In this example, performing six comparisons in a particular order guarantees a complete sorting of four values. If certain branches of the tree had performed different comparisons, the depth of the tree could have been reduced. b<c<a<d a<d<b<c c<b<a<d b<a<d<c c<a<d<b b<c<d<a d<a<b<c c<b<d<a b<d<a<c c<d<a<b d<a<c<b a<d<c<b a<c<b<d d<c<b<a a<b<c<d d<b<c<a b<c<a<d b<c<d<a b<c<a<d b<c<d<a d<a<b<c a<d<b<c d<a<b<c a<d<b<c c<b<a<d c<b<d<a c<b<a<d c<b<d<a b<a<d<c b<d<a<c b<a<d<c b<d<a<c c<a<d<b c<d<a<b c<a<d<b c<d<a<b d<a<c<b a<d<c<b d<a<c<b a<d<c<b c<a<b<d b<d<c<a a<c<d<b d<b<a<c b<c<a<d c<b<a<d b<c<d<a c<b<d<a b<c<a<d c<b<a<d b<c<d<a c<b<d<a a<b<d<c b<a<d<c b<d<a<c b<a<d<c b<d<a<c c<a<d<b c<d<a<b c<a<d<b c<d<a<b c<d<b<a d<b<c<a d<c<b<a d<b<c<a d<c<b<a d<a<b<c d<a<c<b a<d<b<c a<d<c<b d<a<b<c d<a<c<b a<d<b<c a<d<c<b a<b<c<d a<c<b<d a<b<c<d a<c<b<d a<b<c<d a<c<b<d a<c<d<b a<b<c<d a<c<b<d a<c<d<b c<a<b<d c<a<d<b c<d<a<b c<a<b<d c<a<d<b c<d<a<b b<c<a<d c<b<a<d b<c<d<a c<b<d<a c<d<b<a b<c<a<d c<b<a<d b<c<d<a c<b<d<a c<d<b<a d<c<a<b d<b<c<a b<d<c<a d<c<b<a d<b<c<a b<d<c<a d<c<b<a b<a<c<d d<a<b<c a<b<d<c d<a<c<b a<d<b<c a<d<c<b d<a<b<c a<b<d<c d<a<c<b a<d<b<c a<d<c<b b<a<d<c d<b<a<c b<d<a<c b<a<d<c d<b<a<c b<d<a<c a<b<c<d c<a<b<d c<a<d<b a<c<b<d a<c<d<b c<d<a<b a<b<c<d c<a<b<d c<a<d<b a<c<b<d a<c<d<b c<d<a<b b<a<c<d b<c<a<d c<b<a<d b<c<d<a c<b<d<a c<d<b<a b<a<c<d b<c<a<d c<b<a<d b<c<d<a c<b<d<a c<d<b<a d<a<b<c a<b<d<c d<a<c<b a<d<b<c d<c<a<b a<d<c<b d<a<b<c a<b<d<c d<a<c<b a<d<b<c d<c<a<b a<d<c<b b<a<d<c d<b<a<c d<b<c<a b<d<a<c b<d<c<a d<c<b<a b<a<d<c d<b<a<c d<b<c<a b<d<a<c b<d<c<a d<c<b<a a<b<c<d c<a<b<d d<a<b<c a<b<d<c c<a<d<b d<a<c<b a<c<b<d a<c<d<b a<d<b<c c<d<a<b d<c<a<b a<d<c<b a<b<c<d c<a<b<d d<a<b<c a<b<d<c c<a<d<b d<a<c<b a<c<b<d a<c<d<b a<d<b<c c<d<a<b d<c<a<b a<d<c<b b<a<c<d b<a<d<c b<c<a<d c<b<a<d d<b<a<c b<c<d<a c<b<d<a d<b<c<a b<d<a<c b<d<c<a c<d<b<a d<c<b<a b<a<c<d b<a<d<c b<c<a<d c<b<a<d d<b<a<c b<c<d<a c<b<d<a d<b<c<a b<d<a<c b<d<c<a c<d<b<a d<c<b<a a<b<c<d b<a<c<d c<a<b<d d<a<b<c a<b<d<c b<a<d<c c<a<d<b d<a<c<b a<c<b<d b<c<a<d c<b<a<d d<b<a<c a<c<d<b b<c<d<a c<b<d<a d<b<c<a a<d<b<c b<d<a<c c<d<a<b d<c<a<b a<d<c<b b<d<c<a c<d<b<a d<c<b<a a<b<c<d b<a<c<d c<a<b<d d<a<b<c a<b<d<c b<a<d<c c<a<d<b d<a<c<b a<c<b<d b<c<a<d c<b<a<d d<b<a<c a<c<d<b b<c<d<a c<b<d<a d<b<c<a a<d<b<c b<d<a<c c<d<a<b d<c<a<b a<d<c<b b<d<c<a c<d<b<a d<c<b<a