Sorting Why? Displaying in order Faster Searching Categories Internal List elements manipulated in RAM Faster Limited by amount of RAM External External (secondary) storage areas used Slower Used for Larger Lists Limited by secondary storage (Disk Space)

Sorting Why? Displaying in order Faster Searching Categories Internal List elements manipulated in RAM Faster Limited by amount of RAM External External (secondary) storage areas used Slower Used for Larger Lists Limited by secondary storage (Disk Space)

Basic Internal Sort Types • Exchange (e.g., bubble sort) Single list Incorrectly ordered pairs swapped as found • Selection Two lists (generally); Selection with exchange uses one list Largest/Smallest selected in each pass and moved into position • Insertion One or two lists (two more common) Each item from original list inserted into the correct position in the new list

Exchange Sorts • Point to bottom element • Compare with element above: Bubble Sort 1: The largest element ‘bubbles’ up Given: 7 2 6 1 3 4 8 10 9 5 • Point to bottom element • Compare with element above: • if the element is greater, swap positions (in this case, swap) • if the element is smaller, reset the bottom pointer (not here) • Continue the process until the largest element is at the end • This will require n-1 comparisons (9 for our example) (where n = the length of the unsorted list) • At the end of the pass: • The largest number is in the last position • The length of the unsorted list has been shortened by 1

How does this work?? Comparison: Pass #1: 7 2 6 1 3 4 8 10 9 5 2 7 6 1 Swap 2 2 7 6 1 3 4 8 10 9 5 Swap 3 2 6 7 1 3 4 8 10 9 5 Swap 4 2 6 1 7 3 4 8 10 9 5 Swap 5 2 6 1 3 7 4 8 10 9 5 Swap 6 2 6 1 3 4 7 8 10 9 5 Don’t Swap

• 9 (n - 1) comparisons were required Continuing Comparison: Pass #1: 7 2 6 1 3 4 7 8 10 9 5 Don’t Swap 8 2 6 1 3 4 7 8 10 9 5 Swap 9 2 6 1 3 4 7 8 9 10 5 Swap The new list appears as 2 6 1 3 4 7 8 9 5 10 Note: • 9 (n - 1) comparisons were required • We know that the largest element is at the end of the list

Continuing Comparison: Pass #2: 1 (10) 2 6 1 3 4 7 8 9 5 10 Don’t’ Swap 2 (11) 2 6 1 3 4 7 8 9 5 10 Swap 3 (12) 2 1 6 3 4 7 8 9 5 10 Swap 4 (13) 2 1 3 6 4 7 8 9 5 10 Swap 5 (14) 2 1 3 4 6 7 8 9 5 10 Don’t Swap 6 (15) 2 1 3 4 6 7 8 9 5 10 Don’t Swap 7 (16) 2 1 3 4 6 7 8 9 5 10 Don’t Swap 8 (17) 2 1 3 4 6 7 8 9 5 10 Swap

Continuing Comparison: Pass #3: 1 (18) 2 1 3 4 6 7 8 5 9 10 Swap 2 (19) 1 2 3 4 6 7 8 5 9 10 Don’t Swap 3 (20) 1 2 3 4 6 7 8 5 9 10 Don’t Swap 4 (21) 1 2 3 4 6 7 8 5 9 10 Don’t Swap 5 (22) 1 2 3 4 6 7 8 5 9 10 Don’t Swap 6 (23) 1 2 3 4 6 7 8 5 9 10 Don’t Swap 7 (24) 1 2 3 4 6 7 8 5 9 10 Swap

Continuing Comparison: Pass #4: 1 (25) 1 2 3 4 6 7 5 8 9 10 Don’t Swap 2 (26) 1 2 3 4 6 7 5 8 9 10 Don’t Swap 3 (27) 1 2 3 4 6 7 5 8 9 10 Don’t Swap 4 (28) 1 2 3 4 6 7 5 8 9 10 Don’t Swap 5 (29) 1 2 3 4 6 7 5 8 9 10 Don’t Swap 6 (30) 1 2 3 4 6 7 5 8 9 10 Don’t Swap

Right ??? Comparison: Pass #5: 1 2 3 4 6 5 7 8 9 10 1 2 3 4 6 5 7 8 9 Continuing Comparison: Pass #5: 1 (31) 1 2 3 4 6 5 7 8 9 10 Don’t Swap 2 (32) 1 2 3 4 6 5 7 8 9 10 Don’t Swap 3 (33) 1 2 3 4 6 5 7 8 9 10 Don’t Swap 4 (34) 1 2 3 4 6 5 7 8 9 10 Don’t Swap 5 (35) 1 2 3 4 6 5 7 8 9 10 Swap And the new list 1 2 3 4 5 6 7 8 9 10 Is in order, so we can stop. Right ???

Maximum Comparisons necessary NO. In the WORST case scenario (numbers in reverse order): 10 9 8 7 6 5 4 3 2 1 A bubble sort would yield: Pass After Pass Order Comparisons 1 9 9 8 7 6 5 4 3 2 1 10 2 8 8 7 6 5 4 3 2 1 9 10 3 7 7 6 5 4 3 2 1 8 9 10 4 6 6 5 4 3 2 1 7 8 9 10 5 5 5 4 3 2 1 6 7 8 9 10 6 4 4 3 2 1 5 6 7 8 9 10 7 3 3 2 1 4 5 6 7 8 9 10 2 8 2 1 3 4 5 6 7 8 9 10 1 9 1 2 3 4 5 6 7 8 9 10 Maximum Comparisons necessary 45

S [(n-1)+(n-2)+...+1] or (n2 - n)/2 comparisons. What does this imply ??? If we want to be sure, given an array of n dimensions, we need a maximum of n-1 passes to sort the array, and a total of: S [(n-1)+(n-2)+...+1] or (n2 - n)/2 comparisons. No. Items Max. Passes: (n - 1) Max. Compares: (n2 - n)/2 10 9 45 100 99 4,950 1,000 999 499,500 10,000 9,999 49,995,000 100,000 99,999 4,999,950,000 1,000,000 999,999 499,999,500,000 The C code necessary?

#include "stdafx.h" #include <iostream> using namespace std; void main() { int pass=0, compare=0, swaps=0, top=9, i, j, temp, sorted = 1, iarray[10]={7,2,6,1,3,4,8,10,9,5}; while ((top > 0) && (sorted == 1)) // check end AND if NOT sorted { pass++; // increment ctr sorted = 0; // reset our flag for (i = 0; i < top; i++) // begin pass { compare++; // increment ctr if (iarray[i] > iarray[i+1])/ // ?? out of order { swaps++; // increment ctr sorted = 1; // set the flag temp = iarray[i]; // temp. storage iarray[i] = iarray[i+1]; // swap iarray[i+1] = temp; } cout << "Pass: " << pass << " Comcomparisons: " << swaps << endl; for (j = 0; j < 10; j++) cout << iarray[j]; // print element cout << endl; top--;

include <stdio.h> void main() { int pass=0, compare=0, swaps=0, top=9, i, j, temp, iarray[10]={7,2,6,1,3,4,8,10,9,5}; while (top > 0) // check end { pass++; // increment ctr for (i = 0; i < top; i++) // begin pass { compare++; // increment ctr if (iarray[i] > iarray[i+1]) // ?? out of order { swaps++; // increment ctr temp = iarray[i]; // temp. storage iarray[i] = iarray[i+1]; // swap iarray[i+1] = temp; } printf("%3d %3d %3d: ", pass,compare,swaps); for (j = 0; j < 10; j++) printf("%3d",iarray[j]); // print element printf("\n"); top--;

The Output (modified slightly) would appear as: Pass Comparison Swap Order Pass Comparison Swap Order 1 1 1: 2 7 6 1 3 4 8 10 9 5 1 2 2: 2 6 7 1 3 4 8 10 9 5 1 3 3: 2 6 1 7 3 4 8 10 9 5 1 4 4: 2 6 1 3 7 4 8 10 9 5 1 5 5: 2 6 1 3 4 7 8 10 9 5 1 6 5: 2 6 1 3 4 7 8 10 9 5 1 7 5: 2 6 1 3 4 7 8 10 9 5 1 8 6: 2 6 1 3 4 7 8 9 10 5 1 9 7: 2 6 1 3 4 7 8 9 5 10 2 10 7: 2 6 1 3 4 7 8 9 5 10 2 11 8: 2 1 6 3 4 7 8 9 5 10 2 12 9: 2 1 3 6 4 7 8 9 5 10 2 13 10: 2 1 3 4 6 7 8 9 5 10 2 14 10: 2 1 3 4 6 7 8 9 5 10 2 15 10: 2 1 3 4 6 7 8 9 5 10 2 16 10: 2 1 3 4 6 7 8 9 5 10 2 17 11: 2 1 3 4 6 7 8 5 9 10 3 18 12: 1 2 3 4 6 7 8 5 9 10 3 19 12: 1 2 3 4 6 7 8 5 9 10 3 20 12: 1 2 3 4 6 7 8 5 9 10 3 21 12: 1 2 3 4 6 7 8 5 9 10 3 22 12: 1 2 3 4 6 7 8 5 9 10 3 23 12: 1 2 3 4 6 7 8 5 9 10 3 24 13: 1 2 3 4 6 7 5 8 9 10 4 25 13: 1 2 3 4 6 7 5 8 9 10 4 26 13: 1 2 3 4 6 7 5 8 9 10 4 27 13: 1 2 3 4 6 7 5 8 9 10 4 28 13: 1 2 3 4 6 7 5 8 9 10 4 29 13: 1 2 3 4 6 7 5 8 9 10 4 30 14: 1 2 3 4 6 5 7 8 9 10 5 31 14: 1 2 3 4 6 5 7 8 9 10 5 32 14: 1 2 3 4 6 5 7 8 9 10 5 33 14: 1 2 3 4 6 5 7 8 9 10 5 34 14: 1 2 3 4 6 5 7 8 9 10 5 35 15: 1 2 3 4 5 6 7 8 9 10 6 36 15: 1 2 3 4 5 6 7 8 9 10 6 37 15: 1 2 3 4 5 6 7 8 9 10 6 38 15: 1 2 3 4 5 6 7 8 9 10 6 39 15: 1 2 3 4 5 6 7 8 9 10 7 40 15: 1 2 3 4 5 6 7 8 9 10 7 41 15: 1 2 3 4 5 6 7 8 9 10 7 42 15: 1 2 3 4 5 6 7 8 9 10 8 43 15: 1 2 3 4 5 6 7 8 9 10 8 44 15: 1 2 3 4 5 6 7 8 9 10 9 45 15: 1 2 3 4 5 6 7 8 9 10

How would the C code appear? Since the list IS sorted after 5 passes (35 comparisons), why can’t we stop?? We could, IF we knew the list was sorted: • If we make a pass without swapping any elements, we know the list is sorted (one extra pass is needed) • We need a flag which we set to 0 (zero) before each pass: • If we make any swaps in the pass, we set the flag to 1 • If we exit the loop, and the flag = 0, the list is sorted For our example, we could stop after Pass 6 (39 comparisons) How would the C code appear?

#include "stdafx.h" #include <iostream> using namespace std; void main() { int pass=0, compare=0, swaps=0, top=9, i, j, temp, sorted = 1, iarray[10]={7,2,6,1,3,4,8,10,9,5}; while ((top > 0) && (sorted == 1)) // check end A ND if NOT sorted { pass++; // increment ctr sorted = 0; // reset our flag for (i = 0; i < top; i++) // begin pass { compare++; // increment ctr if (iarray[i] > iarray[i+1]) // ?? out of order { swaps++; // increment ctr sorted = 1; // set the flag temp = iarray[i]; // temp. storage iarray[i] = iarray[i+1]; // swap iarray[i+1] = temp; } cout << "Pass: " << pass << " Comcomparisons: " << swaps << endl; for (j = 0; j < 10; j++) cout << iarray[j]; // print element cout << endl; top--;

Could we refine the bubble sort?? We could ‘bubble - up’ in one pass (as we did before) AND ‘bubble-down’ in the next pass. Consider our list after our first pass (9th comparison): 2 6 1 3 4 7 8 9 5 10 Starting at the top of the list, we now ‘bubble-down’ the smallest element (‘1’ will end up at the bottom of the list): Comparison Pass #2 1 (10) 2 6 1 3 4 7 8 9 5 10 Swap 2 (11) 2 6 1 3 4 7 8 5 9 10 Swap

Continuing: Pass #2: Comparison: 3 (12) 2 6 1 3 4 7 5 8 9 10 Swap 4 (13) 2 6 1 3 4 5 7 8 9 10 Don’t Swap 5 (14) 2 6 1 3 4 5 7 8 9 10 Don’t Swap 6 (15) 2 6 1 3 4 5 7 8 9 10 Don’t Swap 7 (16) 2 6 1 3 4 5 7 8 9 10 Swap 8 (17) 2 1 6 3 4 5 7 8 9 10 Swap

Continuing: Comparison: Pass #3: 1 (18) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 2 (19) 1 2 6 3 4 5 7 8 9 10 Swap 3 (20) 1 2 3 6 4 5 7 8 9 10 Swap 4 (21) 1 2 3 4 6 5 7 8 9 10 Swap 5 (22) 1 2 3 4 5 6 7 8 9 10 Swap 6 (23) 1 2 3 4 5 6 7 8 9 10 Don’t Swap 7 (24) 1 2 3 4 5 6 7 8 9 10 Don’t Swap

Since the List is in order, Can we Stop?? NO: Remember, we need one pass WITHOUT a swap Comparison: Pass #4: 1 (25) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 2 (26) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 3 (27) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 4 (28) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 5 (29) 1 2 6 3 4 5 7 8 9 10 Don’t Swap 6 (30) 1 2 6 3 4 5 7 8 9 10 Don’t Swap

include <stdio.h> void swap(int *swaparray, int a, int b); int sorted = 1; void main() { int bottom = 0, top=9, i, iarray[10]={7,2,6,1,3,4,8,10,9,5}; while ((top > bottom) && (sorted == 1)) // check end AND if NOT sorted { sorted = 0; // reset our flag for (i = bottom; i < top; i++) // begin bubble-up pass if (iarray[i] > iarray[i+1]) // ?? out of order swap(iarray, i, i+1); // Swap the elements top--; if ((top > bottom) && (sorted == 1)) // check end AND if NOT sorted { sorted = 0; // reset our flag for (i = top; i > bottom; i--) // begin bubble-down pass if (iarray[i] < iarray[i-1]) // ?? out of order swap(iarray, i, i-1); // Swap the elements bottom++; } void swap(int *swaparray, int a, int b) { int temp; sorted = 1; // set the flag temp = swaparray[a]; // temp. storage swaparray[a] = swaparray[b]; // swap swaparray[b] = temp;

Are there better sorting methods? YES: Generally speaking, bubble sorts are very slow The Quicksort Method: Generally the fastest internal sorting method intended for longer lists How does a quicksort work? As we have seen, the shorter the list, the faster the sort Quicksort recursively partitions the list into smaller sublists, gradually moving the elements into their correct position

Step 1: Choose a pivot element from list • Optimal Pivot: Median element • One alternative: Median of list The pivot element will divide the list in half 7 2 6 9 4 3 8 10 1 5 Step 2: Partition The List • move numbers larger than pivot to right, smaller numbers to left • compare leftmost with rightmost until a swap is needed Elements in Order: No Swap 7 3 6 9 4 2 8 10 1 5 Elements out of order: Swap needed Swap Elements Elements out of order: Swap needed

Continue with remaining elements: 1 3 6 9 4 2 8 10 7 5 No Swap No Swap Swap No Swap Swap New List: 1 3 2 9 4 6 8 10 7 5 Swap The Left and right partitions are partially sorted: 1 3 2 4 9 6 8 10 7 5 Smaller Elements Larger Elements

Midpoint = (bottom + top )/2 = (0 + 3)/ 2 = 1 1 3 2 4 1 3 2 4 1 2 3 4 Put the LEFT Partition in Order (even though already sorted): Step 1: Select Pivot: Midpoint = (bottom + top )/2 = (0 + 3)/ 2 = 1 1 3 2 4 Array Offset: 0 1 2 3 Repeat Step 2 with the partitioned list: 1 3 2 4 No Swap No Swap Swap We made 1 swap. Our new partitioned list appears as: 1 2 3 4 Smaller Elements Larger Elements

OK – So the list is in order. We can stop, Right??? Not really. The only way to be sure that the complete list is in order is to keep breaking the list down until there no swaps are made or there is only one element on each sublist. Looking at the left sublist: 1 2 All we know is that the elements on it are smaller than the elements on the right sub-list. The order could have been: 2 1 Assume that it was the sublist above. We have to continue making sublists: The list midpoint is (0 + 1)/2 = 0 2 1 Swap NOW we are done since each sublist contains only one element

Midpoint = (bottom + top )/2 = (4 + 9)/ 2 = 6 9 6 8 10 7 5 9 6 8 10 7 Now put the RIGHT Partition in Order: Step 1: Select Pivot: Midpoint = (bottom + top )/2 = (4 + 9)/ 2 = 6 9 6 8 10 7 5 Array Offset: 4 5 6 7 8 9 Repeat Step 2 with the partitioned list: 9 6 8 10 7 5 Swap Swap 5 6 8 10 7 9 No Swap Swap New Partitioned List: 5 6 7 10 8 9 Smaller Elements Larger Elements

Midpoint = (bottom + top )/2 = (4 + 6)/ 2 = 5 5 6 7 5 6 7 Put the new LEFT Partition in Order (already sorted): Step 1: Select Pivot: Midpoint = (bottom + top )/2 = (4 + 6)/ 2 = 5 5 6 7 Array Offset: 4 5 6 Repeat Step 2 with the partitioned list: 5 6 7 No Swap No Swap Since no swaps were made, the partition is in order

Step 1: Find new right pivot: Once again, put the new RIGHT Partition in Order: Step 1: Select Pivot: Midpoint = (bottom + top )/2 = (7 + 9)/ 2 = 8 10 8 9 Array Offset: 7 8 9 Repeat Step 2 with the partitioned list: 10 8 9 Swap No Swap 8 10 9 Step 1: Find new right pivot: Pivot = (8 + 9)/2 = 8 Offset: 8 9 Note that since the (new) left partition contains only 1 (one) element, it MUST be in order Swap Step 2: Check Order: And the new right list: 9 10 Is Sorted (as is the whole list)

This seems very complicated. Is it worth it?? Maybe: • For our list, we needed 22 comparisons and 7 swaps (vs. 30 comparisons and 15 swaps for our 2-ways sort with checks). • The WORST case scenario for a quicksort is: log2 n! Comparing a quicksort with a Bubble Sort: Elements Max. Bubble Sort Max Quicksort 10 45 22 4,950 525 100 999,500 9,965 1,000 49,995,000 132,877 10,000 What About the C Code necessary ?? It’s pretty simple, but it involves a new procedure: RECURSION Recusion is when a function calls itself.

#include <stdio.h> int quicksort(int a[], int first, int last); void swap(int *a, int *b); void main() { int iarray[10] = {7,2,6,9,4,3,8,10,1,5}; quicksort(iarray,0,9); } int quicksort(int list[], int first, int last) { int lower = first, upper = last, bound = list[(first + last)/2]; while (lower <= upper) { while (list[lower] < bound) lower++; while (bound < list[upper]) upper--; } if (lower < upper) swap(&list[lower++],&list[upper--]); } else lower++; if (first < upper) quicksort(list,first,upper); if (upper + 1 < last) quicksort(list,upper+1,last); } void swap(int *a, int *b) { int i, temp; temp = *a; *a = *b; *b = temp; }

Why is sorting important?? This illustrates a major trade-off in programming: Finding elements in a list is much quicker if the list is sorted (as we have seen, a binary search is exponentially faster than a sequential search) Sorting is a difficult and time-consuming task (as is maintaining a sorted list)