Sorting Ordering data

Design and Analysis of Sorting Assumptions –sorting will be internal (in memory) –sorting will be done on an array of elements –the number of elements will be somewhat small (less than a few million)

Why should we sort? Searching –Searching a sorted array is much easier than searching an unsorted array Examples of sorted lists –Dictionaries –File directories –Card catalogues –Programs for graduation

Sorts we will consider Bubble Sort Selection Sort Insertion Sort Shell Sort Quick Sort Merge Sort

Performance There are two operations which affect the performance of sorting algorithms –Comparison comparing two objects to see which one comes first –Exchanges exchanging two objects which are ‘out of order’ Performance is affected by the number of each of these that must be done

Bubble Sort //a is the array to be sorted for(int p = 0; p < a.length-1; p++) for(int k = 1; k < a.length-p; k++) if (a[k].compareTo(a[k-1])< 0) exchange(a[k-1],a[k]);

Analysis of Bubble Sort Assume N is the number of elements in the array The number of comparisons is (N-1)+(N-2)+…+1 =(N-1)N/2 = O( N 2 ) The number of exchanges is between 0 and (N-1)N/2, depending on the data Poorest performance is on reversed data

Selection Sort for(int p = 0; p < a.length-1; p++) { int k = indexOfMin(a,p,a.length-1); exchange(a[p],a[k]); }

Analysis of Selection Sort indexOfMin() clearly performs O( N ) comparisons There are, thus, O( N 2 ) comparisons performed However, there are exactly N-1 exchanges performed (although some of these may exchange an object with itself!)

Insertion Sort for(int p = 1; p < a.length; p++) { tmp = a[p]; int j = p; while (j > 0 && tmp.compareTo(a[j-1])< 0) { a[j] = a[j-1]; j--; } a[j]=tmp; }

Analysis of Insertion Sort The outer (for) loop is O( N ) The inner (while) loop executes 0 to N-1 times, depending on the data Insertion Sort does O( N 2 ) comparisons and data moves If the data is sorted already, insertion sort performs N-1 comparisons and 0 exchanges.

Inversions An inversion in an array, a, is any pair (i,j), where i a[j] The average number of inversions in an array of N elements is N(N-1)/4 Any sort algorithm that sorts by exchanging adjacent elements performs N(N-1)/4=  ( N 2 ) exchanges on average.

Shell sort Developed in 1959 by Donald Shell, to improve the performance of insertion sort. Avoids large amounts of data movement by first comparing elements that are far apart, and then comparing ones less far apart, and so on, gradually shrinking toward insertion sort

Shell Sort algorithm for(int gap = a.length/2; gap > 0; gap = gap == 2 ? 1: (int)(gap/2.2)) for(int i = gap; i < a.length; i++) { tmp = a[i]; int j = i; while(j>= gap && tmp.compareTo(a[j-gap])<0) { a[j]= a[j-gap]; j-= gap; } a[j] = tmp; }

Analysis of Shell Sort It can be shown that the worst case for Shell sort is O( N 2 ) With appropriate choice of decreasing gaps, this performance can be improved [dividing the previous gap by 2.2 performs better in practice than dividing by 2, for example] Shell sort is acceptable in practice, even with N in the tens of thousands

Divide & Conquer Divide and conquer sorting algorithms can possibly break the O( N 2 ) barrier There are two phases of these algorithms –divide array into two smaller arrays –sort and then combine these subarrays The algorithms –QuickSort –MergeSort

MergeSort Algorithm –if the number of elements to sort is 0 or 1, return –recursively sort the left and right halves –merge these two halves into a sorted list

MergeSort private static void mergeSort( Comparable [ ] a, Comparable [ ] tmpArray, int left, int right ) { if( left < right ) { int center = ( left + right ) / 2; mergeSort( a, tmpArray, left, center ); mergeSort( a, tmpArray, center + 1, right ); merge( a, tmpArray, left, center + 1, right ); } }

MergeSort Continued private static void merge( Comparable [ ] a, Comparable [ ] tmpArray, int leftPos, int rightPos, int rightEnd ) { int leftEnd = rightPos - 1; int tmpPos = leftPos; int numElements = rightEnd - leftPos + 1; // Main loop while( leftPos <= leftEnd && rightPos <= rightEnd ) if( a[ leftPos ].compareTo( a[ rightPos ] ) <= 0 ) tmpArray[ tmpPos++ ] = a[ leftPos++ ]; else tmpArray[ tmpPos++ ] = a[ rightPos++ ];

MergeSort Continued while( leftPos <= leftEnd ) // Copy rest of first half tmpArray[ tmpPos++ ] = a[ leftPos++ ]; while( rightPos <= rightEnd ) // Copy rest of right half tmpArray[ tmpPos++ ] = a[ rightPos++ ]; // Copy tmpArray bac for( int i = 0; i < numElements; i++, rightEnd-- ) a[ rightEnd ] = tmpArray[ rightEnd ]; }

Analysis of MergeSort The division phase is done in constant time. Merging is a O( N ) operation It takes log N divisions to get to size 1 Thus, MergeSort is O( N log N ) The major drawback of MergeSort –merging requires twice the space

QuickSort Algorithm to sort S –if the number of elements in S is 0 or 1, return –pick any element v in S, call it the pivot –Partition S into three sets L = { x  S | x < v } M = { x  S | x = v } U = { x  S | x > v } –return QuickSort(L), M, QuickSort(U)

QuickSort private static void quicksort( Comparable [ ] a, int low, int high) { if( low + CUTOFF > high ) insertionSort( a, low, high ); else { // Sort low, middle, high int middle = ( low + high ) / 2; if( a[ middle ].compareTo( a[ low ] ) < 0 ) swapReferences( a, low, middle ); if( a[ high ].compareTo( a[ low ] ) < 0 ) swapReferences( a, low, high ); if( a[ high ].compareTo( a[ middle ] ) < 0 ) swapReferences( a, middle, high );

QuickSort Continued / / Place pivot at position high - 1 swapReferences( a, middle, high - 1 ); Comparable pivot = a[ high - 1 ]; // Begin partitioning int i, j; for( i = low, j = high - 1; ; ) { while( a[ ++i ].compareTo( pivot ) = j ) break; swapReferences( a, i, j ); }

QuickSort Continued // Restore pivot swapReferences( a, i, high - 1 ); quicksort( a, low, i - 1 ); // Sort small elements quicksort( a, i + 1, high ); // Sort large elements } //else }

Analysis of QuickSort In the worst case, one of the sets L or U is empty, and the performance is O( N 2 ) In the best case (L and U are the same size at every stage), the performance is O( N log N ) In the average case, the performance can be shown to be O( N log N )

choosing the pivot Naïve choice – choose the first element –leads to the worst case if data is almost sorted or almost reverse order Another bad choice – choose the element at index (low +hi)/2 –easy to construct data where worst case performance happens

choosing the pivot (2) A better choice - median of three –choose the median of elements at indices low, (low+high)/2, and high and use this as the pivot. –The chance that one of the sets L or U is empty is reduced by this technique

Any algorithm that sorts by using element comparisons only must use at least  log( N !)  comparison for some input sequence  log( N !)  is approximately N log N N The best sort of this type, thus, has performance O( N log N ) in the average case