1 Data Structures and Algorithms Sorting

2  Sorting is the process of arranging a list of items into a particular order  There must be some value on which the order is based  There are many algorithms for sorting a list of items  These algorithms vary in efficiency

3 SORTING  We will examine several algorithms:  Selection Sort  Bubble Sort  Insertion Sort

4 Selection Sort  The approach of Selection Sort:  select one value and put it in its final place in the sort list  repeat for all other values

5 In more detail:  find the smallest value in the list  switch it with the value in the first position  find the next smallest value in the list  switch it with the value in the second position  repeat until all values are placed

6 Selection Sort  An example: original: smallest is 1: smallest is 2: smallest is 3: smallest is 6:  Pick any item and insert it into its proper place in a sorted sublist  repeat until all items have been inserted

7 In more detail Selection Sort:  consider the first item to be sorted as a sub list (of one item)  insert the second item into the sorted sub list, shifting items as necessary to make room for the new addition  insert the third item into the sorted sublist (of two items), shifting as necessary  repeat until all values are inserted into their proper position

8 Insertion Sort  An example of an insertion sort occurs in everyday life while playing cards.  To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place.  This process is repeated until all the cards are in the correct sequence. Both average and worst-case time is O(n 2 ).

9 Sorting  Card players all know how to sort …  First card is already sorted  With all the rest, ¶ Scan back from the end until you find the first card larger than the new one, Ë Move all the lower ones down one slot ¸ insert it «A«A «K«K « 10 ªJªJ «Q«Q ¸

Insertion Sort– Another n^2 sort  An example, : original: insert 9: insert 6: insert 1: insert 2:  Each loop iteration produces a sorted sub list.

11 Insertion Sort

12 INSERTION SORT n elements.  Assuming there are n elements. n - 1 other entries, resulting in a  For each entry, we may need to examine and shift up to n - 1 other entries, resulting in a  O(n 2 ) algorithm.  The insertion sort is an in-place sort. That is, we sort the array in-place, no needed helper array.

13 INSERTION SORT  No extra memory is required.  The insertion sort is also a stable sort.  Stable sorts retain the original ordering of keys when identical keys are present in the input data.

14 Comparing Sorts  Both Selection and Insertion sorts are similar in efficiency  The both have outer loops that scan all elements,  and inner loops that compare the value of the outer loop with almost all values in the list

15 Comparing Selection & Insertion Sort n 2 number of comparisons are made to sort a list of size n  Therefore approximately n 2 number of comparisons are made to sort a list of size n order n 2  We therefore say that these sorts are of order n 2  Other sorts are more efficient: order n log 2 n

16 Sorting - Insertion sort  Complexity  For each card  Scan for smaller card O(n)  Shift cards down O(n)  Insert O(1)  Total O(n)  First card requires O(1), second O(2), …  For n cards operations ç O(n 2 )  i i=1 n

17 Sorting - Insertion sort  Complexity  For each card O(log n)  Scan O(n) O(log n) O(n)  Shift up O(n)  Insert O(1) O(n)  Total O(n)  First card requires O(1), second O(2), …  For n cards operations ç O(n 2 )  i i=1 n Unchanged! Because the shift up operation still requires O(n) time Use binary search to find location!

18 Sorting - Bubble  From the first element Exchange pairs if they’re out of order  Exchange pairs if they’re out of order  Last one must now be the largest  Repeat from the first to n-1  Stop when you have only one element to check

19 Bubble Sort SWAP(a,b) { int t; t=a; a=b; b=t; } void bubble( int a[], int n ) { int i, j; for( i=0; i < n; i++) { /* n passes thru the array */ /*From start to the end of unsorted part */ for( j=1; j < (n-i) ; j++) { /* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }

20 Bubble Sort - Analysis void SWAP(a,b) { int t; t=a; a=b; b=t; // see below} void bubble( int a[], int n ) { int i, j; for( i=0; i < n; i++) { /* n passes thru the array */ /* From start to the end of unsorted part */ for(j=1; j < (n-i); j++) { /* if adjacent items out of order, swap */ if( a[j-1] > a[j] ) SWAP ( a[j-1], a[j]); } O(1) statement

21 Bubble Sort - Analysis void bubble( int a[], int n ) { int i, j; for(i=0;i<n;i++) n passes thru the array for(i=0;i<n;i++) { /* n passes thru the array */ /* From start to the end of unsorted part */ for( j=1;j < (n-i); j++) { for( j=1;j < (n-i); j++) { /* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); } Inner loop n-1, n-2, n-3, …, i iterations O(1) statement

22 Bubble Sort - Analysis Overall  i i=n-1 1 = n(n+1) 2 = O(n 2 ) n outer loop iterations inner loop iteration count  BubbleSort

23 Complexity of the Bubble Sort  Thus:  ( N-1) + (N-2) = N*(N-2)/2 = O(N^2)  Thus Bubble Sort is an O(N^2) algorithm.

24 Sorting - Simple  Bubble sort  O(n 2 )  Very simple!!! code – slow, should be retired.  Insertion sort  Slightly better than bubble sort  Fewer comparisons  Also O(n 2 ) Selection sort is similar  But HeapSort is O(n log n)  Where would you use selection or insertion sort?

25 Simple Sorts  Selection Sort or Insertion Sort  Use when n is small  Simple code compensates for low efficiency!

26 Quicksort  Efficient sorting algorithm  Discovered by C.A.R. Hoare  Example of Divide and Conquer algorithm  Two phases  Partition phase  Divides the work into half  Sort phase  Conquers the halves!

27 Quicksort  Partition  Choose a pivot  Find the position for the pivot so that  all elements to the left are less  all elements to the right are greater < pivot> pivotpivot

28 Quicksort  Conquer  Apply the same algorithm to each half < pivot> pivot pivot< p’ p’> p’< p”p”> p”

29 QuickSort quick_sort(left, right) { Boundary Condition: when left >= right -- do nothing // Boundary Condition: when left >= right -- do nothing // Place first item of current sublist in its correct position, C, in the sublist itself, such that // List[i] <= List[C], for all i, First <= i < C // List[J] > List[C], for all J, C List[C], for all J, C < J <= Last quick_sort (left, C-1); quick_sort (C+1, right); }

30 QuickSort code int partition(int array [], int left, int right) { int i = left, j = right; int i = left, j = right; int tmp; int tmp; int pivot = array[(left + right) / 2]; int pivot = array[(left + right) / 2]; while (i <= j) { while (array[i] < pivot) while (array[i] < pivot) i++; while (array[j] > pivot) while (array[j] > pivot) j--; if (i <= j) if (i <= j) { tmp = array[i]; tmp = array[i]; array[i] = arrar[j]; array[i] = arrar[j]; array[j] = tmp; array[j] = tmp; i++; j--; } }; return i; return i; }

31 Quicksort  Implementation quicksort( int [] a, int low, int high ) { int pivot; /* Termination condition! */ if ( high > low ) { pivot = partition( a, low, high ); quicksort( a, low, pivot-1 ); quicksort( a, pivot+1, high ); } Divide Conquer

32 QuickSort void quickSort(int []array, int left, int right) { int index = partition(array, left, right); // sorts the left side of the array if (left < index - 1) quickSort(array, left, index - 1); // sorts the right side of the array if (index < right) quickSort(array, index, right); }

33 Quicksort - Partition int partition( int []a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low]; // pivot is first element in the array pivot = left = low; right = high; while ( left < right ) { /* Move left while item < pivot */ while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */ while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */ a[low] = a[right]; a[right] = pivot_item; return right; }

34 Quicksort - Partition This example uses int ’s to keep things simple! lowhigh Any item will do as the pivot, choose the leftmost one!

35 Quicksort - Partition Set left and right markers lowhigh pivot : 23 leftright

36 Quicksort - Partition Move the markers until they cross over lowhigh pivot : 23 leftright

37 Quicksort - Partition Move the left pointer while items <= pivot lowhigh pivot : 23 leftright Move right Items >=pivot

38 Quicksort - Partition Swap the two items on the wrong side of the value of pivot lowhigh pivot : 23 leftright

39 Quicksort - Partition left and right have swapped over, so stop lowhigh pivot : 23 leftright

40 Quicksort - Partition Finally, swap the pivot and a[ right} lowhigh pivot : 23 left right

41 Quicksort - Partition Return the position of the pivot lowhigh pivot : 23 right

42 Quicksort - Conquer pivot pivot : 23 Recursively sort left half Recursively sort right half

43 Quicksort - Analysis  Partition  Check every item once O(n)  Conquer  Divide data in half O(log 2 n)  Total  Product O(n log n)  But there’s a catch …………….

44 Quicksort - The truth!  What happens if we use quicksort on data that’s already sorted (or nearly sorted)  We’d certainly expect it to perform well!

45 Quicksort - The truth!  Sorted data pivot < pivot ? > pivot

46 Quicksort - The truth!  Sorted data  Each partition produces  a problem of size 0  and one of size n-1 !  Number of partitions? > pivot pivot

47 Quicksort - The truth!  Sorted data  Each partition produces  a problem of size 0  and one of size n-1 !  Number of partitions?  n each needing time O(n)  Total nO(n) or O(n 2 ) ? Quicksort is as bad as bubble or insertion sort > pivot pivot

48 Quicksort - The truth!  Quicksort’s O(n log n) behaviour  Depends on the partitions being nearly equal  there are O( log n ) of them  On average, this will nearly be the case and quicksort is generally O(n log n)  Can we do anything to ensure O(n log n) time ?  In general, no  But we can improve our chances!!

49 Quicksort - Choice of the pivot  Any pivot will work …  Choose a different pivot …  so that the partitions are equal  then we will see O(n log n) time pivot < pivot > pivot

50 Quicksort - Median-of-3 pivot  Take 3 positions and choose the median  say … First, middle, last  median is 5 ç perfect division of sorted data every time!  O(n log n) time ç Since sorted (or nearly sorted) data is common, median-of-3 is a good strategy especially if you think your data may be sorted!

51 Quicksort - Random pivot  Choose a pivot randomly  Different position for every partition ç On average, sorted data is divided evenly  O(n log n) time Key requirement Pivot choice must take O(1) time

52 Quicksort - Guaranteed O(n log n)?  Never!!  Any pivot selection strategy could lead to O(n 2 ) time Here median-of-3 chooses 2 è One partition of 1 and One partition of 7 Next it chooses 4 è One of 1 and One of

53 Sorting- Key Points  Sorting  Bubble, Insertion, selection sort  O(n 2 ) sorts  Simple code  May run faster for small n, n ~10 (system dependent)  Quick Sort  Divide and conquer  O(n log n)

54 Sorting - Key Points  Quick Sort  O(n log n) but ….  Can be O(n 2 )  Depends on pivot selection  Median-of-3  Random pivot  Better but not guaranteed

55 Quicksort - Why bother?  Use Heapsort instead? Quicksort is generally faster Fewer comparisons and exchanges Some empirical data