1. 1 Data Structures and Algorithms Sorting

2. 2  Sorting is the process of arranging a list of items into a particular order  There must be some value on which the order is based  There are many algorithms for sorting a list of items  These algorithms vary in efficiency

3. 3 SORTING  We will examine several algorithms:  Selection Sort  Bubble Sort  Insertion Sort

4. 4 Selection Sort  The approach of Selection Sort:  select one value and put it in its final place in the sort list  repeat for all other values

5. 5 In more detail:  find the smallest value in the list  switch it with the value in the first position  find the next smallest value in the list  switch it with the value in the second position  repeat until all values are placed

6. 6 Selection Sort  An example: original: 3 9 6 1 2 smallest is 1: 1 9 6 3 2 smallest is 2: 1 2 6 3 9 smallest is 3: 1 2 3 6 9 smallest is 6: 1 2 3 6 9  Pick any item and insert it into its proper place in a sorted sublist  repeat until all items have been inserted

7. 7 In more detail Selection Sort:  consider the first item to be sorted as a sub list (of one item)  insert the second item into the sorted sub list, shifting items as necessary to make room for the new addition  insert the third item into the sorted sublist (of two items), shifting as necessary  repeat until all values are inserted into their proper position

8. 8 Insertion Sort  An example of an insertion sort occurs in everyday life while playing cards.  To sort the cards in your hand you extract a card, shift the remaining cards, and then insert the extracted card in the correct place.  This process is repeated until all the cards are in the correct sequence. Both average and worst-case time is O(n 2 ).

9. 9 Sorting  Card players all know how to sort …  First card is already sorted  With all the rest, ¶ Scan back from the end until you find the first card larger than the new one, Ë Move all the lower ones down one slot ¸ insert it «A«A «K«K « 10 ªJªJ «Q«Q ¸

10. Insertion Sort– Another n^2 sort  An example, : original: 3 9 6 1 2 insert 9: 3 9 6 1 2 insert 6: 3 6 9 1 2 insert 1: 1 3 6 9 2 insert 2: 1 2 3 6 9  Each loop iteration produces a sorted sub list.

11. 11 Insertion Sort

12. 12 INSERTION SORT n elements.  Assuming there are n elements. n - 1 other entries, resulting in a  For each entry, we may need to examine and shift up to n - 1 other entries, resulting in a  O(n 2 ) algorithm.  The insertion sort is an in-place sort. That is, we sort the array in-place, no needed helper array.

13. 13 INSERTION SORT  No extra memory is required.  The insertion sort is also a stable sort.  Stable sorts retain the original ordering of keys when identical keys are present in the input data.

14. 14 Comparing Sorts  Both Selection and Insertion sorts are similar in efficiency  The both have outer loops that scan all elements,  and inner loops that compare the value of the outer loop with almost all values in the list

15. 15 Comparing Selection & Insertion Sort n 2 number of comparisons are made to sort a list of size n  Therefore approximately n 2 number of comparisons are made to sort a list of size n order n 2  We therefore say that these sorts are of order n 2  Other sorts are more efficient: order n log 2 n

16. 16 Sorting - Insertion sort  Complexity  For each card  Scan for smaller card O(n)  Shift cards down O(n)  Insert O(1)  Total O(n)  First card requires O(1), second O(2), …  For n cards operations ç O(n 2 )  i i=1 n

17. 17 Sorting - Insertion sort  Complexity  For each card O(log n)  Scan O(n) O(log n) O(n)  Shift up O(n)  Insert O(1) O(n)  Total O(n)  First card requires O(1), second O(2), …  For n cards operations ç O(n 2 )  i i=1 n Unchanged! Because the shift up operation still requires O(n) time Use binary search to find location!

18. 18 Sorting - Bubble  From the first element Exchange pairs if they’re out of order  Exchange pairs if they’re out of order  Last one must now be the largest  Repeat from the first to n-1  Stop when you have only one element to check

19. 19 Bubble Sort SWAP(a,b) { int t; t=a; a=b; b=t; } void bubble( int a[], int n ) { int i, j; for( i=0; i < n; i++) { /* n passes thru the array */ /*From start to the end of unsorted part */ for( j=1; j < (n-i) ; j++) { /* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }

20. 20 Bubble Sort - Analysis void SWAP(a,b) { int t; t=a; a=b; b=t; // see below} void bubble( int a[], int n ) { int i, j; for( i=0; i < n; i++) { /* n passes thru the array */ /* From start to the end of unsorted part */ for(j=1; j < (n-i); j++) { /* if adjacent items out of order, swap */ if( a[j-1] > a[j] ) SWAP ( a[j-1], a[j]); } O(1) statement

21. 21 Bubble Sort - Analysis void bubble( int a[], int n ) { int i, j; for(i=0;i<n;i++) n passes thru the array for(i=0;i<n;i++) { /* n passes thru the array */ /* From start to the end of unsorted part */ for( j=1;j < (n-i); j++) { for( j=1;j < (n-i); j++) { /* If adjacent items out of order, swap */ if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); } Inner loop n-1, n-2, n-3, …, i iterations O(1) statement

22. 22 Bubble Sort - Analysis Overall  i i=n-1 1 = n(n+1) 2 = O(n 2 ) n outer loop iterations inner loop iteration count  BubbleSort

23. 23 Complexity of the Bubble Sort  Thus:  ( N-1) + (N-2) +... + 2 + 1 = N*(N-2)/2 = O(N^2)  Thus Bubble Sort is an O(N^2) algorithm.

24. 24 Sorting - Simple  Bubble sort  O(n 2 )  Very simple!!! code – slow, should be retired.  Insertion sort  Slightly better than bubble sort  Fewer comparisons  Also O(n 2 ) Selection sort is similar  But HeapSort is O(n log n)  Where would you use selection or insertion sort?

25. 25 Simple Sorts  Selection Sort or Insertion Sort  Use when n is small  Simple code compensates for low efficiency!

26. 26 Quicksort  Efficient sorting algorithm  Discovered by C.A.R. Hoare  Example of Divide and Conquer algorithm  Two phases  Partition phase  Divides the work into half  Sort phase  Conquers the halves!

27. 27 Quicksort  Partition  Choose a pivot  Find the position for the pivot so that  all elements to the left are less  all elements to the right are greater < pivot> pivotpivot

28. 28 Quicksort  Conquer  Apply the same algorithm to each half < pivot> pivot pivot< p’ p’> p’< p”p”> p”

29. 29 QuickSort quick_sort(left, right) { Boundary Condition: when left >= right -- do nothing // Boundary Condition: when left >= right -- do nothing // Place first item of current sublist in its correct position, C, in the sublist itself, such that // List[i] <= List[C], for all i, First <= i < C // List[J] > List[C], for all J, C List[C], for all J, C < J <= Last quick_sort (left, C-1); quick_sort (C+1, right); }

30. 30 QuickSort code int partition(int array [], int left, int right) { int i = left, j = right; int i = left, j = right; int tmp; int tmp; int pivot = array[(left + right) / 2]; int pivot = array[(left + right) / 2]; while (i <= j) { while (array[i] < pivot) while (array[i] < pivot) i++; while (array[j] > pivot) while (array[j] > pivot) j--; if (i <= j) if (i <= j) { tmp = array[i]; tmp = array[i]; array[i] = arrar[j]; array[i] = arrar[j]; array[j] = tmp; array[j] = tmp; i++; j--; } }; return i; return i; }

31. 31 Quicksort  Implementation quicksort( int [] a, int low, int high ) { int pivot; /* Termination condition! */ if ( high > low ) { pivot = partition( a, low, high ); quicksort( a, low, pivot-1 ); quicksort( a, pivot+1, high ); } Divide Conquer

32. 32 QuickSort void quickSort(int []array, int left, int right) { int index = partition(array, left, right); // sorts the left side of the array if (left < index - 1) quickSort(array, left, index - 1); // sorts the right side of the array if (index < right) quickSort(array, index, right); }

33. 33 Quicksort - Partition int partition( int []a, int low, int high ) { int left, right; int pivot_item; pivot_item = a[low]; // pivot is first element in the array pivot = left = low; right = high; while ( left < right ) { /* Move left while item < pivot */ while( a[left] <= pivot_item ) left++; /* Move right while item > pivot */ while( a[right] >= pivot_item ) right--; if ( left < right ) SWAP(a,left,right); } /* right is final position for the pivot */ a[low] = a[right]; a[right] = pivot_item; return right; }

34. 34 Quicksort - Partition This example uses int ’s to keep things simple! 231215384218362927 lowhigh Any item will do as the pivot, choose the leftmost one!

35. 35 Quicksort - Partition Set left and right markers 231215384218362927 lowhigh pivot : 23 leftright

36. 36 Quicksort - Partition Move the markers until they cross over 231215384218362927 lowhigh pivot : 23 leftright

37. 37 Quicksort - Partition Move the left pointer while items <= pivot 231215384218362927 lowhigh pivot : 23 leftright Move right Items >=pivot

38. 38 Quicksort - Partition Swap the two items on the wrong side of the value of pivot 231215384218362927 lowhigh pivot : 23 leftright

39. 39 Quicksort - Partition left and right have swapped over, so stop 231215184238362927 lowhigh pivot : 23 leftright

40. 40 Quicksort - Partition Finally, swap the pivot and a[ right} 23121518 42 38362927 lowhigh pivot : 23 left right

41. 41 Quicksort - Partition Return the position of the pivot 181215234238362927 lowhigh pivot : 23 right

42. 42 Quicksort - Conquer pivot 181215234238362927 pivot : 23 Recursively sort left half Recursively sort right half

43. 43 Quicksort - Analysis  Partition  Check every item once O(n)  Conquer  Divide data in half O(log 2 n)  Total  Product O(n log n)  But there’s a catch …………….

44. 44 Quicksort - The truth!  What happens if we use quicksort on data that’s already sorted (or nearly sorted)  We’d certainly expect it to perform well!

45. 45 Quicksort - The truth!  Sorted data 123456789 pivot < pivot ? > pivot

46. 46 Quicksort - The truth!  Sorted data  Each partition produces  a problem of size 0  and one of size n-1 !  Number of partitions? 123456789 > pivot 23456789 pivot

47. 47 Quicksort - The truth!  Sorted data  Each partition produces  a problem of size 0  and one of size n-1 !  Number of partitions?  n each needing time O(n)  Total nO(n) or O(n 2 ) ? Quicksort is as bad as bubble or insertion sort 123456789 > pivot 23456789 pivot

48. 48 Quicksort - The truth!  Quicksort’s O(n log n) behaviour  Depends on the partitions being nearly equal  there are O( log n ) of them  On average, this will nearly be the case and quicksort is generally O(n log n)  Can we do anything to ensure O(n log n) time ?  In general, no  But we can improve our chances!!

49. 49 Quicksort - Choice of the pivot  Any pivot will work …  Choose a different pivot …  so that the partitions are equal  then we will see O(n log n) time 123456789 pivot < pivot > pivot

50. 50 Quicksort - Median-of-3 pivot  Take 3 positions and choose the median  say … First, middle, last  median is 5 ç perfect division of sorted data every time!  O(n log n) time ç Since sorted (or nearly sorted) data is common, median-of-3 is a good strategy especially if you think your data may be sorted! 123456789

51. 51 Quicksort - Random pivot  Choose a pivot randomly  Different position for every partition ç On average, sorted data is divided evenly  O(n log n) time Key requirement Pivot choice must take O(1) time

52. 52 Quicksort - Guaranteed O(n log n)?  Never!!  Any pivot selection strategy could lead to O(n 2 ) time Here median-of-3 chooses 2 è One partition of 1 and One partition of 7 Next it chooses 4 è One of 1 and One of 5 149625783 124965783

53. 53 Sorting- Key Points  Sorting  Bubble, Insertion, selection sort  O(n 2 ) sorts  Simple code  May run faster for small n, n ~10 (system dependent)  Quick Sort  Divide and conquer  O(n log n)

54. 54 Sorting - Key Points  Quick Sort  O(n log n) but ….  Can be O(n 2 )  Depends on pivot selection  Median-of-3  Random pivot  Better but not guaranteed

55. 55 Quicksort - Why bother?  Use Heapsort instead? Quicksort is generally faster Fewer comparisons and exchanges Some empirical data