1 Sorting

2 Fundamental problem in computing science  putting a collection of items in order Often used as part of another algorithm  e.g. sort a list, then do many binary searches  e.g. looking for identical items in an array: unsorted: do O(n 2 ) comparisons sort O(??), then do O(n) comparisons There is a sort function in java.util.Arrays

3 Sorting Example 12, 2, 23, -3, 21, 14 Easy…. but think about a systematic approach….

4 Sorting Example 4, 3, 5435, 23, -324, 432, 23, 22, 29, 11, 31, 21, 21, 17, -5, -79, -19, 312, 213, 432, 321, 11, 1243, 12, 15, 1, -1, 214, 342, 76, 78, 765, 756, -465, -2, 453, 534, 45265, 65, 23, 89, 87684, 2, 234, 6657, 7, 65, -42,432, 876, 97, 0, -11, -65, -87, 645, 74, 645 How well does your intuition generalize to big examples?

5 Sorting There are many algorithms for sorting Each has different properties:  easy/hard to understand  fast/slow for large lists  fast/slow for short lists  fast in all cases/on average

6 Selection Sort Find the smallest item in the list Switch it with the first position Find the next smallest item Switch it with the second position Repeat until you reach the last element

7 Selection Sort: Example Original list: Smallest is 8: Smallest is 14: Smallest is 17: Smallest is 23: DONE! Animated Illustration

8 Selection Search: Running Time Scan entire list (n steps) Scan rest of the list (n-1 steps)…. Total steps: n + (n -1) + (n-2) + … + 1 = n(n+1)/2 [Gauss] = n 2 /2 +n/2 So, selection sort is O(n 2 )

9 Swapping One key feature in the implementation of selection sort: swapping two values Requires three assignment statements temp = first;\\ don’t forget first first = second;\\ give first the swap value second = temp;\\ now retrieve stored value Full implementation in text (a bit involved with Comparable discussion)

10 Selection Sort in Java public void selectionSort(int[] arr) { int i,j,min,temp; for(j=0; j < arr.length-1; j++) { min=j; for(i=j+1; i < arr.length; i++) if(arr[i] < arr[min]) min=i; if(j!=min) { temp=arr[j]; arr[j]=arr[min]; arr[min]=temp; }

11 Insertion Sort Consider the first item to be a sorted sublist (of one item) Insert the second item into the sublist, shifting the first item as needed to make space Insert item into the sorted sublist (of two items) shifting as needed Repeat until all items have been inserted

12 Insertion Sort: Example Original list: Start with sorted list {8}: Insert 17, nothing to shift: Insert 2, shift the rest: Insert 14, shift {17,23}: Insert 75, nothing to shift: Animated Illustration

13 Insertion Sort: Pseudocode for pos from 1 to (n-1): val = array[pos] // get element pos into place i = pos – 1 while i >= 0 and array[i] > val: array[i+1] = array[i] i-- array[i+1] = val

14 Insertion Sort: Running Time Requires n-1 passes through the array  pass i must compare and shift up to i elements  total maximum number of comparisons/moves: … + n = n(n-1)/2 So insertion sort is also O(n 2 )  not bad… but there are faster algorithms  it turns out insertion sort is generally faster than other algorithms for “small” arrays (n<10??)

15 Insertion Sort in Java public void insertionSort(int[] arr) { int i,k,temp; for(k=1; k < arr.length; k++) { temp=arr[k]; i=k; while(i > 0 && temp < arr[i-1]) { arr[i]=arr[i-1]; i--; } arr[i]=temp; }

16 Selection and Insertion Same running time  Choice between is largely up to you Think about combining search/sort  Sorting list is O(n 2 )  Then binary search O(log n)  But just doing linear search is faster O(n)  Sorting first doesn’t pay off with these algorithms

17 Merge Sort: Basics Merge sort is (yet another) sorting algorithm Usually defined recursively  The recursive calls work on smaller and smaller parts of the list Idea:  Split list into two halves[looks familiar]  Recursively sort each half  “merge” the two halves into a single list

18 Merge Sort: Example Split: Recursively sort: Original list: Merge?

19 Example merge Must put the next-smallest element into the merged list at each point Each next-smallest could come from either list Animated Illustration

20 Remarks Note that merging just requires checking the smallest element of each half  Just one comparison step  This is where the recursive call saves time We need to give a formal specification of the algorithm to really check running time

21 Merge Sort Algorithm mergeSort(array,first,last): // sort array[first] to array[last-1] if last – first ≤ 1: return // length 0 or 1 already sorted mid = (first + last)/2 mergeSort(array,first,mid) //recursive call 1 mergeSort(array,mid,last) //recursive call 2 merge(array,first,mid,last)

22 Merge Algorithm (incorrect) merge(array,first,mid,last): //merge array[first to mid-1] and array[mid to last -1] leftpos = first rightpos = mid for newpos from 0 to last-first: if array[leftpos] ≤ array[rightpos]: newarray[newpos] = array[leftpos] leftpos++ else: newarray[newpos] = array[rightpos] rightpos++ copy newarray to array[first to (last-1)]

23 Problem? The algorithm starts correctly, but has an error as it finishes  Eventually one of the halves will empty Then the “if” will compare against ??  the element past one of the halves… one of: ?? ?? 52 leftposrightpos leftposrightpos

24 Solution Must prevent this: we can only look at the correct parts of the array So: compare only until we reach the end of one half Then, just copy the rest over

25 Merge Algorithm (1) merge(array,first,mid,last): //merge array[first to mid-1] and array[mid to last -1] leftpos = first rightpos = mid newpos = 0 while leftpos<mid and rightpos≤last-1: if array[leftpos] ≤ array[rightpos]: newarray[newpos] = array[leftpos] leftpos++; newpos++ else: newarray[newpos] = array[rightpos] rightpos++; newpos++ (continues)

26 Merge Algorithm (2) merge(array,first,mid,last): … //code from last slide while leftpos<mid: // copy the rest left half (if any) newarray[newpos] = array[leftpos] leftpos++; newpos++ while rightpos≤last-1// copy the rest of right half (if any) newarray[newpos] = array[rightpos] rightpos++; newpos++ copy newarray to array[first to (last-1)]

27 Example array: newarray: leftpos: rightpos: newpos: merge(array,3,7,11): compare loop fill

28 Example array: newarray: leftpos: rightpos: newpos: merge(array,8,12,16): compare loop fill

29 Running Time What is the running time for merge sort? recursive calls * work per call?  yes, but overly simplified  work per call changes We know: the merge algorithm takes O(n) steps to merge a total of n elements

30 Merge Sort Recursive Calls Each level has a total of n elements n elements n/2 n/

31 Merge Sort: Running Time Steps to merge each level: O(n) Number of levels: O(log n) Total time: O(n log n)  Much faster than selection/insertion which both take O(n 2 ) In general, no sorting algorithm can do better than O(n log n)  There are some algorithms that are faster for limited cases

32 In Place Sorting Merging requires extra storage  an extra array with n elements (can be re-used by all merges)  insertion sort only requires a few storage variables An algorithm that uses at most O(1) extra storage is called “in-place”  insertion sort/selection sort are both in-place  merge sort is not

33 Stable Sorting In merge sort, equal elements are kept in order  think of sorting a spreadsheet with other columns  rows with equal values in the sort column stay in order A stable sorting algorithm has this property  insertion sort and merge sort have this property  selection sort does not