COMP 103 SORTING Lindsay Groves 2016-T2 Lecture 26 Marcus Frean, Thomas Khuene, Lindsay Groves Lindsay Groves School of Engineering and Computer Science, Victoria University of Wellington 2016-T2 Lecture 26

RECAP-TODAY 2 RECAP Algorithms and costs for various set/bag/map implementations Binary Search, BST and heap have O(log n) cost Made possible by ordering properties TODAY Introduction to sorting Two algorithms for sorting a list: Selection Sort Insertion Sort Announcements:

Why Sort? May want to present information in different ways 3 May want to present information in different ways Eg list telephone subscribers by name, phone number or address. We can find items in an array much faster if they are sorted finding1-in-a-billion takes as long as the ArraySet takes to find1-in-30 But... constructing a sorted array one item at a time is very slow : O(n2) for 10 million items you need  50,000,000,000,000 steps at 10nS per step ⇒ 500,000 seconds = 58 days . There are sorting algorithms that are much faster if you sort the whole array at once: O(n log n) for 10 million items you need  250,000,000 steps ⇒ 2.5 seconds

Why study sorting algorithms? 4 Important practical problem – default utility isn’t always the best option. Illustrates: Important algorithm design strategies Interesting uses of data structures Algorithm analysis techniques Introduces important algorithms that every computer science or software engineering student should know

Ways of sorting Selection-based sorts: Insertion-based sorts: 5 Selection-based sorts: find the next largest/smallest item and put in place build the correct list in order incrementally Insertion-based sorts: for each item, insert it into an ordered sublist build a sorted list, but keep changing it Compare-and-Swap-based sorts: find two items that are out of order, and swap them keep “improving” the list

Ways to rate sorting algorithms 6 Efficiency What is the (worst-case) order of the algorithm? How does the algorithm deal with border cases? Requirements on Data Does the algorithm need random-access to data? Does it need anything more than “compare” and “swap”? Space Usage Can the algorithm sort in-place, or does it need extra space? Stability Will the algorithm ever reverse the order of “equal” items? Important when sorting records/objects with multiple fields; E.g. employee name, address, phone number, employee number, …

Selection-based Sorts 7 search for minimum here Selection Sort (slow) HeapSort (fast) 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

Insertion-based Sorts 8 Insertion Sort (slow) Shell Sort (pretty fast) Merge Sort (fast) (Divide and Conquer) 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

Compare and Swap Sorts Bubble Sort (easy but terrible performance) 9 Bubble Sort (easy but terrible performance) QuickSort (very fast) (Divide and Conquer) ...and so on... 1 2 3 4 5 6 7 8 9 10 11 things bubble up quickly, but bubble down slowly

Other Sorts 10 Radix Sort (only works with certain data types) (eg sort names on 1st character, then 2nd, …) Permutation Sort (very slow) Random Sort (Generate and Test) 1 2 3 4 5 6 7 8 9 10 11 Radix Sort requires data that can be mapped to integers as it sorts by grouping keys by the individual digits which share the same significant position and value

Coding it: some design decisions... 11 We could sort Lists ⇒ general and flexible but efficiency depends on how the List is implemented or just sort Arrays ⇒ less general efficiency is well-defined NB: method toArray() converts any Collection to an array May sort the actual array (or file), or an index into it We could require items to be Comparable i.e. any item can call compareTo(otherObj) ...on another ("natural ordering") OR provide a Comparator i.e. the sorter can call compare(obj1, obj2) ...on any two items. We will sort an Array, using a Comparator: public void …Sort(E[] data, int size, Comparator<E> comp) array number of items comparator

Selection Sort Find the smallest item and move it to position 0 12 Find the smallest item and move it to position 0 Find the next smallest item and move it to position 1 … What do we to with the item that was there? Move all the items before it up one (easy for linked list) Swap with the item to be moved cost (comparisons) = (n-1) + (n-2) + (n-3) + … + 3 + 2 + 1 = n(n-1)/2 = O(n2) = best, worst, and average - always does the same comparisons and swaps. 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

Selection Sort 13 public void selectionSort(E[ ] data, int size, Comparator<E> comp) { // for each position, from 0 up, find the next smallest item // and swap it into place for (int i=0; i<size-1; i++) { int minPos = i; for (int j=i+1; j<size; j++) if (comp.compare(data[j], data[minPos]) < 0) minPos=j; swap(data, i, minPos); } cost (comparisons) = (n-1) + (n-2) + (n-3) + … + 3 + 2 + 1 = n(n-1)/2 = O(n2) = best, worst, and average - always does the same comparisons and swaps. 1 2 3 4 5 6 7 8 9 10 11 1 2 3 4 5 6 7 8 9 10 11

Efficiency of Selection Sort 14 Cost: step? best case: what is it? cost: worst case: average case: all cases the same: have to scan all remaining items every time (n-1)+(n-2)+(n-3)+…1 = n(n-1)/2 = O(n2)

Selection Sort Analysis 15 Efficiency worst-case: O(n2) average case exactly the same. Requirements on Data Needs random-access data, but easy to modify for files Needs compare and swap Space Usage in-place Stability ?? Performance on Nearly Sorted Data No faster when given nearly sorted data

Insertion Sort 16 Place first item into the output list (bottom part of the array) Add next item to the output list so that it is still sorted. … How do we add each item to the output list? Find required position and move others up to make a space (easy with link list) Swap with the item to the left as long as it is larger Move items up one as long as they are larder, then insert the new item 1 2 3 4 5 6 7 8 9 10 11

Insertion Sort for (int i=1; i<size; i++) { E item = data[i]; 17 public void insertionSort(E[] data, int size, Comparator<E> comp) { // for each item, from 0, insert into place in the sorted region (0..i-1) for (int i=1; i<size; i++) { E item = data[i]; int place = i; while (place > 0 && comp.compare(item, data[place-1]) < 0) {  data[place] = data[place-1]; place--; } data[place]= item; 1 2 3 4 5 6 7 8 9 10 11

Efficiency of Insertion Sort 18 first: what is the step? best case: what is it? cost: worst case: average case: almost sorted cost: best case: already sorted: don’t have to move anything, just n-1 comparisons. O(n) worst case: reverse sorted: have to move everything down to the bottom: 1+2+3+4…n-1 = n(n-1)/2 = O(n2) Averace case: random order: have to move half way down on average: n(n-1)/4 = O(n2), but twice as fast as selection or bubble sort almost sorted case: only have to move a few items into place. or only move items a short way: O(n) ie, Insertion sort is very good when you know the list is almost sorted already.

Insertion Sort Analysis 19 Efficiency worst-case: O(n2) average case: half the cost of worst case. Requirements on Data Needs random-access data Needs compare and swap Space Usage in-place Stability ?? Performance on Nearly Sorted Data Very fast (O(n)) when given nearly sorted data

BubbleSort 20 Step through the array comparing successive pairs of items and swapping if they are out of order. Continue till no pairs are swapped. Can improve by: Recording where the last swap occurred Bubbling up and down in alternate passes (shaker sort)

Can we do better? 21 InsertionSort, SelectionSort and BubbleSort are all slow (apart from Insertion on nearly-sorted lists) Problems Selection Sort compares every pair of items, i.e., it ignores the results of previous comparisons. Insertion Sort (and Bubble Sort) only compares adjacent items only moves items one step at a time Can we do better? How? How much better?