Sorting and Searching "There's nothing in your head the sorting hat can't see. So try me on and I will tell you where you ought to be." -The Sorting Hat, Harry Potter and the Sorcerer's Stone

Sorting and Searching Fundamental problems in computer science and programming. Sorting done to make searching easier. Multiple algorithms to solve the same problem. How do we know which algorithm is "better"? Look at searching first. Examples will use arrays of ints to illustrate algorithms. CS 221 - Computer Science II

Searching

Searching Given a list of data, find the location of a particular value or report that value is not present. Linear Search Intuitive approach: Start at first item. Is it the one I am looking for? If not, go to next item. Repeat until found or all items checked. If items not sorted or unsortable, this approach is necessary. CS 221 - Computer Science II

Linear Search /* return the index of the first occurrence of target in list, or -1 if target not present in list */ public int linearSearch(int list[], int target) { int i = 0; while(i < list.length && list[i] != target) i++; if(i >= list.length) return -1; else return i; } CS 221 - Computer Science II

Question 1 What is the average case Big-O of linear search in an array with n items? O(n) O(n2) O(1) O(log(n)) O(n log(n)) CS 221 - Computer Science II

Question 1 What is the average case Big-O of linear search in an array with n items? O(n) O(n2) O(1) O(log(n)) O(n log(n)) CS 221 - Computer Science II

Searching in a Sorted List If items are sorted, we can divide and conquer. Divide your work in half with each step. Generally a good thing. Uses recursion. Binary Search: List in sorted in ascending order. Start at middle of list. Is that the item? Done. If not, is it less than item? Look in second half of list. is it greater than? Look in first half of list. Repeat until found, or sub-list size = 0. CS 221 - Computer Science II

Recursive Binary Search public static int search(int list[], int target) { return b-search(list, target, 0, list.length – 1); } public static int b-search(int list[], int target, int lo, int hi) { if( lo <= hi ){ int mid = (hi + lo) / 2; if( list[mid] == target ) return mid; else if( list[mid] > target ) return b-search(list, target, lo, mid – 1); else return b-search(list, target, mid + 1, hi); return -1; CS 221 - Computer Science II

Binary Search I Given target and sorted array list[], find index i such that list[i] = target, or report that no such index exists. Example 1: Search for 33. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search I 33 < 53, so look in lower half. 6 13 14 25 33 43 51 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search I 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search I 25 < 33, so look in upper half. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search I 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search I 43 < 33, so look in lower half. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search I 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search I 33 = 33, so found value. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi mid CS 221 - Computer Science II

Binary Search I Done. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi mid CS 221 - Computer Science II

Binary Search II Example 2: Search for 90. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search II 90 > 53, so look in upper half. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search II 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search II 90 < 93, so look in lower half. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search II 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search II 90 > 72, so look in upper half. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo mid hi CS 221 - Computer Science II

Binary Search II 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi CS 221 - Computer Science II

Binary Search II 90 > 84, so look in upper half? 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 lo hi mid CS 221 - Computer Science II

Binary Search II lo > hi, so no list left to search. Done. 6 13 14 25 33 43 51 53 64 72 84 93 95 96 97 1 2 3 4 5 6 7 8 9 10 11 12 13 14 hi lo CS 221 - Computer Science II

Question 2 What is the worst case Big O of binary search in an array with n items, if an item is present? O(n) O(n2) O(1) O(log(n)) O(n log(n)) CS 221 - Computer Science II

Question 2 What is the worst case Big O of binary search in an array with n items, if an item is present? O(n) O(n2) O(1) O(log(n)) O(n log(n)) CS 221 - Computer Science II

Other Searching Algorithms Interpolation Search More like what people really do. Binary Search Trees Hash Table Searching Best-First A* Interpolation search is an algorithm for searching for a given key in an indexed array that has been ordered by numerical values assigned to the keys (key values). It parallels how humans search through a telephone book for a particular name, the key value by which the book's entries are ordered. In each search step it calculates where in the remaining search space the sought item might be, based on the key values at the bounds of the search space and the value of the sought key, usually via a linear interpolation. The key value actually found at this estimated position is then compared to the key value being sought. If it is not equal, then depending on the comparison, the remaining search space is reduced to the part before or after the estimated position. This method will only work if calculations on the size of differences between key values are sensible. Best-first search is a search algorithm which explores a graph by expanding the most promising node chosen according to a specified rule. In computer science, A* (pronounced as "A star") is a computer algorithm that is widely used in pathfinding and graph traversal, the process of plotting an efficiently directed path between multiple points, called "nodes". It enjoys widespread use due to its performance and accuracy. CS 221 - Computer Science II

Sorting

Sorting Fun: Why Not Bubble Sort? CS 221 - Computer Science II

Sorting A fundamental application for computers. Makes finding data (searching) faster. Many different algorithms for sorting. The "simple" sorts run in quadratic time O(n2). Selection Sort Insertion Sort Bubble Sort CS 221 - Computer Science II

Selection Sort Given an array of length n, Search elements 0 through n-1, select smallest. Swap it with the element at location 0. Search elements 1 through n-1, select smallest. Swap it with the element at location 1. Search elements 2 through n-1, select smallest. Swap it with the element at location 2. Search elements 3 through n-1, select smallest. Swap it with the element at location 3. Continue in this fashion until there’s nothing left to search. CS 221 - Computer Science II

Selection Sort Algorithm public static void selectionSort(int list[]) { int min; int temp; for(int i = 0; i < list.length - 1; i++) { min = i; for(int j = i + 1; j < list.length; j++) { if( list[j] < list[min] ) min = j; } temp = list[i]; list[i] = list[min]; list[min] = temp; Big O? CS 221 - Computer Science II

Sorting an Array of Integers Example: An array of integers, sort from smallest to largest. The picture shows a graphical representation of an array which we will sort so that the smallest element ends up at the front, and the other elements increase to the largest at the end. The bar graph indicates the values which are in the array before sorting--for example the first element of the array contains the integer 45. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Repeatedly select the smallest element, and move this element to the front. The first sorting algorithm that we'll examine is called Selectionsort. It begins by going through the entire array and finding the smallest element. In this example, the smallest element is the number 8 at location [4] of the array. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Swap the smallest entry with the first entry. Once we have found the smallest element, that element is swapped with the first element of the array... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side Part of the array is now sorted. At this point, we can view the array as being split into two sides: To the left of the dotted line is the "sorted side", and to the right of the dotted line is the "unsorted side". Our goal is to push the dotted line forward, increasing the number of elements in the sorted side, until the entire array is sorted. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side Find the smallest element in the unsorted side. Each step of the Selectionsort works by finding the smallest element in the unsorted side. At this point, we would find the number 15 at location [5] in the unsorted side. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side Swap with the first element of unsorted side. This small element is swapped with the number at the front of the unsorted side, as shown here... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side We have increased the size of the sorted side by one element. ...and the effect is to increase the size of the sorted side by one element. As you can see, the sorted side always contains the smallest numbers, and those numbers are sorted from small to large. The unsorted side contains the rest of the numbers, and those numbers are in no particular order. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side The process continues... Smallest from unsorted Again, we find the smallest entry in the unsorted side... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side The process continues... Swap with front ...and swap this element with the front of the unsorted side. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side is bigger Sorted side Unsorted side The process continues... The sorted side now contains the three smallest elements of the array. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side Keep adding one more number to the sorted side. Here is the array after increasing the sorted side to four elements. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice Sorted side Unsorted side Stop when the unsorted side has just one number, since that number must be the largest number. And now the sorted side has five elements. In fact, once the unsorted side is down to a single element, the sort is completed. At this point the 5 smallest elements are in the sorted side, and so the the one largest element is left in the unsorted side. We are done... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Selection Sort in Practice The array is now sorted. ...The array is sorted. The basic algorithm is easy to state and also easy to program. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort Another of the O(n2) sorts: Start with first item, assume it’s sorted. Compare the second item to the first. If it’s smaller, swap. Compare the third item to the second. If smaller, swap. Compare again with first, if smaller swap again. And so forth… CS 221 - Computer Science II

Insertion Sort Code Big O? public void insertionSort(int list[]) { int temp, j; for(int i = 1; i < list.length; i++) { temp = list[i]; j = i; while( j > 0 && temp < list[j - 1]) { // swap elements list[j] = list[j - 1]; list[j - 1] = temp; j--; } Big O? CS 221 - Computer Science II

Insertion Sort in Practice Like Selection Sort, Insertion Sort algorithm views the array as having a sorted side and an unsorted side. Now we'll look at another sorting method called Insertionsort. The end result will be the same: The array will be sorted from smallest to largest. But the sorting method is different. However, there are some common features. As with the Selectionsort, the Insertionsort algorithm also views the array as having a sorted side and an unsorted side, ... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side The sorted side starts with just the first element, which is not necessarily the smallest element. ...like this. However, in the Selectionsort, the sorted side always contained the smallest elements of the array. In the Insertionsort, the sorted side will be sorted from small to large, but the elements in the sorted side will not necessarily be the smallest entries of the array. Because the sorted side does not need to have the smallest entries, we can start by placing one element in the sorted side--we don't need to worry about sorting just one element. But we do need to worry about how to increase the number of elements that are in the sorted side. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side The sorted side grows by taking the front element from the unsorted side... The basic approach is to take the front element from the unsorted side... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side ...and inserting it in its place in the sorted side. ...and insert this element at the correct spot of the sorted side. In this example, the front element of the unsorted side is 20. So the 20 must be inserted before the number 45 which is already in the sorted side. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side The sorted side contains values in order from smallest to largest, although they may not the smallest elements in the array. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side Sometimes we are lucky and the new inserted item doesn't need to move at all. Sometimes we are lucky and the newly inserted element is already in the right spot. This happens if the new element is larger than anything that's already in the array. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side Sometimes we’re lucky twice in a row. Sometimes we are lucky twice in a row. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Copy the new element to a separate location. Sorted side Unsorted side Copy the new element to a separate location. The actual insertion process requires a bit of work that is shown here. The first step of the insertion is to make a copy of the new element. Usually this copy is stored in a local variable. It just sits off to the side, ready for us to use whenever we need it. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Shift elements in the sorted side, creating an open space for the new element. After we have safely made a copy of the new element, we start shifting elements from the end of the sorted side. These elements are shifted rightward, to create an "empty spot" for our new element to be placed. In this example we take the last element of the sorted side and shift it rightward one spot... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Shift elements in the sorted side, creating an open space for the new element. ...like this. Is this the correct spot for the new element? No, because the new element is smaller than the next element in the sorted section. So we continue shifting elements rightward... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Continue shifting elements... This is still not the correct spot for our new element, so we shift again... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Continue shifting elements... ...and shift one more time... [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements ...until you reach the location for the new element. Finally, this is the correct location for the new element. In general there are two situations that indicate the "correct location" has been found: 1. We reach the front of the array (as happened here), or 2. We reached an element that is less than or equal to the new element. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Shifting Elements Sorted side Unsorted side Copy the new element back into the array, at the correct location. Once the correct spot is found, we copy the new element back into the array. The number of elements in the sorted side has increased by one. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Sorted side Unsorted side The last element must also be inserted. Start by copying it... The last element of the array also needs to be inserted. Start by copying it to a safe location. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Insertion Sort in Practice Done. The new element is inserted into the array. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort Start at the beginning of the list: Compare the first two elements. If the first is greater than the second, swap them. Compare second to the third. If the second is greater than the third, swap them. Continue doing this for each pair of adjacent elements to the end of the data set. Start again with the first two elements, repeating until no swaps have occurred on the last pass. CS 221 - Computer Science II

Bubble Sort Code Big O? public static void bubbleSort( int[] num ) {      int stop = num.length - 1; // marks end of unsorted part of list      boolean flag = true; // set flag to true for first pass      int temp;   //holding variable      while (flag) {             flag= false; //set flag to false for possible swap             for(int i=0;  i < stop;  i++) {                    if ( num[i] > num[i+1] )   { //swap elements                            temp = num[i];                                   num[i] = num[i+1];                            num[i+1] = temp;                           flag = true;   //shows a swap occurred                    }             } stop--;       } } Big O? CS 221 - Computer Science II

Bubble Sort in Practice The Bubble Sort algorithm looks at pairs of entries in the array, and swaps their order if needed. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Swap? [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Yes! [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Swap? [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice No. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Swap? [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice No. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Swap? [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Yes! [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Swap? [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Largest element in correct place. Yes! [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Repeat for first n–1 elements. Swap? No. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Swap? No. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Second largest element in correct place. Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Repeat for first n–2 elements. Swap? No. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Loop over array n-1 times, swapping pairs of entries as needed. Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Loop over array n-1 times, swapping pairs of entries as needed. Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Loop over array n-1 times, swapping pairs of entries as needed. Unsorted side Sorted side Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Third largest element in correct place. Loop over array n-1 times, swapping pairs of entries as needed. Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Bubble Sort in Practice Unsorted side Sorted side Continue looping, until done, or make pass through remaining list and no swaps. Swap? Yes. [0] [1] [2] [3] [4] [5] CS 221 - Computer Science II

Comparing Algorithms All simple sorting algorithms (Selection, Insertion, and Bubble) have a worst-case time of O(n2), making them impractical for large arrays. But they are easy to program, easy to debug, and work well on smaller data sets. Which algorithm do you think will be faster given random data? Selection Sort? Insertion Sort? Bubble Sort? Selection Sort is intermediate in speed. Insertion Sort is usually the fastest of the three--in fact, for small arrays (say, 10 or 15 elements), insertion sort is faster than more complicated sorting algorithms. Bubble Sort is very slow, but better for mostly sorted list. CS 221 - Computer Science II

Sub-Quadratic Sorting Algorithms Sub-Quadratic = Big O better than O(n2)

Divide and Conquer Very important algorithmic strategy in computer science: Divide problem into smaller parts. Independently solve the parts. Combine these solutions to get overall solution. Idea 1 : Partition array into items that are “small” and items that are “large”, then recursively sort the two sets  Quicksort Idea 2: Divide array into two halves, recursively sort left and right halves, then merge two halves  Mergesort CS 221 - Computer Science II

Quicksort Invented by C.A.R. (Tony) Hoare A Divide-and-Conquer approach that uses recursion: If the list has 0 or 1 elements, it’s sorted Otherwise, pick any element p in the list. This is called the pivot value Partition the list, minus the pivot, into two sub-lists: one of values less than the pivot and another of those greater than the pivot. equal values go to either. Return the Quicksort of the first list followed by the Quicksort of the second list. CS 221 - Computer Science II

public void quicksort(Comparable list[], int lo, int hi) { if(lo < hi) int p = partition(list, lo, hi); quicksort(list, lo, p - 1); quicksort(list, p + 1, hi); } public static void swap(Object a[], int index1, int index2) Object tmp = a[index1]; a[index1] = a[index2]; a[index2] = tmp; CS 221 - Computer Science II

public int partition(Comparable list[], int lo, int hi) { // pivot at start position int pivot = list[lo]; // keep track of last value <= pivot int i = lo; // move smaller values down in list for(int j = lo + 1; j <= hi) if(list[j] <= pivot) i++; swap(list[i], list[j]); } // put pivot in correct position in partitioned list swap(list[i], list[lo]); // index of pivot value return i; CS 221 - Computer Science II

Quicksort Tree Use binary tree to represent the execution of Quicksort Each node represents a recursive call of Quicksort Stores Unsorted sequence before the execution and its pivot. Sorted sequence at the end of the execution. The leaves are calls on sub-sequences of size 0 or 1 9 7 2 4 6  2 4 6 7 9 2 4  2 4 9 7  7 9 2  2 - - 9  9 CS 221 - Computer Science II

Example: Quicksort Execution Pivot selection – last value in list: 6 7 9 4 3 7 1 2 6 - CS 221 - Computer Science II

Example: Quicksort Execution Partition around 6 Recursive call on sub-list with smaller values 7 9 4 3 7 1 2 6 - 4 3 1 2 - 7 9 7 - CS 221 - Computer Science II

Example: Quicksort Execution Partition around 2 Recursive call on sub-list of smaller values 7 9 4 3 7 1 2 6 - 4 3 1 2 - 7 9 7 - 1 - 4 3 - CS 221 - Computer Science II

Example: Quicksort Execution Base case Return from sub-list of smaller values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3 - CS 221 - Computer Science II

Example: Quicksort Execution Recursive call on sub-list of larger values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3 - CS 221 - Computer Science II

Example: Quicksort Execution Partition around 3 Recursive call on sub-list of smaller values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3 - - 4 - CS 221 - Computer Science II

Example: Quicksort Execution Base case Return from sub-list of smaller values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3  3  4 - CS 221 - Computer Science II

Example: Quicksort Execution Recursive call on sub-list of larger values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3  3  4 - CS 221 - Computer Science II

Example: Quicksort Execution Base case Return from sub-list of larger values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 7 9 7 - 1  1 4 3  3 4  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Return from sub-list of larger values 7 9 4 3 7 1 2 6 - 4 3 1 2  1 2 3 4 7 9 7 - 1  1 4 3  3 4  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Return from sub-list of smaller values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7 - 1  1 4 3  3 4  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Recursive call on sub-list of larger values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7 - 1  1 4 3  3 4  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Partition around 7 Recursive call on sub-list of smaller values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7 - 1  1 4 3  3 4 7 - 9 - 3  3 3 CS 221 - Computer Science II

Example: Quicksort Execution Base case Return from sub-list of smaller values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7  7 7 1  1 4 3  3 4 7  7 9 -  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Recursive call on sub-list of larger values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7  7 7 1  1 4 3  3 4 7  7 9 -  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Base case Return from sub-list of larger values 7 9 4 3 7 1 2 6  1 2 3 4 6 4 3 1 2  1 2 3 4 7 9 7  7 7 9 1  1 4 3  3 4 7  7 9  9  4  4 CS 221 - Computer Science II

Example: Quicksort Execution Return from sub-list of larger values 7 9 4 3 7 1 2 6  1 2 3 4 6 7 7 9 4 3 1 2  1 2 3 4 7 9 7  7 7 9 1  1 4 3  3 4 7  7 9  9 3  3 CS 221 - Computer Science II

Example: Quicksort Execution Done 7 9 4 3 7 1 2 6  1 2 3 4 6 7 7 9 4 3 1 2  1 2 3 4 7 9 7  7 7 9 1  1 4 3  3 4 7  7 9  9  4  4 CS 221 - Computer Science II

Question 3 What is the Big-O of Quicksort? O(2n) O(n2) O(n log(n)) CS 221 - Computer Science II

Question 3 What is the Big-O of Quicksort? O(2n) O(n2) O(n log(n)) CS 221 - Computer Science II

Quicksort Analysis: Best Case What is best case running time? Assume keys are random, uniformly distributed. Recursion: Partition splits list in two sub-lists of size (n/2). Quicksort each sub-list. Depth of recursion tree? O(log n) Number of accesses in partition? O(n) Best case running time: O(n*log n) CS 221 - Computer Science II

Quicksort Analysis: Worst Case What is worst case running time? List already sorted. Recursion: Partition splits array in two sub-arrays: one sub-array of size 0. the other sub-array of size n-1. Quicksort each sub-array. Depth of recursion tree? O(n) Number of accesses per partition? O(n) Worst case running time: O(n2) CS 221 - Computer Science II

Average Case for Quicksort If the list is already sorted, Quicksort is terrible: O(n2) It is possible to construct other bad cases. However, Quicksort is usually O(n*log n) The constants are so good, Quicksort is generally the fastest known algorithm. Most real-world sorting is done by Quicksort. CS 221 - Computer Science II

Mergesort Algorithm Don Knuth cites John von Neumann as the creator of this algorithm: If a list has 1 element or 0 elements, it’s sorted. If a list has more than 1, split into two separate lists. Perform this algorithm on each of those smaller lists. Take the 2 sorted lists and merge them together. CS 221 - Computer Science II

Mergesort Code /** * perform a merge sort on the data in c * @param c c != null, all elements of c * are the same data type */ public static void sort(Comparable c[]) { Comparable temp[] = new Comparable[ c.length ]; mergeSort(c, temp, 0, c.length - 1); } private static void mergeSort(Comparable list[], Comparable temp[], int low, int high) { if( low < high){ int center = (low + high) / 2; mergeSort(list, temp, low, center); mergeSort(list, temp, center + 1, high); merge(list, temp, low, center + 1, high); CS 221 - Computer Science II

Merge Code CS 221 - Computer Science II private static void merge( Comparable list[], Comparable temp[], int leftPos, int rightPos, int rightEnd) { int leftEnd = rightPos - 1; int tempPos = leftPos; int numElements = rightEnd - leftPos + 1; //main loop while( leftPos <= leftEnd && rightPos <= rightEnd){ if( list[ leftPos ].compareTo(list[rightPos]) <= 0) temp[ tempPos ] = list[ leftPos ]; leftPos++; } else temp[ tempPos ] = list[ rightPos ]; rightPos++; tempPos++; //copy rest of left half while( leftPos <= leftEnd) //copy rest of right half while( rightPos <= rightEnd) //Copy temp back into list for(int i = 0; i < numElements; i++, rightEnd--) list[ rightEnd ] = temp[ rightEnd ]; CS 221 - Computer Science II

Merge-Sort Tree Use binary tree to represent the execution of Merge-sort Each node represents a recursive call of Merge-sort. Stores Unsorted sequence before the execution and its pivot. Sorted sequence at the end of the execution. The leaves are calls on sub-sequences of size 0 or 1. 7 2 9 4  2 4 7 9 7 2  2 7 9 4  4 9 7  7 2  2 9  9 4  4 CS 221 - Computer Science II

Example: Merge-sort Execution List to be sorted 5 9 4 3 7 1 2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 3 and 7. 5 9 4 3/7 1 2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 9 and 4. 5 9 4 3 7 1 2 6 - 5 9/4 3 - 7 1 2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9 - 4 3 - CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 5 and 9. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5/9 - 4 3 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9 - 4 3 - 5 - 9 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 5. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9 - 4 3 - 5  5 9 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9 - 4 3 - 5  5 9 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. Second sub-list returns 9. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9 - 4 3 - 5  5 9  9 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 5 and 9. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 CS 221 - Computer Science II

Example: Merge-sort Execution Return 59 from first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 4 and 3. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4/3 - 5  5 9  9 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 4 - 3 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 4. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 4  4 3 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 4  4 3 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 3. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3 - 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 4 and 3. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Return 34 from second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3 - 7 1 2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 59 and 34. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution First sub-list returns 3459. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 1 and 2. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1/2 6 - 5 9  5 9 4 3  3 4 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1 - 2 6 - 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 7 and 1. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7/1 - 2 6 - 5  5 9  9 4  4 3  3 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1 - 2 6 - 5  5 9  9 4  4 3  3 7 - 1 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 7. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1 - 2 6 - 5  5 9  9 4  4 3  3 7  7 1 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1 - 2 6 - 5  5 9  9 4  4 3  3 7  7 1 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 1. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1 - 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 7 and 1. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 CS 221 - Computer Science II

Example: Merge-sort Execution First sub-list returns 17. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 CS 221 - Computer Science II

Example: Merge-sort Execution Divide values in half between 2 and 6. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2/6 - 5  5 9  9 4  4 3  3 7  7 1  1 CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on first sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 2 - 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 2. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Recursive call on second sub-list. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6 - CS 221 - Computer Science II

Example: Merge-sort Execution Base case. First sub-list returns 6. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6 - 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 2 and 6. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Second sub-list returns 26. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6 - 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 17 and 26. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6  1 2 6 7 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Second sub-list returns 1267. 5 9 4 3 7 1 2 6 - 5 9 4 3  3 4 5 9 7 1 2 6  1 2 6 7 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Merge 3479 and 1267. 5 9 4 3 7 1 2 6  1 2 3 4 6 5 7 9 5 9 4 3  3 4 5 9 7 1 2 6  1 2 6 7 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Example: Merge-sort Execution Done. 5 9 4 3 7 1 2 6  1 2 3 4 6 5 7 9 5 9 4 3  3 4 5 9 7 1 2 6  1 2 6 7 5 9  5 9 4 3  3 4 7 1  1 7 2 6  2 6 5  5 9  9 4  4 3  3 7  7 1  1 2  2 6  6 CS 221 - Computer Science II

Merge Sort Runtime Analysis What is running time? No best, worst case. Runtime independent of data sorting. Recursion: Partition splits list in two sub-lists of size (n/2). Merge sort each sub-list. Depth of recursion tree? O(log n) Cost to merge? O(n) Running time: O(n*log n) CS 221 - Computer Science II

Analysis of Sorting Algorithms Best Case Average Case Worst Case Selection Sort O(n2) Insertion Sort O(n) Bubble Sort Quicksort O(n*log n) Merge Sort CS 221 - Computer Science II

Can We Do Better? Yes, sort of. CS 221 - Computer Science II

Time complexity of Sorting Of the several sorting algorithms discussed so far, the best ones: Merge sort and Quicksort (best one in practice): O(n*log(n)) on average, O(n2) worst case. Can we do better than O(n*log(n))? No. Proven: Any comparison-based sorting algorithm will need at least O(n*log(n)) operations. CS 221 - Computer Science II

Restrictions on Sorting Problem Suppose: Sorting values that repeat. But the values have a limit (digits from 0 to 9). Sorting should be easier. Is it possible to come up with an algorithm better than O(n*log(n))? Yes. Strategy will not involve comparisons. CS 221 - Computer Science II

Bucket Sort Suppose the values are in the range 0..m-1. Need: Start with m empty buckets numbered 0 to m-1. Scan the list: place element i in bucket i. Output the buckets in order. Need: An array of buckets. Values to be sorted will be the indexes to the buckets. No comparisons necessary. CS 221 - Computer Science II

Example 4 2 1 3 0 00 1 1 2 2 22 3 3 4 4 1 2 3 4 CS 221 - Computer Science II

Values versus Entries If sorting values, each bucket is just a counter that we increment whenever a value matching the bucket’s number is encountered. If we were sorting entries according to keys, then each bucket is a queue. Entries are enqueued into a matching bucket. Entries will be dequeued back into the array after the scan. CS 221 - Computer Science II

Bucket Sort Analysis Bucket initialization: O(m) From array to buckets: O(n) From buckets to array: O(n) This stage is a nested loop but only dequeue from each bucket until they are all empty. n dequeue operations in all. Strictly speaking, time complexity is O(n + m). But since m likely small compared to n, bucket sort is actually O(n). CS 221 - Computer Science II

Sorting Integers Can we perform Bucket Sort on any array of positive integers? Yes, but the number of buckets depends on the maximum integer value. If you are sorting 1000 integers and the maximum value is 999999, you will need 1 million buckets! Therefore, time complexity is not really O(n) because m is much > than n Actual time complexity is O(m). Can we do better? CS 221 - Computer Science II

Radix Sort Suppose: Repeatedly sort by digit. Perform multiple bucket sorts on integers starting with the rightmost digit. If maximum value is 999999, only ten buckets (not 1 million) will be necessary. Use this strategy when the keys are integers, and there is a reasonable limit on their values. Number of passes (bucket sort stages) will depend on the number of digits in the maximum value. CS 221 - Computer Science II

Example: first pass 12 58 37 64 52 36 99 63 18 9 20 88 47 20 12 52 63 64 36 37 47 99 9 58 1888 20 12 52 63 64 36 37 47 58 18 88 99 9 CS 221 - Computer Science II

Example: second pass 20 12 52 63 64 36 37 47 58 18 88 9 99 9 12 18 20 3637 47 52 58 63 64 99 88 9 12 18 20 36 37 47 52 58 63 64 88 99 CS 221 - Computer Science II

Radix Sort Analysis Given a fixed number of buckets and p sort stages (number of digits in maximum value), then radix sort is O(n). Each of the p bucket sort stages take O(n) time. Strictly speaking, time complexity is O(p*n), but p should be relatively small compared to n. Note: p=log10m, where m is the maximum value in the list. CS 221 - Computer Science II

About Radix Sort Since the buckets are reused at each stage, only 10 buckets are needed regardless of number of stages. Radix sort can apply to words: Set a limit to the number of letters in a word. Use 27 buckets (or more, depending on the letters/characters allowed) one for each letter plus a “blank” character. The word-length limit is exactly the number of bucket sort stages needed. CS 221 - Computer Science II

Radix Sort Summary Bucket sort and Radix sort are O(n) algorithms only because we have imposed restrictions on the input list to be sorted. Sorting, in general, takes O(n*log(n)) time CS 221 - Computer Science II

Final Comments Language libraries have sorting algorithms Hybrid sorts Java Arrays and Collections classes C++ Standard Template Library Hybrid sorts When size of unsorted list or portion of array is small: use insertion or selection sort Otherwise: use O(n log(n)) sort like Quicksort of Merge sort Many other special case sorting algorithms exist, like radix sort. CS 221 - Computer Science II

Comparison of Various Sorts Num Items Selection Insertion Quicksort 1000 16 5 2000 59 49 6 4000 271 175 8000 1056 686 16000 4203 2754 11 32000 16852 11039 45 64000 expected? 68 128000 158 256000 335 512000 722 1024000 1550 times in milliseconds CS 221 - Computer Science II

CS 221 - Computer Science II

Binary Search list Is middle item what we are looking for? low item middle item high item Is middle item what we are looking for? If not, is it more or less than the target? (Assume lower) list low middle high item item item and so forth… CS 221 - Computer Science II

Binary Search (Iterative) 2 3 5 7 11 13 17 19 23 29 31 37 41 47 43 53 public static int b-search(int list[], int target) { int result = -1; int low = 0; int high = list.length - 1; int mid; while( result == -1 && low <= high ) { mid = low + ((high - low) / 2); if( list[mid] == target ) result = mid; else if( list[mid] < target) low = mid + 1; else high = mid - 1; } return result; CS 221 - Computer Science II

Selection Sort Code public static void selectionSort(int list[]){ int min; int temp; for(int i = 0; i < list.length - 1; i++){ min = i; for(int j = i + 1; j < list.length; j++){ if( list[j] < list[min] ) min = j; temp = list[i]; list[i] = list[min]; list[min] = temp; } CS 221 - Computer Science II

Selection Sort in Practice 44 68 191 119 119 37 83 82 191 45 158 130 76 153 39 25 What is the actual number of statements executed of the selection sort code, given a list of n elements? What is the Big O? CS 221 - Computer Science II

Insertion Sort in Practice 44 68 191 119 119 37 83 82 191 45 158 130 76 153 39 25 What is the actual number of statements executed of the selection sort code, given a list of n elements? What is the Big O? CS 221 - Computer Science II

Example of Bubble Sort 7 2 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8 2 7 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8 2 7 8 5 4 2 5 7 4 8 2 4 5 7 8 Sorted! 2 7 5 8 4 2 5 4 7 8 2 7 5 4 8 CS 221 - Computer Science II

Attendance Question 3 Is selection sort always stable? Yes No CS 221 - Computer Science II

Attendance Question 4 Is the version of insertion sort shown always stable? Yes No CS 221 - Computer Science II

Quicksort in Action 39 23 17 90 33 72 46 79 11 52 64 5 71 Pick middle element as pivot: 46 Partition list: 23 17 5 33 39 11 46 79 72 52 64 90 71 - quicksort the less than list Pick middle element as pivot: 33 23 17 5 11 33 39 -quicksort the less than list, pivot now 5 {} 5 23 17 11 - quicksort the less than list, base case - quicksort the greater than list Pick middle element as pivot: 17 and so on…. CS 221 - Computer Science II

Quicksort on Another Data Set 44 68 191 119 119 37 83 82 191 45 158 130 76 153 39 25 CS 221 - Computer Science II

Mergesort When implementing, one temporary array is used instead of multiple temporary arrays. Why? CS 221 - Computer Science II

ShellSort Created by Donald Shell in 1959 Wanted to stop moving data small distances (in the case of insertion sort and bubble sort) and stop making swaps that are not helpful (in the case of selection sort) Start with sub arrays created by looking at data that is far apart and then reduce the gap size CS 221 - Computer Science II

ShellSort in practice 46 2 83 41 102 5 17 31 64 49 18 Gap of five. Sort sub array with 46, 5, and 18 5 2 83 41 102 18 17 31 64 49 46 Gap still five. Sort sub array with 2 and 17 5 2 83 41 102 18 17 31 64 49 46 Gap still five. Sort sub array with 83 and 31 5 2 31 41 102 18 17 83 64 49 46 Gap still five Sort sub array with 41 and 64 5 2 31 41 102 18 17 83 64 49 46 Gap still five. Sort sub array with 102 and 49 5 2 31 41 49 18 17 83 64 102 46 Continued on next slide: CS 221 - Computer Science II

Completed Shellsort 5 2 31 41 49 18 17 83 64 102 46 Gap now 2: Sort sub array with 5 31 49 17 64 46 5 2 17 41 31 18 46 83 49 102 64 Gap still 2: Sort sub array with 2 41 18 83 102 5 2 17 18 31 41 46 83 49 102 64 Gap of 1 (Insertion sort) 2 5 17 18 31 41 46 49 64 83 102 Array sorted CS 221 - Computer Science II

Shellsort on Another Data Set 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 44 68 191 119 119 37 83 82 191 45 158 130 76 153 39 25 Initial gap = length / 2 = 16 / 2 = 8 initial sub arrays indices: {0, 8}, {1, 9}, {2, 10}, {3, 11}, {4, 12}, {5, 13}, {6, 14}, {7, 15} next gap = 8 / 2 = 4 {0, 4, 8, 12}, {1, 5, 9, 13}, {2, 6, 10, 14}, {3, 7, 11, 15} next gap = 4 / 2 = 2 {0, 2, 4, 6, 8, 10, 12, 14}, {1, 3, 5, 7, 9, 11, 13, 15} final gap = 2 / 2 = 1 CS 221 - Computer Science II

ShellSort Code public static void shellsort(Comparable[] list) { Comparable temp; boolean swap; for(int gap = list.length / 2; gap > 0; gap /= 2) for(int i = gap; i < list.length; i++) { Comparable tmp = list[i]; int j = i; for( ; j >= gap && tmp.compareTo( list[j - gap] ) < 0; j -= gap ) list[ j ] = list[ j - gap ]; list[ j ] = tmp; } } CS 221 - Computer Science II

Bucket sort algorithm Algorithm BucketSort( S ) ( values in S are between 0 and m-1 ) for j  0 to m-1 do // initialize m buckets b[j]  0 for i  0 to n-1 do // place elements in their b[S[i]]  b[S[i]] + 1 // appropriate buckets i  0 for j  0 to m-1 do // place elements in buckets for r  1 to b[j] do // back in S S[i]  j i  i + 1 CS 221 - Computer Science II

Bucket sort algorithm Algorithm BucketSort( S ) ( S is an array of entries whose keys are between 0..m-1 ) for j  0 to m-1 do // initialize m buckets initialize queue b[j] for i  0 to n-1 do // place in buckets b[S[i].getKey()].enqueue( S[i] ); i  0 for j  0 to m-1 do // place elements in while not b[j].isEmpty() do // buckets back in S S[i]  b[j].dequeue() i  i + 1 CS 221 - Computer Science II

Stable Sorting A property of sorts If a sort guarantees the relative order of equal items stays the same then it is a stable sort [71, 6, 72, 5, 1, 2, 73, -5] subscripts added for clarity [-5, 1, 2, 5, 6, 71, 72, 73] result of stable sort Real world example: sort a table in Wikipedia by one criteria, then another sort by country, then by major wins CS 221 - Computer Science II

Example: 1st and 2nd passes 12 58 37 64 52 36 99 63 18 9 20 88 47 sort by rightmost digit 20 12 52 63 64 36 37 47 58 18 88 9 99 sort by leftmost digit 9 12 18 20 36 37 47 52 58 63 64 88 99 CS 221 - Computer Science II

Radix Sort and Stability Radix sort works as long as the bucket sort stages are stable sorts Stability is a property of sorts: A sort is stable if it guarantees the relative order of equal items stays the same Given these numbers: [71, 6, 72, 5, 1, 2, 73, -5] (subscripts added for clarity) A stable sort would be: [-5, 1, 2, 5, 6, 71, 72, 73] CS 221 - Computer Science II