Based on slides created by Ethan Apter & Marty Stepp CSE 143 Lecture 16 Sorting & Merge Sort Based on slides created by Ethan Apter & Marty Stepp

Sorting sorting: Rearranging the values in an array or collection into a specific order (usually into their "natural ordering"). one of the fundamental problems in computer science can be solved in many ways: there are many sorting algorithms some are faster/slower than others some use more/less memory than others some work better with specific kinds of data some can utilize multiple computers / processors, ... comparison-based sorting : determining order by comparing pairs of elements: <, >, compareTo, …

Sorting algorithms bubble sort: swap adjacent pairs that are out of order selection sort: look for the smallest element, move to front insertion sort: build an increasingly large sorted front portion merge sort: recursively divide the array in half and sort it heap sort: place the values into a sorted tree structure quick sort: recursively partition array based on a middle value other specialized sorting algorithms: bucket sort: cluster elements into smaller groups, sort them radix sort: sort integers by last digit, then 2nd to last, then ... ...

Selection sort selection sort: Orders a list of values by repeatedly putting the smallest or largest unplaced value into its final position. The algorithm: Look through the list to find the smallest value. Swap it so that it is at index 0. Look through the list to find the second-smallest value. Swap it so that it is at index 1. ... Repeat until all values are in their proper places.

Selection sort example Initial array: After 1st, 2nd, and 3rd passes: index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25 index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value -4 18 22 27 30 36 50 68 91 56 85 42 98 25 index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value -4 22 27 30 36 50 68 91 56 18 85 42 98 25 index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value -4 22 27 30 36 50 68 91 56 18 85 42 98 25

Selection sort code // Rearranges the elements of a into sorted order using // the selection sort algorithm. public static void selectionSort(int[] a) { for (int i = 0; i < a.length - 1; i++) { // find index of smallest remaining value int min = i; for (int j = i + 1; j < a.length; j++) { if (a[j] < a[min]) { min = j; } // swap smallest value its proper place, a[i] swap(a, i, min);

Bubble Sort index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25 bubble sort: Make repeated passes, swapping adjacent values slower than selection sort (has to do more swaps) index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value 18 -4 22 27 30 36 50 68 56 85 42 91 25 98 22 50 91 98

Insertion Sort index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25 insertion sort: Shift each element into a sorted sub-array faster than selection sort (examines fewer values) index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 value -4 18 22 27 30 36 50 68 91 56 85 42 98 25 sorted sub-array (indexes 0-7) 7

O(n2) Algorithms Insertion sort: add values to a sorted list so as to maintain sorted order the “sorted list” can originally be empty so that you can sort many values similar to what we did in SortedIntList (HW#1) Selection sort: scan an unsorted list for the smallest value, and move the smallest value to the front. Repeat this process for the remaining values until the whole list is sorted Both insertion sort and selection sort are O(n2) algorithms

Merge Sort Merge sort: divide a list into two halves sort the halves recombine the sorted halves into a sorted whole Merge sort is an example of a “divide and conquer” algorithm divide and conquer algorithm: an algorithm that repeatedly divides the given problem into smaller pieces that can be solved more easily it’s easier to sort the two small lists than the one big list

Merge Sort Picture 1 2 3 4 5 6 7 22 18 12 -4 58 31 42 split 22 18 12 1 2 3 4 5 6 7 22 18 12 -4 58 31 42 split 22 18 12 -4 58 7 31 42 22 18 12 -4 58 7 31 42 22 18 12 -4 58 7 31 42 merge 18 22 -4 12 7 58 31 42 -4 12 18 22 7 31 42 58 1 2 3 4 5 6 7 -4 12 18 22 31 42 58

mergeInto Merge sort: We’re going to write the following method: divide a list into two halves sort the halves recombine the sorted halves into a sorted whole We’re going to write the following method: // post: copies the values from list1 and list2 // into result so that the values in result // are in sorted order private static void mergeInto(int[] result, int[] list1, int[] list2) { ... } We’re going to write this part first

mergeInto // post: copies the values from list1 and list2 // into result so that the values in result // are in sorted order private static void mergeInto(int[] result, int[] list1, int[] list2) {

mergeInto So how do we actually merge the two sorted lists into one sorted result list? One (wrong) way would just be to blindly copy the values from the two sorted lists into the result list and then sort the result list this doesn’t take advantage of the fact that the two lists are already sorted Instead, we’ll repeatedly select the smallest value from both sorted lists and put this value into the result list because the two sorted lists are sorted, we know that their smallest values are found at the front

mergeInto So to compare the smallest values, we’ll do something like this if (list1[0] <= list2[0]) result[0] = list1[0]; else result[0] = list2[0]; Obviously, this only handles the very first value We need to use a loop and update our indexes in order to get this working correctly But how many indexes do we need?

mergeInto Indexes We have three arrays, so we need three indexes we need an index for list1, which tells us how many values from list1 we’ve copied so far we need an index for list2, which tells us how many values from list2 we’ve copied so far we need an index for result, which tells us how many values total we’ve copied from list1 and list2 we could just use the sum of the first two indexes for this index So we have the following indexes: int i1 = 0; // index for list1 int i2 = 0; // index for list2 int i = 0; // index for result (equals i1 + i2)

mergeInto So now we’ll update our previous code to use a loop and our indexes int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (list1[i1] <= list2[i2]) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; }

mergeInto So now we’ll update our previous code to use a loop and our indexes int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (list1[i1] <= list2[i2]) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; } But this doesn’t quite work!

mergeInto Right now, our code always compares a value from list1 with a value from list2 But, because we’re coping a single value into result per loop iteration, we’ll finish copying all the values from one of the sorted lists before the other So we also need to check if we’ve copied all the values from a list if we’ve already copied all the values from list1, copy the value from list2 if we’ve already copied all the values from list2, copy the value from list1

mergeInto Updated mergeInto code: int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (i2 >= list2.length || (i1 < list1.length && list1[i1] <= list2[i2])) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; } This code is finished (it just needs to be put inside the method header) This code is a little subtle. It relies on the short-circuiting property of && and ||

mergeInto Final Code private static void mergeInto(int[] result, int[] list1, int[] list2) { int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (i2 >= list2.length || (i1 < list1.length && list1[i1] <= list2[i2])) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; }

mergeInto’s Preconditions But notice our mergeInto method has some preconditions: both list1 and list2 must already be sorted result’s length must equal list1’s length plus list2’s length Updated comment: // pre: list1 and list2 are sorted // result.length == list1.length + list2.length // post: copies the values from list1 and list2 // into result so that the values in result // are in sorted order

Merge Sort Recall that merge sort consists of the following steps: divide a list into two halves sort the halves recombine the sorted halves into a sorted whole We’ve written the “recombine” step (mergeInto) But now we have to write the first two steps Let’s define our public sort method: // post: sorts the given array into non-decreasing order public static void sort(int[] list) { ... }

Merge Sort // post: sorts the given array into non-decreasing order public static void sort(int[] list) { // Divide the list into two halves // Sort the halves // Recombine the sorted halves into a sorted whole

Dividing the List in Half First we have to divide our list into two lists that are half the size of the original list To do this, we need to create a place to store these two lists we’ll create two new integer arrays A first attempt at creating the two new integer arrays: int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length / 2]; But what if the length of our array is odd? e.g. if list.length == 1001, then list.length / 2 == 500 and we’ll miss an element! we need to handle the odd case

Dividing the List in Half Here’s some better code: int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; And now we can also copy the first half of the list into list1 and the second half of the list into list2 Here’s the code for copying the values for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length];

Merge Sort So here’s our code for sort so far: public static void sort(int[] list) { // Divide the list into two halves int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length]; // Sort the halves // Recombine the sorted halves into a sorted whole } So, we’ve successfully divided the list in half, and we’ve written code that can merge two sorted lists so we still have to sort the two halves

Sorting the Halves But how will we sort the two halves? if only we had a method that could sort a list... oh wait! That’s what we’re writing! If we’re going to make our sort method recursive, we need to modify it to have a base case what’s an easy list to sort? the empty list (sorted by definition) the list with one element (sorted by definition) So if our sort is called on lists with size <= 1, we don’t have to do anything

Base Case Modified sort to have a base case: public static void sort(int[] list) { if (list.length > 1) { int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length]; ... } Now we can just call sort on the two halves (list1 and list2), and we’ll eventually reach our base case

Sorting the Halves Modified sort that calls itself to sort the two halves: public static void sort(int[] list) { if (list.length > 1) { int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length]; sort(list1); sort(list2); ... } Finally, we just merge the two halves into the original list

sort Final Code Final version of sort: public static void sort(int[] list) { if (list.length > 1) { int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length]; sort(list1); sort(list2); mergeInto(list, list1, list2); }

Merge Sort is Stable Merge sort can be written so that it is a stable sort stable sort: a sorting algorithm that maintains the relative positions of equal elements So, if there are multiple 3s in a list, a stable sort will put put the 3 that occurred first before the others put the 3 that occurred second after the first and before the others ... the 3 that occurred last after the others And the way we’ve written merge sort makes it stable because of how we divided our list into halves in sort and because of how we break ties during the comparison of values in mergeInto

Analyzing Our Performance Let’s compare the performance of our merge sort to Sun’s Arrays.sort Here’s a sample run from IntArraySorter.java (on the website) run on a 2.53 GHz Intel Core 2 Duo Macbook This program compares the performance of the merge sort written in CSE 143 against the performance of Sun's Arrays.sort method by timing both methods on random, identical int arrays of length 2500000 Sun's time: 0.507 seconds Our time: 1.135 seconds Ratio (Sun's/our): 2.24 On my desktop: Sun's time: 1.332 seconds ÏϧÏOur time: 3.365 seconds ÏϧÏRatio (Sun's/our): 2.53

Analyzing Our Performance Sun’s sort is faster than ours but only by a factor of about 2.5 That’s really good! It didn’t take us long to write this, and Sun’s Arrays.sort is a professional sorting method Sun has had years to fine-tune the performance of Arrays.sort, and yet we wrote a reasonably competitive merge sort in less than an hour! So what’s the complexity of merge sort?

sort Final Code Final version of sort: public static void sort(int[] list) { if (list.length > 1) { int[] list1 = new int[list.length / 2]; int[] list2 = new int[list.length - list.length / 2]; for (int i = 0; i < list1.length; i++) list1[i] = list[i]; for (int i = 0; i < list2.length; i++) list2[i] = list[i + list1.length]; sort(list1); sort(list2); mergeInto(list, list1, list2); }

mergeInto Final Code private static void mergeInto(int[] result, int[] list1, int[] list2) { int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (i2 >= list2.length || (i1 < list1.length && list1[i1] <= list2[i2])) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; }

Complexity of Merge Sort To determine the time complexity, let’s break our merge sort into pieces and analyze the pieces Remember, merge sort consists of: divide a list into two halves sort the halves recombine the sorted halves into a sorted whole Dividing the list in half and recombining the lists are pretty easy to analyze: both have O(n) time complexity But what about sorting the halves?

Complexity of Merge Sort We can think of merge sort as occurring in levels at the first level, we want to sort the whole list at the second level, we want to sort the two half lists at the third level, we want to sort the four quarter lists ... We know there’s O(n) work at each level from dividing/recombining the lists But how many levels are there? if we can figure this out, our time complexity is just O(n * num_levels)

Complexity of Merge Sort Because we divide the array in half each time, there are log(n) levels So merge sort is an O(n log(n)) algorithm this is a big improvement over the O(n2) sorting algorithms log(n) levels O(n) work at each level

Complexity Classes Complexity Class Name Example O(1) constant time popping a value off a stack O(log n) logarithmic time binary search on an array O(n) linear time scanning all elements of an array O(n log n) log-linear time binary search on a linked list and good sorting algorithms O(n2) quadratic time poor sorting algorithms (like inserting n items into SortedIntList) O(n3) cubic time (example in lecture 11) O(2n) exponential time Really hard problems. These grow so fast that they’re impractical

Examples of Each Complexity Class’s Growth Rate From Reges/Stepp, page 708: assume that all complexity classes can process an input of size 100 in 100ms Input Size (n) O(1) O(log n) O(n) O(n log n) O(n2) O(n3) O(2n) 100 100ms 200 115ms 200ms 240ms 400ms 800ms 32.7 sec 400 130ms 550ms 1.6 sec 6.4 sec 12.4 days 800 145ms 1.2 sec 51.2 sec 36.5 million years 1600 160ms 2.7 sec 25.6 sec 6 min 49.6 sec 4.21 * 1024 years 3200 175ms 3.2 sec 6 sec 1 min 42.4 sec 54 min 36 sec 5.6 * 1061 years