1. Based on slides created by Ethan Apter & Marty Stepp
CSE 143 Lecture 16
Sorting & Merge Sort
Based on slides created by Ethan Apter & Marty Stepp

2. Sorting
sorting: Rearranging the values in an array or collection into a specific order (usually into their "natural ordering").
one of the fundamental problems in computer science
can be solved in many ways:
there are many sorting algorithms
some are faster/slower than others
some use more/less memory than others
some work better with specific kinds of data
some can utilize multiple computers / processors, ...
comparison-based sorting : determining order by comparing pairs of elements:
<, >, compareTo, …

3. Sorting algorithms
bubble sort: swap adjacent pairs that are out of order
selection sort: look for the smallest element, move to front
insertion sort: build an increasingly large sorted front portion
merge sort: recursively divide the array in half and sort it
heap sort: place the values into a sorted tree structure
quick sort: recursively partition array based on a middle value
other specialized sorting algorithms:
bucket sort: cluster elements into smaller groups, sort them
radix sort: sort integers by last digit, then 2nd to last, then ...
...

4. Selection sort
selection sort: Orders a list of values by repeatedly putting the smallest or largest unplaced value into its final position.
The algorithm:
Look through the list to find the smallest value.
Swap it so that it is at index 0.
Look through the list to find the second-smallest value.
Swap it so that it is at index 1.
...
Repeat until all values are in their proper places.

5. Selection sort example
Initial array:
After 1st, 2nd, and 3rd passes:
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value -4 18 22 27 30 36 50 68 91 56 85 42 98 25
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value -4 22 27 30 36 50 68 91 56 18 85 42 98 25
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value -4 22 27 30 36 50 68 91 56 18 85 42 98 25

6. Selection sort code
// Rearranges the elements of a into sorted order using
// the selection sort algorithm.
public static void selectionSort(int[] a) {
for (int i = 0; i < a.length - 1; i++) {
// find index of smallest remaining value
int min = i;
for (int j = i + 1; j < a.length; j++) {
if (a[j] < a[min]) {
min = j; }
// swap smallest value its proper place, a[i]
swap(a, i, min);

7. Bubble Sort
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25
bubble sort: Make repeated passes, swapping adjacent values
slower than selection sort (has to do more swaps)
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value 18 -4 22 27 30 36 50 68 56 85 42 91 25 98 22 50 91 98

8. Insertion Sort
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value 22 18 -4 27 30 36 50 68 91 56 85 42 98 25
insertion sort: Shift each element into a sorted sub-array
faster than selection sort (examines fewer values)
index 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16
value -4 18 22 27 30 36 50 68 91 56 85 42 98 25
sorted sub-array (indexes 0-7) 7

9. O(n2) Algorithms
Insertion sort: add values to a sorted list so as to maintain sorted order
the “sorted list” can originally be empty so that you can sort many values
similar to what we did in SortedIntList (HW#1)
Selection sort: scan an unsorted list for the smallest value, and move the smallest value to the front. Repeat this process for the remaining values until the whole list is sorted
Both insertion sort and selection sort are O(n2) algorithms

10. Merge Sort
Merge sort:
divide a list into two halves
sort the halves
recombine the sorted halves into a sorted whole
Merge sort is an example of a “divide and conquer” algorithm
divide and conquer algorithm: an algorithm that repeatedly divides the given problem into smaller pieces that can be solved more easily
it’s easier to sort the two small lists than the one big list

11. Merge Sort Picture 1 2 3 4 5 6 7 22 18 12 -4 58 31 42 split 22 18 121 2 3 4 5 6 7 22 18 12 -4 58 31 42
split 22 18 12 -4 58 7 31 42 22 18 12 -4 58 7 31 42 22 18 12 -4 58 7 31 42
merge 18 22 -4 12 7 58 31 42 -4 12 18 22 7 31 42 58 1 2 3 4 5 6 7 -4 12 18 22 31 42 58

12. mergeInto Merge sort: We’re going to write the following method:
divide a list into two halves
sort the halves
recombine the sorted halves into a sorted whole
We’re going to write the following method:
// post: copies the values from list1 and list2
// into result so that the values in result
// are in sorted order
private static void mergeInto(int[] result,
int[] list1,
int[] list2) {
... }
We’re going to write this part first

13. mergeInto
// post: copies the values from list1 and list2 // into result so that the values in result // are in sorted order private static void mergeInto(int[] result, int[] list1, int[] list2) {

14. mergeInto
So how do we actually merge the two sorted lists into one sorted result list?
One (wrong) way would just be to blindly copy the values from the two sorted lists into the result list and then sort the result list
this doesn’t take advantage of the fact that the two lists are already sorted
Instead, we’ll repeatedly select the smallest value from both sorted lists and put this value into the result list
because the two sorted lists are sorted, we know that their smallest values are found at the front

15. mergeInto
So to compare the smallest values, we’ll do something like this
if (list1[0] <= list2[0])
result[0] = list1[0];
else
result[0] = list2[0];
Obviously, this only handles the very first value
We need to use a loop and update our indexes in order to get this working correctly
But how many indexes do we need?

16. mergeInto Indexes We have three arrays, so we need three indexes
we need an index for list1, which tells us how many values from list1 we’ve copied so far
we need an index for list2, which tells us how many values from list2 we’ve copied so far
we need an index for result, which tells us how many values total we’ve copied from list1 and list2
we could just use the sum of the first two indexes for this index
So we have the following indexes:
int i1 = 0; // index for list1
int i2 = 0; // index for list2
int i = 0; // index for result (equals i1 + i2)

17. mergeInto
So now we’ll update our previous code to use a loop and our indexes
int i1 = 0;
int i2 = 0;
for (int i = 0; i < result.length; i++) {
if (list1[i1] <= list2[i2]) {
result[i] = list1[i1];
i1++;
} else {
result[i] = list2[i2];
i2++; }

18. mergeInto
So now we’ll update our previous code to use a loop and our indexes
int i1 = 0;
int i2 = 0;
for (int i = 0; i < result.length; i++) {
if (list1[i1] <= list2[i2]) {
result[i] = list1[i1];
i1++;
} else {
result[i] = list2[i2];
i2++; }
But this doesn’t quite work!

19. mergeInto
Right now, our code always compares a value from list1 with a value from list2
But, because we’re coping a single value into result per loop iteration, we’ll finish copying all the values from one of the sorted lists before the other
So we also need to check if we’ve copied all the values from a list
if we’ve already copied all the values from list1, copy the value from list2
if we’ve already copied all the values from list2, copy the value from list1

20. mergeInto Updated mergeInto code:
int i1 = 0;
int i2 = 0;
for (int i = 0; i < result.length; i++) {
if (i2 >= list2.length || (i1 < list1.length &&
list1[i1] <= list2[i2])) {
result[i] = list1[i1];
i1++;
} else {
result[i] = list2[i2];
i2++; }
This code is finished (it just needs to be put inside the method header)
This code is a little subtle. It relies on the short-circuiting property of && and ||

21. mergeInto Final Code
private static void mergeInto(int[] result, int[] list1, int[] list2) { int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (i2 >= list2.length || (i1 < list1.length && list1[i1] <= list2[i2])) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; }

22. mergeInto’s Preconditions
But notice our mergeInto method has some preconditions:
both list1 and list2 must already be sorted
result’s length must equal list1’s length plus list2’s length
Updated comment:
// pre: list1 and list2 are sorted
// result.length == list1.length + list2.length
// post: copies the values from list1 and list2
// into result so that the values in result
// are in sorted order

23. Merge Sort Recall that merge sort consists of the following steps:
divide a list into two halves
sort the halves
recombine the sorted halves into a sorted whole
We’ve written the “recombine” step (mergeInto)
But now we have to write the first two steps
Let’s define our public sort method:
// post: sorts the given array into non-decreasing order
public static void sort(int[] list) {
... }

24. Merge Sort // post: sorts the given array into non-decreasing order
public static void sort(int[] list) {
// Divide the list into two halves
// Sort the halves
// Recombine the sorted halves into a sorted whole

25. Dividing the List in Half
First we have to divide our list into two lists that are half the size of the original list
To do this, we need to create a place to store these two lists
we’ll create two new integer arrays
A first attempt at creating the two new integer arrays:
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length / 2];
But what if the length of our array is odd?
e.g. if list.length == 1001, then list.length / 2 == 500
and we’ll miss an element!
we need to handle the odd case

26. Dividing the List in Half
Here’s some better code:
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
And now we can also copy the first half of the list into list and the second half of the list into list2
Here’s the code for copying the values
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];

27. Merge Sort So here’s our code for sort so far:
public static void sort(int[] list) {
// Divide the list into two halves
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];
// Sort the halves
// Recombine the sorted halves into a sorted whole }
So, we’ve successfully divided the list in half, and we’ve written code that can merge two sorted lists
so we still have to sort the two halves

28. Sorting the Halves But how will we sort the two halves?
if only we had a method that could sort a list...
oh wait! That’s what we’re writing!
If we’re going to make our sort method recursive, we need to modify it to have a base case
what’s an easy list to sort?
the empty list (sorted by definition)
the list with one element (sorted by definition)
So if our sort is called on lists with size <= 1, we don’t have to do anything

29. Base Case Modified sort to have a base case:
public static void sort(int[] list) {
if (list.length > 1) {
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];
... }
Now we can just call sort on the two halves (list1 and list2), and we’ll eventually reach our base case

30. Sorting the Halves
Modified sort that calls itself to sort the two halves:
public static void sort(int[] list) {
if (list.length > 1) {
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];
sort(list1);
sort(list2);
... }
Finally, we just merge the two halves into the original list

31. sort Final Code Final version of sort:
public static void sort(int[] list) {
if (list.length > 1) {
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];
sort(list1);
sort(list2);
mergeInto(list, list1, list2); }

32. Merge Sort is Stable
Merge sort can be written so that it is a stable sort
stable sort: a sorting algorithm that maintains the relative positions of equal elements
So, if there are multiple 3s in a list, a stable sort will put
put the 3 that occurred first before the others
put the 3 that occurred second after the first and before the others
...
the 3 that occurred last after the others
And the way we’ve written merge sort makes it stable
because of how we divided our list into halves in sort
and because of how we break ties during the comparison of values in mergeInto

33. Analyzing Our Performance
Let’s compare the performance of our merge sort to Sun’s Arrays.sort
Here’s a sample run from IntArraySorter.java (on the website) run on a 2.53 GHz Intel Core 2 Duo Macbook
This program compares the performance of the
merge sort written in CSE 143 against the
performance of Sun's Arrays.sort method by
timing both methods on random, identical int
arrays of length
Sun's time: seconds
Our time: seconds
Ratio (Sun's/our): 2.24
On my desktop: Sun's time: seconds ÏϧÏOur time: seconds ÏϧÏRatio (Sun's/our): 2.53

34. Analyzing Our Performance
Sun’s sort is faster than ours
but only by a factor of about 2.5
That’s really good! It didn’t take us long to write this, and Sun’s Arrays.sort is a professional sorting method
Sun has had years to fine-tune the performance of Arrays.sort, and yet we wrote a reasonably competitive merge sort in less than an hour!
So what’s the complexity of merge sort?

35. sort Final Code Final version of sort:
public static void sort(int[] list) {
if (list.length > 1) {
int[] list1 = new int[list.length / 2];
int[] list2 = new int[list.length - list.length / 2];
for (int i = 0; i < list1.length; i++)
list1[i] = list[i];
for (int i = 0; i < list2.length; i++)
list2[i] = list[i + list1.length];
sort(list1);
sort(list2);
mergeInto(list, list1, list2); }

36. mergeInto Final Code
private static void mergeInto(int[] result, int[] list1, int[] list2) { int i1 = 0; int i2 = 0; for (int i = 0; i < result.length; i++) { if (i2 >= list2.length || (i1 < list1.length && list1[i1] <= list2[i2])) { result[i] = list1[i1]; i1++; } else { result[i] = list2[i2]; i2++; }

37. Complexity of Merge Sort
To determine the time complexity, let’s break our merge sort into pieces and analyze the pieces
Remember, merge sort consists of:
divide a list into two halves
sort the halves
recombine the sorted halves into a sorted whole
Dividing the list in half and recombining the lists are pretty easy to analyze:
both have O(n) time complexity
But what about sorting the halves?

38. Complexity of Merge Sort
We can think of merge sort as occurring in levels
at the first level, we want to sort the whole list
at the second level, we want to sort the two half lists
at the third level, we want to sort the four quarter lists
...
We know there’s O(n) work at each level from dividing/recombining the lists
But how many levels are there?
if we can figure this out, our time complexity is just O(n * num_levels)

39. Complexity of Merge Sort
Because we divide the array in half each time, there are log(n) levels
So merge sort is an O(n log(n)) algorithm
this is a big improvement over the O(n2) sorting algorithms
log(n) levels
O(n) work at each level

40. Complexity Classes Complexity Class Name Example O(1) constant time
popping a value off a stack
O(log n)
logarithmic time
binary search on an array
O(n)
linear time
scanning all elements of an array
O(n log n)
log-linear time
binary search on a linked list and good sorting algorithms
O(n2)
quadratic time
poor sorting algorithms (like inserting n items into SortedIntList)
O(n3)
cubic time
(example in lecture 11)
O(2n)
exponential time
Really hard problems. These grow so fast that they’re impractical

41. Examples of Each Complexity Class’s Growth Rate
From Reges/Stepp, page 708:
assume that all complexity classes can process an input of size 100 in 100ms
Input Size (n)
O(1)
O(log n)
O(n)
O(n log n)
O(n2)
O(n3)
O(2n)
100
100ms
200
115ms
200ms
240ms
400ms
800ms
32.7 sec
400
130ms
550ms
1.6 sec
6.4 sec
12.4 days
800
145ms
1.2 sec
51.2 sec
36.5 million years
1600
160ms
2.7 sec
25.6 sec
6 min 49.6 sec
4.21 * 1024 years
3200
175ms
3.2 sec
6 sec
1 min 42.4 sec
54 min 36 sec
5.6 * 1061 years