Sorting Algorithms Written by J.J. Shepherd

Sorting Review For each one of these sorting problems we are assuming ascending order so smallest to largest Two You’ve Seen Before – Selection – Bubble

Selection Sort Scans through the data structure and finds the smallest element then swaps that element with the first element Then it looks for the next smallest and does the same This is repeated until the end of the data structure is reached

Selection Sort Look for the smallest element in the array since the smallest value goes first index value

Selection Sort The first value is assumed to be the smallest index value

Selection Sort The next value is examine and it is smaller than the first index, so that’s assumed to be the smallest value. Store that index index value

Selection Sort This value is larger, so keep going index value

Selection Sort This value is larger, so keep going index value

Selection Sort This value is smaller, so store this index index value

Selection Sort This value is larger, so move on index value

Selection Sort This value is larger, so move on index value

Selection Sort This value is smaller, so save this index index value

Selection Sort This value is larger, so move on index value

Selection Sort This value is larger, so move on index value

Selection Sort Now we’ve reached the end so we swap the stored smallest value with the value at the first index index value

Selection Sort That index is complete so we never test it again We move on finding the next smallest value index value

Selection Sort It starts on the next index index value

Selection Sort After a while we discover that this is the next smallest value index value

Selection Sort Swap these values index value

Selection Sort Start the process again for the next smallest value index value

Selection Sort Eventually this is the result index value

Selection Sort Theoretically how long does this take in the worst case scenario? Again let’s remember Big O One function (f(x)) is bounded by another (g(x)) given some large constant (M)

Selection Sort Let’s assume the data structure has n elements in there. Then how many times will this iteration run?

Selection Sort Search for the smallest element = n Search for the next smallest = n-1 Search for the next smallest = n-2 … The final element = 1

Selection Sort If we add all of these searches together we can say it roughly takes n 2 times to sort every element. Thus O(n 2 )

Bubble Sort The idea is you keep swapping values which are out of order until no more swaps are made The idea is the largest values “bubble up” to the top of the data structure

Bubble Sort Examine the two side-by-side elements if the right one is larger than the left one swap index value

Bubble Sort Left is larger than right. SWAP index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right SWAP! index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right. SWAP! index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right. SWAP! index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right. SWAP! index value

Bubble Sort Move forward index value

Bubble Sort Left is less than right so it is sorted. Move forward index value

Bubble Sort Left is larger than right. SWAP! index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right. SWAP index value

Bubble Sort Move forward index value

Bubble Sort Left is larger than right. SWAP! index value

Bubble Sort We’ve reached the end but since there was at least one swap the process has to start all over again from the beginning index value

Bubble Sort Theoretically how long does Bubble Sort run in the worst case scenario? What is the worst case scenario for bubble sort?

Bubble Sort The worst case scenario is we are given a data structure of n values that are sorted… Backwards Let’s examine the swaps involved with this case.

Bubble Sort The first iteration takes n swaps The next takes n-1 swaps The next takes n-2 swaps … Finally 0 swaps

Bubble Sort If we add all of these swaps together we can say it roughly takes n 2 times to sort every element. Thus O(n 2 )

Merge Sort A divide and conquer algorithm that splits apart a data structure in half over and over again and then finally merges the elements together piece by piece Similar concept to binary search but applied to sorting

Merge Sort Split the structure in half until single elements remain index value

Merge Sort Split the structure in half until single elements remain

Merge Sort Split the structure in half until single elements remain

Merge Sort Split the structure in half until single elements remain

Merge Sort Finally we have single elements so we can start merging

Merge Sort It’s sort of hard to see how merging works in the first step as it’s just one comparison

Merge Sort The idea of merging is for each smaller data structure we assume they have been sorted in the previous step. In this way we do not need to resort those data structure only sort them versus the others

Merge Sort Now we continue to merge

Merge Sort Check the first two values. The smaller one is added to the new data structure and its index is moved forward. The other remains the same

Merge Sort Check the indexed values. The smaller one is added to the new data structure and its index is moved forward. The other remains the same

Merge Sort Check the indexed values. The smaller one is added to the new data structure and its index is moved forward. The other remains the same

Merge Sort The second data structure reached its end so the rest of the first data structure is simply added to the end

Merge Sort The second data structure reached its end so the rest of the first data structure is simply added to the end

Merge Sort Similarly let’s look at the next merge

Merge Sort Similarly let’s look at the next merge

Merge Sort Similarly let’s look at the next merge

Merge Sort Similarly let’s look at the next merge

Merge Sort Similarly let’s look at the next merge

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort And the next one

Merge Sort Finally index value

Merge Sort Theoretically how long does merge sort take? There are essentially two steps that work in conjunction with each other – Dividing the structure – Merging it back together

Merge Sort We can actually visualize how long it takes n n/2 n/ … 1 1

Merge Sort Dividing the structure takes lg(n) time n n/2 n/ … 1 1 lg(n)

Merge Sort Merging takes n time n n/2 n/ … 1 1 lg(n) n

Merge Sort If combine the dividing with the merging parts we finally get that it takes n*lg(n) time Thus O(nlgn)

Was All of This Really Worth it?

Common Big O Complexities

Quick Sort Look for the first element to the left that is larger than the pivot index value i j

Quick Sort Look for the first element to the left that is larger than the pivot index value i j

Quick Sort Look for the first element to the right of the pivot that’s less than the pivot index value i j

Quick Sort Look for the first element to the right of the pivot that’s less than the pivot index value i j

Quick Sort Swap those elements! index value j i

Quick Sort Repeat that process. Look for one that’s greater than on the left side index value i j

Quick Sort Look for one that is less than on the right side index value ji

Quick Sort Swap! index value ij

Quick Sort Continue on index value ij

Quick Sort Continue on index value ij

Quick Sort Continue on index value ij

Quick Sort Swap! index value i

Quick Sort Now since i = j we need to split the data structure and put the pivot in the center index value ij

Quick Sort Now we repeat the same process for the smaller structures

Quick Sort How long does this take theoretically? What is its worst case scenario?

Quick Sort Strangely enough its worst case scenario is an already sorted array. In this one unique case the pivot is selected every time and is swapped in and out of places n times for a data structure of size in so technically it is O(n 2 )

Quick Sort However since this is a rare case, and assuming the pivot is randomly chosen and not fixed then the average case becomes  (nlgn)

Wrapping up asymptotics Big O (O) – is the worst case Big Omega (  ) – is the best case scenario Bit Theta (  ) – is the average case scenario

Formal Definitions Big O for f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

Formal Definitions Big O for f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

Formal Definitions Big Omega for f(n) = Ω (g(n)) means there are positive constants c and k, such that 0 ≤ cg(n) ≤ f(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

Formal Definitions Big Theta for f(n) = Θ (g(n)) means there are positive constants c 1, c 2, and k, such that 0 ≤ c 1 g(n) ≤ f(n) ≤ c 2 g(n) for all n ≥ k. The values of c 1, c 2, and k must be fixed for the function f and must not depend on n. IE in between Big O and Big Omega