Quicksort This is probably the most popular sorting algorithm. It was invented by the English Scientist C.A.R. Hoare It is popular because it works well in a variety of situations – it is a good general purpose algorithm

Quicksort The desirable features are It is in-place (uses only a small stack as auxiliary structure – if recursion is used the stack is implicit) It works in O(n log n) in the average case Drawbacks It is recursive – implementation is complicated without recursion it takes O(n 2 ) in the worst case It is fragile: a simple mistake in the implementation can go unnoticed and can cause it to perform badly

A Look at divide-and-conquer Approach Several recursive algorithms work by using recursive calls to sub-problems that are similar to the original but smaller in size Solve the sub-problems recursively And then combine the solution into a solution for the larger instance

A Look at divide-and-conquer Approach Divide-and-Conquer approach has three main steps Divide the problem into a number of smaller sub- problems Conquer the sub-problems either by calling the recursion for them or solving them with a straightforward approach if they are small enough Combine the solutions of the sub-problems into the solution for the large (original) problems

The Quicksort Idea (Employing Divide-and-Conquer) The basic idea of Quicksort published in 1962 is as follows: 1. Pick one item from the list and call it pivot 2. Partition the items in the list using the pivot. 3. Reorganize the list so that all the elements that are smaller than the pivot are in the left partition and all elements are greater than the pivot are in the right partition. 4. Use recursion to sort the partitions 1. Pick one item from the list and call it pivot 2. Partition the items in the list using the pivot. 3. Reorganize the list so that all the elements that are smaller than the pivot are in the left partition and all elements are greater than the pivot are in the right partition. 4. Use recursion to sort the partitions

Partitioning (Quicksort II) A key step in the Quicksort algorithm is partitioning the array We choose some (any) number p in the array to use as a pivot We partition the array into three parts: p numbers less than p numbers greater than or equal to p p

The Partition of the List The partition is the crucial part of Quicksort and it must rearrange the list to make the following conditions true after its execution 1. The element chosen as pivot, say list[i], is in its final place in the list 2. All the elements in list[first],...,list[i-1] are less than or equal to list[i] 3. All the elements in list[i+1],...,list[last] are greater than or equal to list[i] 1. The element chosen as pivot, say list[i], is in its final place in the list 2. All the elements in list[first],...,list[i-1] are less than or equal to list[i] 3. All the elements in list[i+1],...,list[last] are greater than or equal to list[i]

Quicksort Two phases Partition phase Divides the work into half Sort phase Conquers the halves!

Quicksort Partition Choose a pivot Find the position for the pivot so that all elements to the left are less all elements to the right are greater < pivot > pivot pivot

Quicksort Conquer Apply the same algorithm to each half < pivot> pivot pivot< p’ p’> p’< p”p”> p”

Defining partition() After calling the function quicksort(list, first, last) we expect that the list should look like something as above The split will be assigned the value of the index of the pivot used in the partition function partition 1: all items <= pivot partition 2: all items > pivot first split last

Building the partitions In the beginning we don't know which elements should be in partition 1 or 2. Our partition function should therefore assume two "empty" partitions and build them gradually To start with 2 empty partitions we can set lastSmall to be first and i to be first+1 elements yet to be processed first lastSmall i i last

Tracing the Partition Function first(0) i(1) lastSmall(0) If the value at i is less than the pivot advance lastSmall pivot

Tracing the Partition Function first(0) = pivot first(0) = pivot i(1) lastSmall(1) If value at i is less than the pivot advance lastSmall and swap value at I and lastSmall

Tracing the Partition Function first(0) i(1) lastSmall(1) If value at i is less than the pivot advance lastSmall and swap value at i and lastSmall - so swap 4 with itself first(0) = pivot first(0) = pivot

Tracing the Partition Function first(0) i(2) lastSmall(1) If value at i is greater than the pivot: Do nothing

Tracing the Partition Function first(0) i(3) lastSmall(1) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Tracing the Partition Function first(0) i(3) lastSmall(2) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Tracing the Partition Function first(0) i(3) lastSmall(2) If value at i is less than the pivot advance lastSmall and swap i and lastSmall. Swap 5 and 7

Tracing the Partition Function first(0) i(4) lastSmall(2)

Tracing the Partition Function first(0) i(4) lastSmall(3) If value at i is less than the pivot advance lastSmall and swap i and lastSmall. Swap 1 and 7

Tracing the Partition Function first(0) i(4) lastSmall(3)

Tracing the Partition Function first(0) i(5) lastSmall(3) If value at I is not less than the pivot do nothing – advance i

Tracing the Partition Function first(0) i(6) lastSmall(3) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Tracing the Partition Function first(0) i(6) lastSmall(4) If value at i is less than the pivot advance lastSmall and swap i and lastSmall - 2 and 7

Tracing the Partition Function first(0) i(7) lastSmall(4)

Tracing the Partition Function first(0) i(7) lastSmall(4)

Tracing the Partition Function first(0) pivot first(0) pivot lastSmall(4) The final step is to swap the pivot and lastsmall

Tracing the Partition Function first(0) pivot first(0) pivot lastSmall(4) The final step is to swap the pivot and lastsmall and split the array at lastsmall

Tracing the Partition Function Split –(4) The final step is to swap the pivot and lastsmall and split the array at lastsmall

Partition function for Quicksort public int partition (int list[], int first, int last) { int lastSmall(first); for (int i=first+1; i<=last; i++) if (list[i] <= list[first]) { ++lastSmall; swap(list,lastSmall,i); } swap(list,first,lastSmall); return lastSmall; }// position of the pivot returned public int partition (int list[], int first, int last) { int lastSmall(first); for (int i=first+1; i<=last; i++) if (list[i] <= list[first]) { ++lastSmall; swap(list,lastSmall,i); } swap(list,first,lastSmall); return lastSmall; }// position of the pivot returned

Quicksort(0,3) split(4) The rest of the sort is done in the next slides for those who would like to trace the whole sort.

Quicksort(0,3) first(0) i(1) lastSmall(0) If value at i is less than the pivot Advance lastSmall and swap i and lastSmall

Quicksort(0,3) first(0) i(2) lastSmall(0)

Quicksort(0,3) first(0) i(3) lastSmall(0) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Quicksort(0,3) first(0) i(3) lastSmall(1) The 1 is less than lastSmall so we swap the 1 and the 4

Quicksort(0,3) first(0) i(3) lastSmall(1)

Quicksort(0,3) first(0) i(3) lastSmall(1) We have come to the end of the first partition, so lastSmall is the new pivot and we swap first which is 2 and lastSmall (1)

Quicksort(0,3) split(1) We have now split at index 1 and must do the same partition algorithm to the 1(even though there is only one element in the partition)

Quicksort(0,0)

Quicksort(2,3) first(2) i(3) lastSmall(2) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Quicksort(2,3) first(2) i(3) lastSmall(3)

Quicksort(2,3) first(2) i(3) lastSmall(3)

Quicksort(2,3) first(2) lastSmall(3)

Quicksort(2,3) first(2) lastSmall(3) If value at i is less than the pivot advance lastSmall and swap i and lastSmall

Quicksort(2,3) split(3)

Quicksort(2,2)

Quicksort(4,3) Page

Quicksort(5,7) first(5) i(6) lastSmall(5)

Quicksort(5,7) first(5) i(6) lastSmall(6)

Quicksort(5,7) first(5) i(6) lastSmall(6)

Quicksort(5,7) first(5) i(7) lastSmall(6)

Quicksort(5,7) first(5) i(7) lastSmall(7)

Quicksort(5,7) first(5) i(7) lastSmall(7)

Quicksort(5,7) first(5) lastSmall(6)

Quicksort(5,7) first(5) lastSmall(7)

Quicksort(5,7) split(7)

Quicksort(5,6) first(5) i(6) lastSmall(5)

Quicksort(5,6) first(5) i(6) lastSmall(6)

Quicksort(5,6) first(5) i(6) lastSmall(6)

Quicksort(5,6) first(5) lastSmall(6)

Quicksort(5,6) first(5) lastSmall(6)

Quicksort(5,6) split(6)

Quicksort(5,5)

Quicksort(7,6)

Quicksort(8,7)

Quicksort(8,7)

public static void quickSort(Integer a[], int first, int last) { // PreCondtion: in an array to be sorted from //a[first]..a[last] if (first >= last) return; // PostCondition: we have an empty array or all //partitions have completed // call method to do the partition int split = partition(a, first, last); quickSort(a, first, split - 1); // recursive sort left quickSort(a, split + 1, last); // recursive sort right // post condition: a is sorted }

private static int partition(Integer a[], int first, int last) { int lastSmall = first; // place lastmall at index 0 for (int i = first + 1; i <= last; i++) { // if element in i is less than pivot if (a[i].compareTo(a[first]) <= 0) { lastSmall++; // increment last small and call swap swapElements(a, lastSmall, i); } } // put pivot in proper position swapElements(a, first, lastSmall); return lastSmall; }

Quicksort - Analysis Partition Check every item onceO(n) Conquer Divide data in halfO(log 2 n) Total ProductO(n log n) Same as Heapsort quicksort is generally faster Fewer comparisons Details later But there’s a catch …………….

Quicksort - The truth! What happens if we use quicksort on data that’s already sorted (or nearly sorted) We’d certainly expect it to perform well!

Quicksort - The truth! Sorted data pivot < pivot ? > pivot

Quicksort - The truth! Sorted data Each partition produces a problem of size 0 and one of size n-1! Number of partitions? > pivot pivot

Quicksort - The truth! Sorted data Each partition produces a problem of size 0 and one of size n-1! Number of partitions? n each needing time O(n) Total nO(n) or O(n 2 ) ? Quicksort is as bad as bubble or insertion sort > pivot pivot

Best case Analysis of Quicksort Page 75 The best thing that could happen to quicksort is if the pivot divides the list exactly in two halves each time (as below) n 7n 7n 7n n n-3

Quicksort - The truth!  Quicksort’s O(n log n) behaviour  Depends on the partitions being nearly equal  there are O( log n ) of them  On average, this will nearly be the case  and quicksort is generally O(n log n)  Can we do anything to ensure O(n log n) time?  In general, no But we can improve our chances!!

Quicksort - Choice of the pivot Any pivot will work … Choose a different pivot … so that the partitions are equal then we will see O(n log n) time pivot < pivot > pivot

Quicksort - Median-of-3 pivot Take 3 positions and choose the median say … First, middle, last median is 5 perfect division of sorted data every time! O(n log n) time çSince sorted (or nearly sorted) data is common, median-of-3 is a good strategy especially if you think your data may be sorted!

Quicksort - Random pivot Choose a pivot randomly Different position for every partition ç On average, sorted data is divided evenly ç O(n log n) time Key requirement Pivot choice must take O(1) time

Quicksort - Guaranteed O(n log n)? Never Guaranteed!! Any pivot selection strategy could lead to O(n 2 ) time Here median-of-3 chooses 2 One partition of 1 and One partition of 7 Next it chooses 4 One of 1 and One of

Worst case for quicksort In the worst case, recursion may be n levels deep (for an array of size n ) But the partitioning work done at each level is still n O(n) * O(n) = O(n 2 ) So worst case for Quicksort is O(n 2 ) When does this happen? When the array is sorted to begin with!

Final comments Quicksort is the fastest known sorting algorithm For optimum efficiency, the pivot must be chosen carefully “Median of three” is a good technique for choosing the pivot However, no matter what you do, there will be some cases where Quicksort runs in O(n 2 ) time

The recurrence for quicksort (assuming we're successful in always dividing the list in two parts of equal size) gives us a function that it is equivalent to O(n log n) FOR THE AVERAGE CASE AVERAGE CASE

We examine sorting algorithms on 4 different initial conditions. These visualizations are intended to: Show how each algorithm operates. Show that there is no best sorting algorithm. Show the advantages and disadvantages of each algorithm. Show that worse-case asymptotic behavior is not the deciding factor in choosing an algorithm. Show that the initial condition (input order and key distribution) affects performance as much as the algorithm choice. The ideal sorting algorithm would have the following properties: Stable: Equal keys aren't reordered. Operates in place, requiring O(1) extra space. Worst-case O(n·lg(n)) key comparisons. Worst-case O(n) swaps. Adaptive: Speeds up to O(n) when data is nearly sorted or when there are few unique keys. There is no algorithm that has all of these properties, and so the choice of sorting algorithm depends on the application.