1. Recursive sorting: Quicksort and its Complexity

2. Sorting ¶ « A « K « 10 « 2 ª J ª 2 · ¨ Q © 2 © 9 © 9 ¸
Card players all know how to sort …
First card is already sorted
With all the rest,
Scan back from the end until you find the first card larger than the new one,
Move all the lower ones up one slot
insert it A K 10 2 J 2 Q 2 9 9

3. Sorting - Insertion sort
Complexity
For each card
Scan O(n)
Shift up O(n)
Insert O(1)
Total O(n)
First card requires O(1), second O(2), …
For n cards operations ç O(n2)
S i
i=1 n

4. Sorting - Insertion sort
Complexity
For each card
Scan O(n) O(log n)
Shift up O(n)
Insert O(1)
Total O(n)
First card requires O(1), second O(2), …
For n cards operations ç O(n2)
Use binary search!
Unchanged!
Because the
shift up operation
still requires O(n)
time
S i
i=1 n

5. Insertion Sort - Implementation
A challenge for you
The code in the notes (and on the Web) has an error
First person to a correct version gets up to 2 extra marks added to their final mark if that would move them up a grade!
ie if you had x8% or x9%, it goes to (x+1)0%
To qualify, you need to point out the error in the original, as well as supply a corrected version!

6. Sorting - Bubble From the first element
Exchange pairs if they’re out of order
Last one must now be the largest
Repeat from the first to n-1
Stop when you have only one element to check

7. Bubble Sort - Analysis O(1) statement /* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }
void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }
O(1) statement

8. Bubble Sort - Analysis Inner loop O(1) statement
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }
void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }
Inner loop
n-1, n-2, n-3, … , 1 iterations
O(1) statement

9. Bubble Sort - Analysis Outer loop n iterations
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }
void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }
Outer loop n iterations

10. Bubble Sort - Analysis Overall n(n+1) S i = = O(n2) 2
/* Bubble sort for integers */
#define SWAP(a,b) { int t; t=a; a=b; b=t; }
void bubble( int a[], int n ) {
int i, j;
for(i=0;i<n;i++) { /* n passes thru the array */
/* From start to the end of unsorted part */
for(j=1;j<(n-i);j++) {
/* If adjacent items out of order, swap */
if( a[j-1]>a[j] ) SWAP(a[j-1],a[j]); }
Overall 1
n(n+1)
S i =
= O(n2) 2
i=n-1
inner loop iteration count
n outer loop iterations

11. Sorting - Simple Bubble sort Insertion sort But HeapSort is O(n log n)
Very simple code
Insertion sort
Slightly better than bubble sort
Fewer comparisons
Also O(n2)
But HeapSort is O(n log n)
Where would you use bubble or insertion sort?

12. Simple Sorts Bubble Sort or Insertion Sort Use when n is small
Simple code compensates for low efficiency!

13. Recursive Array Programming
Recursive function definitions assume that a function works for a smaller value.
With arrays, “a smaller value” means a shorter array, i.e., a subarray, contiguous elements from the original array
We’ll define a recursive function over an array by using the same function over a subarray, and a base case
Subscripts will mark the lower and upper bounds of the subarrays

14. Subarrays myArray Base case [0] [1] [2] [3] [4] [5] [6] [7] [8] [9]
1 through 9
2 through 9
3 through 9
4 through 9
5 through 9
6 through 9
7 through 9
8 through 9
9 through 9
Base case

15. Example: Recursively find sum of array elements, A[lo] to A[hi]
Assume sum( ) properly returns sum of elements for a smaller array of doubles
Then we could write: double sum (double[ ] A, int lo, int hi) { return ( A[lo] + sum(A, lo + 1, hi) ); }
But we’re not done; what’s the base case?

16. Base Case is when Subarray is Empty, hi is less than lo
double sum (double[ ] A, int lo, int hi) { if (hi < lo) return 0.0; else return ( A[lo] + sum(A, lo + 1, hi) ); }
Yes, we could have defined this using (hi == lo) as the base case…

17. Recursive Sorting Algorithms
We can use this same idea of recursive functions over subarrays to rewrite our sorting algorithms
Let’s see how this works for selection sort, insertion sort, and then some new sorting algorithms

18. More Recursive Sorting: Quicksort
Quicksort is an O(n2) algorithm in the worst case, but its running time is usually proportional to n log2 n; it is the method of choice for many sorting jobs
We’ll first look at the intuition behind the algorithm: take an array V E R O N I C A
arrange it so the small values are on the left half and all the big values are on the right half: E I C A V R O N

19. C A E I O N V R A C E I N O R V Quicksort Intuition and again:
Then, do it again: C A E I O N V R
and again: A C E I N O R V
The divide-and-conquer strategy will, in general, take log2n steps to move the A into position. The intuition is that, doing this for n elements, the whole algorithm will take O(n log2 n) steps.

20. What Quicksort is really doing
1. Partition array a into smaller elements and larger ones – smaller ones from a[0]…a[m-1] (not necessarily in order), larger ones in positions a[m+1]…a[length-1] (not necessarily in order), and the “middle” element in a[m]. So a: 5 27 12 3 18 11 7 19
might be partitioned (with m=4) as: 5 7 3 11 12 27 18 19
smaller than pivot
larger than pivot
pivot

21. quicksort(), next 2 steps
2. Recursively sort a[0]…a[m-1]. Our a becomes: 3 5 7 11 12 27 18 19
pivot
3. Recursively sort a[m+1]…a[length-1]. Our a becomes: 3 5 7 11 12 18 19 27
pivot

22. quicksort( ) private void quicksort(double[] a, int lo, int hi) {
int m;
if (hi > lo+1) { // there are at least 3 elements
// so sort recursively
m = partition(a, lo, hi);
quicksort(a, lo, m-1);
quicksort(a, m+1, hi); }
else // the base case… …

23. The base case…
We have the base case in this recursion when the subarray a[lo]…a[hi] contains zero elements (lo==hi+1), one element (lo==hi), or two elements (lo==hi-1).
With no elements, we ignore it
With one element, it’s already sorted
With two elements, we just (might) swap them:
// 0, 1, or 2 elements, so sort directly if (hi == lo+1 && a[lo] > a[hi]) swap(a, lo, hi);

24. quicksort( ) private void quicksort(double[] a, int lo, int hi) {
int m;
if (hi > lo+1) { // there are at least 3 elements
// so sort recursively
m = partition(a, lo, hi);
quicksort(a, lo, m-1);
quicksort(a, m+1, hi); }
else // 0, 1, or 2 elements, so sort directly
if (hi == lo+1 && a[lo] > a[hi])
swap(a, lo, hi);

25. The outside world’s version
As usual, we overload quicksort( ) and provide a public version that “hides” the last two arguments:
public void quicksort(double[] a) {
quicksort(a, 0, a.length-1); }

26. Now, all we need is partition( )
int partition(double[] a, int lo, int hi) {
// Choose middle element among a[lo]…a[hi],
// and move other elements so that a[lo]…a[m-1]
// are all less than a[m] and a[m+1]…a[hi] are
// all greater than a[m] //
// m is returned to the caller … }

27. Are you feeling lucky?
Now, this would work great if we knew the median value in the array segment, and could choose it as the pivot (i.e., know which small values go left and which large values go right). But we can’t know the median value without actually sorting the array!
So instead, we somehow pick a pivot and hope that it is near median.
If the pivot is the worst choice (each time), the algorithm becomes O(n2). If we are roughly dividing the subarrays in half each time, we get an O(nlog2n) algorithm.

28. How to pick the median value
There are many techniques for doing this. In practice, one good way of choosing the pivot is to take the median of three elements, specifically a[lo+1], a[(lo+hi)/2], and a[hi]
We’ll choose their median using the method medianLocation( )

29. medianLocation( ) int medianLocation(double[] a,
int i, int j, int k) {
if (a[i] <= a[j])
if (a[j] <= a[k])
return j;
else if (a[i] <= a[k])
return k;
else return i;
else // a[j] < a[i]
if (a[i] <= a[k])
return i;
else if (a[j] <= a[k])
else return j; }

30. The partitioning process
From the three red elements, we choose the median, a[hi] 8 4 9 1 10 2 6 3 5 7
lo+1
(lo+hi)/2 hi
We swap the median with a[lo], and start the partitioning process on the rest: 7 4 9 1 10 2 6 3 5 8
lo+1
(lo+hi)/2 hi

31. The partitioning process
We shuffle around elements from a[lo+1] to a[hi] so that all elements less than the pivot (a[lo]) appear to all the elements greater than the pivot
Until we are done, we have no way of knowing how many elements are less and how many elements are greater 7 4 5 1 3 2 6 10 8 9
lo+1
(lo+hi)/2 m hi

32. The partitioning process7 4 5 1 3 2 6 10 8 9
lo+1
(lo+hi)/2 m hi
m is the largest subscript that contains a value less than the pivot
We have discovered (in our example) that m = 6
We then swap a[m] with a[lo], placing the pivot in its rightful position, in a[m], then continue to sort the left and right subarrays recursively 6 4 5 1 3 2 7 10 8 9 lo
lo+1
(lo+hi)/2 m hi
pivot

33. How partition( ) works
We will use a 3-argument method: int partition(double[ ] a, int lo, int hi) and a 4-argument method: int shuffle(double[ ] a, int lo, int hi, double pivot)
The 3-argument “partition” moves the pivot element into a[lo], calls the 4-argument method “shuffle” to shuffle the elements in the subarray a[lo+1]…a[hi], then swaps a[lo] into a[m] and returns m

34. 3-argument partition( )
int partition(double[] a, int lo, int hi) {
// Choose middle element among a[lo]…a[hi],
// and move other elements so that a[lo]…a[m-1]
// are all less than a[m] and a[m+1]…a[hi] are
// all greater than a[m] //
// m is returned to the caller swap(a, lo, medianLocation(a, lo+1, hi, (lo+hi)/2));
int m = shuffle(a, lo+1, hi, a[lo]);
swap(a, lo, m); return m; }

35. How the 4-argument shuffle( ) works
The 4-argument method does the main work, recursively calling itself on subarrays
We of course assume shuffle( ) works on any smaller array…
If the first element of the array is less than or equal to pivot, it’s already in the right place, just call shuffle( ) recursively on the rest:
if (a[lo] <= pivot) // a[lo] in correct half return shuffle(a, lo+1, hi, pivot);
(this is the “current” lo, not the lo of the whole array)

36. How the 4-argument shuffle( ) works
If a[lo] > pivot, then a[lo] belongs in the upper half of the subarray, and we swap it with a[hi]
We still don’t know where the new a[lo] value should go, so we shuffle recursively on the subarray that includes a[lo] but not a[hi]
if (a[lo] <= pivot) // a[lo] in correct half return shuffle(a, lo+1, hi, pivot); else { // a[lo] in wrong half swap(a, lo, hi); return shuffle(a, lo, hi-1, pivot); }

37. How the 4-argument shuffle( ) works
The base case is the one element subarray, when lo==hi. Is the one element “small” or “large”?
If it is small (less than the pivot), then it is at the middle point m (and we can swap it with the pivot)
Otherwise, it is just above the middle point (and we want the pivot swapped with the element just below it)

38. 4-argument shuffle( )
int shuffle(double[] a, int lo, int hi, double pivot) {
if (hi == lo)
if (a[lo] < pivot)
return lo;
else
return lo-1;
else if (a[lo] <= pivot) // a[lo] in correct half
return shuffle(a, lo+1, hi, pivot);
else { // a[lo] in wrong half
swap(a, lo, hi);
return shuffle(a, lo, hi-1, pivot); }

39. Example of partition
The starting array: 4 5 3 2 6 1 lo hi
Choose the median from among: 4 5 3 2 6 1 lo
lo+1
(lo+hi)/2 hi
Swap the median with a[lo]: 3 5 4 2 6 1
pivot

40. Example of partition
Now, shuffle the subarray (not counting pivot); a is now the new subarray… 3 5 4 2 6 1
pivot lo hi
a[lo] > pivot, so swap it with a[hi], and continue with the shuffle: 3 1 4 2 6 5
pivot lo hi

41. Example of partition
a[lo] is now less than pivot, so we leave it and continue with the shuffle: 3 1 4 2 6 5
pivot lo hi
Now a[lo] is greater than pivot, so we swap it with a[hi] and continue with the shuffle: 3 1 6 2 4 5
pivot lo hi

42. Example of partition
a[lo] is again greater than pivot, so we swap it with a[hi] and continue with the shuffle: 3 1 2 6 4 5
pivot lo hi
a[lo] is less than the pivot, so lo (i.e., index 2) is returned by the 4-argument shuffle( ); the 3-argument partition( ) then swaps the pivot and the middle element, also returning index 2 2 1 3 6 4 5
Now we’re ready to recursively quicksort the left and right subarrays
pivot

43. quicksort( ) private void quicksort(double[] a, int lo, int hi) {
int m;
if (hi > lo+1) { // there are at least 3 elements
// so sort recursively
m = partition(a, lo, hi);
quicksort(a, lo, m-1);
quicksort(a, m+1, hi); }
else // 0, 1, or 2 elements, so sort directly
if (hi == lo+1 && A[lo] > A[hi])
swap(a, lo, hi);

44. Performance of QuickSort
Best case: O(nlog2n)
Worst case: O(n2) – when the pivot is always the second-largest or second-smallest element (since medianLocation won’t let us choose the smallest or largest)
Average case over all possible arrangements of n array elements: O(nlog2n)

45. Quicksort Efficient sorting algorithm
Discovered by C.A.R. Hoare
Example of Divide and Conquer algorithm
Two phases
Partition phase
Divides the work into half
Sort phase
Conquers the halves!

46. Quicksort Partition < pivot pivot > pivot Choose a pivot
Find the position for the pivot so that
all elements to the left are less
all elements to the right are greater
< pivot
pivot
> pivot

47. Quicksort Conquer < pivot > pivot < p’ p’ > p’ pivot
Apply the same algorithm to each half
< pivot
> pivot
< p’ p’
> p’
pivot
< p” p”
> p”

48. Quicksort Implementation quicksort( void *a, int low, int high ) {
int pivot;
/* Termination condition! */
if ( high > low )
pivot = partition( a, low, high );
quicksort( a, low, pivot-1 );
quicksort( a, pivot+1, high ); }
Divide
Conquer

49. Any item will do as the pivot, choose the leftmost one!
Quicksort - Partition
This example
uses int’s
to keep things
simple!
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
Any item will do as the pivot,
choose the leftmost one! 23 12 15 38 42 18 36 29 27
low
high

50. Set left and right markers
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
Set left and right markers
left
right 23 12 15 38 42 18 36 29 27
low
pivot: 23
high

51. Quicksort - Partition Move the markers until they cross over left
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
Move the markers
until they cross over
left
right 23 12 15 38 42 18 36 29 27
low
pivot: 23
high

52. Move the left pointer while it points to items <= pivot
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
Move the left pointer while
it points to items <= pivot
left
right
Move right
similarly 23 12 15 38 42 18 36 29 27
low
pivot: 23
high

53. on the wrong side of the pivot
Quicksort - Partition
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
Swap the two items
on the wrong side of the pivot
left
right 23 12 15 38 42 18 36 29 27
pivot: 23
low
high

54. Quicksort - Partition left and right have swapped over, so stop right
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
left and right
have swapped over,
so stop
right
left 23 12 15 18 42 38 36 29 27
low
pivot: 23
high

55. Quicksort - Partition right left 23 12 15 18 42 38 36 29 27 low
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
right
left 23 12 15 18 42 38 36 29 27
low
pivot: 23
high
Finally, swap the pivot
and right

56. Quicksort - Partition right pivot: 23 18 12 15 23 42 38 36 29 27 low
int partition( int *a, int low, int high ) {
int left, right;
int pivot_item;
pivot_item = a[low];
pivot = left = low;
right = high;
while ( left < right ) {
/* Move left while item < pivot */
while( a[left] <= pivot_item ) left++;
/* Move right while item > pivot */
while( a[right] >= pivot_item ) right--;
if ( left < right ) SWAP(a,left,right); }
/* right is final position for the pivot */
a[low] = a[right];
a[right] = pivot_item;
return right;
right
pivot: 23 18 12 15 23 42 38 36 29 27
low
high
Return the position
of the pivot

57. Quicksort - Conquer pivot pivot: 23 18 12 15 23 42 38 36 29 27
Recursively
sort left half
Recursively
sort right half

58. Quicksort - Analysis Partition Conquer Total
Check every item once O(n)
Conquer
Divide data in half O(log2n)
Total
Product O(n log n)
Quicksort is generally faster
Fewer comparisons
Details later (and assignment 2!)
But there’s a catch …………….

59. Quicksort - The truth!
What happens if we use quicksort on data that’s already sorted (or nearly sorted)
We’d certainly expect it to perform well!

60. Quicksort - The truth! Sorted data pivot ? 1 2 3 4 5 6 7 8 9

61. Quicksort - The truth! Sorted data Each partition produces pivot 1 2 3
a problem of size 0
and one of size n-1!
Number of partitions?
pivot 1 2 3 4 5 6 7 8 9
> pivot
pivot 2 3 4 5 6 7 8 9
> pivot

62. Quicksort - The truth! Sorted data pivot 1 2 3 4 5 6 7 8 9 > pivot
Each partition produces
a problem of size 0
and one of size n-1!
Number of partitions?
n each needing time O(n)
Total nO(n) or O(n2)
Quicksort is as bad as bubble or insertion sort
pivot 1 2 3 4 5 6 7 8 9
> pivot
pivot 2 3 4 5 6 7 8 9
> pivot

63. Quicksort - The truth! Quicksort’s O(n log n) behaviour
Depends on the partitions being nearly equal
there are O( log n ) of them
On average, this will nearly be the case
and quicksort is generally O(n log n)
Can we do anything to ensure O(n log n) time?
In general, no
But we can improve our chances!!

64. Quicksort - Choice of the pivot
Any pivot will work …
Choose a different pivot …
so that the partitions are equal
then we will see O(n log n) time
pivot 1 2 3 4 5 6 7 8 9
> pivot
< pivot

65. Quicksort - Median-of-3 pivot
Take 3 positions and choose the median
say … First, middle, last
median is 5
perfect division of sorted data every time!
O(n log n) time
Since sorted (or nearly sorted) data is common, median-of-3 is a good strategy
especially if you think your data may be sorted! 1 2 3 4 5 6 7 8 9

66. Quicksort - Random pivot
Choose a pivot randomly
Different position for every partition
On average, sorted data is divided evenly
O(n log n) time
Key requirement
Pivot choice must take O(1) time

67. Quicksort - Guaranteed O(n log n)?
Never!!
Any pivot selection strategy could lead to O(n2) time
Here median-of-3 chooses 2
One partition of 1 and
One partition of 7
Next it chooses 4
One of 1 and
One of 5 1 4 9 6 2 5 7 8 3 1 2 4 9 6 5 7 8 3

68. Key Points Sorting Bubble, Insert O(n2) sorts Simple code
May run faster for small n, n ~10 (system dependent)
Quick Sort
Divide and conquer
O(n log n)

69. Key Points Quick Sort Better but not guaranteed O(n log n) but ….
Can be O(n2)
Depends on pivot selection
Median-of-3
Random pivot
Better but not guaranteed

70. Quicksort - Why bother? Use Heapsort instead?
Quicksort is generally faster
Fewer comparisons and exchanges
Some empirical data

71. Quicksort - Why bother? Divide by n log n Divide by n2 Reporting data
Normalisation works when you have a hypothesis to work with!
Divide by n log n
Divide by n2