1. 1
Lecture 13
CS2013

2. Quick-Sort
Presentation for use with the textbook Data Structures and Algorithms in Java, 6th edition, by M. T. Goodrich, R. Tamassia, and M. H. Goldwasser, Wiley, 2014
Quick-Sort 
4 2  2 4
7 9  7 9
2  2
9  9
Quick-Sort

3. Quick-Sort
Quick-sort is a randomized sorting algorithm based on the divide-and-conquer paradigm:
Divide: pick a random element x (called pivot) and partition S into
L elements less than x
E elements equal x
G elements greater than x
Recur: sort L and G
Conquer: join L, E and G x x L G E x
Quick-Sort

4. Partition We partition an input sequence as follows:
We remove, in turn, each element y from S and
We insert y into L, E or G, depending on the result of the comparison with the pivot x
Each insertion and removal is at the beginning or at the end of a sequence, and hence takes O(1) time
Thus, the partition step of quick-sort takes O(n) time
Algorithm partition(S, p)
Input sequence S, position p of pivot
Output subsequences L, E, G of the elements of S less than, equal to, or greater than the pivot, resp.
L, E, G  empty sequences
x  S.remove(p)
while S.isEmpty()
y  S.remove(S.first())
if y < x
L.addLast(y)
else if y = x
E.addLast(y)
else { y > x }
G.addLast(y)
return L, E, G
Quick-Sort

5. Java Implementation
Quick-Sort

6. Quick-Sort Tree
An execution of quick-sort is depicted by a binary tree
Each node represents a recursive call of quick-sort and stores
Unsorted sequence before the execution and its pivot
Sorted sequence at the end of the execution
The root is the initial call
The leaves are calls on subsequences of size 0 or 1 
4 2  2 4
7 9  7 9
2  2
9  9
Quick-Sort

7. Worst-case Running Time
The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element
One of L and G has size n - 1 and the other has size 0
The running time is proportional to the sum
n + (n - 1) + …
Thus, the worst-case running time of quick-sort is O(n2)
dept h
time n 1
n - 1 … …
Quick-Sort

8. Expected Running Time
Consider a recursive call of quick-sort on a sequence of size s
Good call: the sizes of L and G are close to equal
Bad call: one of L and G is much larger than the other
 1 1
Good call
Bad call
Bad pivots
Good pivots
Bad pivots
Quick-Sort

9. Expected Running Time, Part 2
Best case: the pivots are all perfect. Height of tree is ceiling of O(log n)
Typical case: some of the pivots are good, some are bad.
Height still O(log n), with a different base
Worst case: all the pivots
are bad: height of tree is O(n)
The amount or work done at the nodes of the same depth is O(n), because all the nodes appear at every depth
Thus, the best case and expected running time of quick- sort is O(n log n); worst case is O(n2)
Quick-Sort

10. In-Place Quick-Sort Quick-sort can be implemented to run in-place
In the partition step, we use replace operations to rearrange the elements of the input sequence such that
the elements less than the pivot have rank less than h
the elements equal to the pivot have rank between h and k
the elements greater than the pivot have rank greater than k
The recursive calls consider
elements with rank less than h
elements with rank greater than k
Algorithm inPlaceQuickSort(S, l, r)
Input sequence S, ranks l and r
Output sequence S with the elements of rank between l and r rearranged in increasing order
if l  r
return
i  a random integer between l and r
x  S.elemAtRank(i)
(h, k)  inPlacePartition(x)
inPlaceQuickSort(S, l, h - 1)
inPlaceQuickSort(S, k + 1, r)
Quick-Sort

11. In-Place Partitioning
Perform the partition using two indices to split S into L and E U G (a similar method can split E U G into E and G).
Repeat until j and k cross:
Scan j to the right until finding an element > x.
Scan k to the left until finding an element < x.
Swap elements at indices j and k j k
(pivot = 6) j k
Quick-Sort

12. Java Implementation
Quick-Sort

13. Merge Sort Merge Sort is a recursive divide and conquer algorithm13
Merge Sort
Merge Sort is a recursive divide and conquer algorithm
Recursive method splits the list repeatedly until it is made up of single-element sublists, then merges them as the recursion unwinds:
mergeSort(list):
firstHalf = mergeSort(firstHalf);
secondHalf = mergeSort(secondHalf);
list = merge(firstHalf, secondHalf);
merge:
add the lesser of firstHalf [0] and secondHalf [0] to the new, larger list
repeat until one of the (already sorted) sublists is exhausted
add the rest of the remaining sublist to the larger list.

14. 14
Merge Sort

15. Merge Sort Time Thus, Merge Sort is O(n log n)15
Merge Sort Time
Assume n is a power of 2. This assumption makes the math simpler. If n is not a power of 2, the difference is irrelevant to the order of complexity.
Merge sort splits the list into two sublists, sorts the sublists using the same algorithm recursively, and then merges the sublists. Each recursive call merge sorts half the list, so the depth of the recursion is the number of times you need to split n to get lists of size 1, ie log n. The single-item lists are, obviously, sorted.
Merge Sort reassembles the list in log n steps, just as it broke the list down.
The total size of all the sublists is n, the original size of the unsorted list. To merge the sublists, across all the sublists at one level of recursion, takes at most n-1 comparisons and n copies of one element from a sublist to the merged list. The total merge time is 2n-1, which is O(n).
The O(n) merging happens log n times as the full sorted array is built.
Thus, Merge Sort is O(n log n)

16. Merge Sort Time Here it is again, but with more math.16
Merge Sort Time
Here it is again, but with more math.
Let T(n) denote the time required for sorting an array of n elements using merge sort. Without loss of generality, assume n is a power of 2 and k is log(n). The merge sort algorithm splits the array into two sublists, sorts the sublists using the same algorithm recursively, and then merges the sublists. So,
The first T(n/2) is the time required for sorting the first half of the list and the second T(n/2) is the time required for sorting the second half.

17. Useful Mathematic Summations17
Useful Mathematic Summations

18. 18
Merge Sort Complexity
Merging two sublists takes at most n-1 comparisons plus n copies. The merge time is 2n-1.
a =2, so a-1 = 1
2log n = n and T(1) = 1
a0+a1+a2+...a(k-1) = ak+1-1 / a-1 n n n n T ( n ) 2  2 T ( )  2 n  1  2 ( 2 T ( )  2  1 )  2 n  1  2 T ( )  2 n  2  2 n  1 2 4 2 2 2 n 2 k T ( ) 2 n 2 k  1    
...  2 n 2 2 n 1
After we expand this k times, to get to n    2 k n  2
log n T ( ) 2 n 2
log n 1    
...  2 n n  2  2 n  1 2
log n  n  2 n
log n  2
log n  1  2 n
log n  1  O ( n
log n )
Sum of all the +2n terms
Subtractive term, so the -1 from the summation becomes +1

19. 19
Bucket Sort
All sort algorithms discussed so far are general sorting algorithms that work for any types of keys (e.g., integers, strings, and any comparable objects).
These algorithms sort the elements by comparing their keys.
The lower bound for general sorting algorithms is O(nlogn). So, no sorting algorithms based on comparisons can perform better than O(n log n).
However, if the keys are small integers, you can use bucket sort without having to compare the keys.

20. 20
Bucket Sort
The bucket sort algorithm works as follows. Assume the keys are in the range from 0 to N-1. We need N buckets labeled 0, 1, ..., and N-1. If an element’s key is i, the element is put into the bucket i. Each bucket holds the elements with the same key value. You can use an ArrayList to implement a bucket.
Bucket Sort is O(n).
Searching: finding the bucket is O(1). Searching the list in one bucket is O(1) in the best case and O(n) in the worst case.
Note the similarity to a hash map with a very simple hash function.

21. BST Time Complexity
If a BST is perfectly balanced, i.e., a complete binary tree, its height (number of levels minus 1) is the floor of log n. If it is completely unbalanced, its height is n-1
The time complexity for search, insertion and deletion is the height of the tree. In the worst case, it is O(n). In the best case it is O(log n).
We want our trees to be balanced, so that we get the O(log n) behavior 21

22. Balancing Binary Search Trees
We can keep trees in balance by rotating nodes in various ways after insertions and deletions.
Rebalancing trees has a cost. In general, however, the more often we search the tree relative to the frequency with which we insert or delete nodes, the more advantageous it is to keep the tree balanced.
It is possible to maintain a BST so that it is always perfectly balanced, but in most cases we can be a little bit lax about this in order to reduce the cost of rebalancing.
The compromise is to maintain a reasonably well-balanced tree, i.e., one in which the heights of the two subtrees for every node are about the same.
One way to accomplish this is by using an AVL Tree. 22

23. AVL Tree Definition AVL trees are balanced
An AVL Tree is a binary search tree such that for every internal node v of T, the heights of the children of v can differ by at most 1
An example of an AVL tree where the heights are shown next to the nodes
AVL Trees 23

24. 3 4
n(1)
n(2)
Height of an AVL Tree
The height of an AVL tree storing n keys is O(log n).
Proof (by induction): Let us bound n(h): the minimum number of internal nodes of an AVL tree of height h.
We easily see that n(1) = 1 and n(2) = 2
For n > 2, an AVL tree with the minimum number of nodes for height h contains the root node, one AVL subtree of height n-1 and another of height n-2.
That is, n(h) = 1 + n(h-1) + n(h-2), which is at least exponential (there is some b such that n would grow at least as fast as b ^ h)
The rate of growth of the height with n is the opposite of the rate of growth of the minimum number of nodes.
Big O is an upper bound
If the number of nodes required for a height is at least exponential, the growth of the height is O(log n)
Thus the height of an AVL tree is O(log n)
AVL Trees 24

25. Insertion Insertion is as in a binary search tree
Always done by expanding an external node.
Example: 44 17 78 32 50 88 48 62 44 17 78 32 50 88 48 62 54
c=z
a=y
b=x w
before insertion
after insertion
AVL Trees 25

26. Trinode Restructuring
Let (a,b,c) be the inorder listing of x, y, z
Perform the rotations needed to make b the topmost node of the three
c=y
b=x
a=z T0 T1 T2 T3
Single rotation around b
Double rotation around c and a
b=y
a=z
c=x T0 T1 T2 T3
b=x
c=y
a=z T0 T1 T2 T3
b=y
a=z
c=x T0 T1 T2 T3
AVL Trees 26

27. Restructuring (as Single Rotations)
AVL Trees 27

28. Restructuring (as Double Rotations)
AVL Trees 28

29. Removal
Removal begins as in a binary search tree. Its parent, w, may cause an imbalance.
Example: 44 17 78 32 50 88 48 62 54 44 17 62 50 78 48 54 88
before deletion of 32
after deletion
AVL Trees 29

30. Rebalancing after a Removal
Let z be the first unbalanced node encountered while traveling up the tree from w. Also, let y be the child of z with the larger height, and let x be the child of y with the larger height
We perform a trinode restructuring to restore balance at z
As this restructuring may upset the balance of another node higher in the tree, we must continue checking for balance until the root of T is reached 62
a=z 44 44 78 w 17 62
b=y 17 50 88
c=x 50 78 48 54 48 54 88
AVL Trees 30

31. AVL Tree Performance AVL tree storing n items
The data structure uses O(n) space
A single restructuring takes O(1) time
using a linked-structure binary tree
Searching takes O(log n) time
height of tree is O(log n), no restructures needed
Insertion takes O(log n) time
initial find is O(log n)
restructuring up the tree, maintaining heights is O(log n)
Removal takes O(log n) time
AVL Trees 31