1. Divide & Conquer Sorting
Merge Sort
Divide & Conquer Sorting
7 2  
7  2  2 7
9  4  4 9
7  7
2  2
9  9
4  4
Divide and Conquer Sorting

2. Divide and Conquer Sorting
Divide-and conquer is a general algorithm design paradigm:
Divide: divide the input data S in two disjoint subsets S1 and S2. Continue division until problem is ‘tractable’.
Compute: solve the subproblems associated with S1 and S2
Conquer: combine the solutions for S1 and S2 into a solution for S
Mergesort and Quicksort are two examples.
Divide and Conquer Sorting

3. Divide and Conquer Sorting
Merge-Sort
Merge-sort on an input sequence S with n elements consists of three steps:
Divide: partition S into two sequences S1 and S2 of about n/2 elements each, until only 1 element remains in each S1 and S2
Compute: sort S1 and S2
Conquer: merge S1 and S2 into a unique sorted sequence
Divide and Conquer Sorting

4. Merging Two Sorted Sequences
The conquer step of merge-sort consists of merging two sorted sequences A and B into a sorted sequence S containing all of the elements of A and B
Merging two sorted sequences, each with n/2 elements takes O(n) time. Why?
Each element must be ‘scanned’ before being merged!
Divide and Conquer Sorting

5. Divide and Conquer Sorting
Execution Example
Partition
   
7 2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

6. Execution Example (cont.)
Continue partition until problem size is 1
 
7 2  9 4  
7 2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

7. Execution Example (cont.)
Continue partition until problem size is 1
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

8. Execution Example (cont.)
‘Base’ case of 1!
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

9. Execution Example (cont.)
Here, too, base case of 1!
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

10. Execution Example (cont.)
Merge
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

11. Execution Example (cont.)
Recursive call, …, base case, merge
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

12. Divide and Conquer Sorting
Now you do the right hand side
Divide and Conquer Sorting

13. Execution Example (cont.)
Merge
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

14. Execution Example (cont.)
Recursive call, …, merge, merge
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

15. Execution Example (cont.)
Merge
 
7 2  9 4  
7  2  2 7
9 4  4 9
3 8  3 8
6 1  1 6
7  7
2  2
9  9
4  4
3  3
8  8
6  6
1  1
Divide and Conquer Sorting

16. Analysis of Merge-Sort
The “height” h of the merge-sort tree is O(log n)
This is a consequence of the dividing the problem in 2 at each step,
The overall amount of work done at each merge i is O(n)
Thus, the total running time of merge-sort is O(n log n)
size n …
Divide and Conquer Sorting

17. Running times in practice
Home PC executes 108 comparisons/sec
Supercomputer does 1012 comps/sec.
Insertion Sort (n2) Mergesort (n log n)
How do you think they got these numbers
Good algorithms are better than a supercomputer!
Computer
thousand
million
billion
home
instant
2.8 hours
317 years
super
1 second
1.6 weeks
thousand
million
billion
instant
1 sec
18 mins
Divide and Conquer Sorting

18. Introducing Quicksort
Mergesort
Sorts “at the end”
Partition is left/right
i.e. merge is sorting step
Quicksort
Sorts “at beginning”
Partition is < or > less than or greater than
i.e. partition is sorting step!
Divide and Conquer Sorting

19. Divide and Conquer Sorting
Quick-Sort
Quick-sort is a sorting algorithm based on the divide-and-conquer:
Divide: pick a random element x (called pivot) and partition S into
L elements less than x
E elements equal x
G elements greater than x
Re-apply: above on L and G
Conquer: join L, E and G x x L G E x
Divide and Conquer Sorting

20. Divide and Conquer Sorting
Execution Example
Pivot selection 
< 6
> 6  
2  2
9 4  4 9
3  3
8  8
9  9
4  4
Divide and Conquer Sorting

21. Execution Example (cont.)
Partition, repeat pivot selection.   
< 2
>2
2  2
9 4  4 9
3  3
8  8
9  9
4  4
Divide and Conquer Sorting

22. Execution Example (cont.)
Partition, select pivot, base case
   
1  1
9 4  4 9
3  3
8  8
9  9
4  4
Divide and Conquer Sorting

23. Execution Example (cont.)
partition, select pivot, smallest case, join   
1  1
4 3  3 4
3  3
8  8
< 3
> 3
9  9
4  4
Divide and Conquer Sorting

24. Divide and Conquer Sorting
Now you do the right hand side
Divide and Conquer Sorting

25. Execution Example (cont.)
Partition, pivot selection   
1  1
4 3  3 4
8  8
9  9
9  9
4  4
Divide and Conquer Sorting

26. Execution Example (cont.)
Partition, …, select pivot, smallest case   
< 7
> 7
1  1
4 3  3 4
8  8
9  9
9  9
4  4
Divide and Conquer Sorting

27. Execution Example (cont.)
Join, join   
1  1
4 3  3 4
8  8
9  9
9  9
4  4
Divide and Conquer Sorting

28. ‘Flat’ ways to view Quicksort
(sound)
Divide and Conquer Sorting

29. Worst-case Running Time
Divide and Conquer Sorting

30. Worst-case Running Time
The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element
One of L and G has size n - 1 and the other has size 0
The running time is proportional to the sum
n + (n - 1) + …
Thus, the worst-case running time of quick-sort is O(n2)
Divide and Conquer Sorting

31. Worst-case Running Time
Merge Sort
Worst-case Running Time
The worst case for quick-sort occurs when the pivot is the unique minimum or maximum element
One of L and G has size n - 1 and the other has size 0
The running time is proportional to the sum
n + (n - 1) + …
Thus, the worst-case running time of quick-sort is O(n2)
depth
time n 1
n - 1 … …
Divide and Conquer Sorting

32. Divide and Conquer Sorting
Expected Running Time
Consider a call of quick-sort on a sequence of size s
Good case: the sizes of L and G are roughly the same
Bad case: the sizes of L and G differ greatly (not well-divided)
A case is good with probability 1/2
1/2 of the possible pivots cause good calls:
 1 1
Good case
Bad case
Bad pivots
Good pivots
Bad pivots
Divide and Conquer Sorting

33. Running times in practice
Home PC executes 108 comparisons/sec
Supercomputer does 1012 comps/sec.
Insertion Sort (N2) Mergesort (N log N)
Quicksort (N log N)
Great algorithms better than good ones!
Computer
thousand
million
billion
home
instant
2.8 hours
317 years
super
1 second
1.6 weeks
thousand
million
billion
instant
1 sec
18 mins
thousand
million
billion
instant
0.3 sec
6 mins
Divide and Conquer Sorting