1. Divide and Conquer

2. Sorting algorithms
Insertion, selection and bubble sort have quadratic worst-case performance
The faster comparison based algorithm ?
O(nlogn)
Mergesort and Quicksort

3. Sorting Algorithms
Courtesy of

4. Divide and Conquer Divide: a big problem into smaller problems
Conquer: smaller problems recursively
Combine: the solutions of smaller problems
“Big problems are hard to solve, but smaller ones are much easier”

5. Example a problem of size n subproblem 1 subproblem 2 of smaller size
of size smaller size
subproblem 1
of smaller size
a solution to
the original problem
a problem of size n

6. Sub Problem 1 a subproblem of size n’ sub_subproblem 1
of size smaller size
sub_subproblem 1
of smaller size
a solution to
Sub_subproblem 1
the subproblem
a subproblem of size n’

7. Example: Binary Search
Divide: a list into two smaller lists
Conquer: search one smaller list recursively
Combine: trivial

8. Sorting
Sorting takes an unordered collection and makes it an ordered one. 77 42 35 12
101 5 5 12 35 42 77
101

9. Example: Merge Sort Divide: a list of n into two smaller lists
(1) a list of n/2 numbers
(2) a list of n/2 numbers
Conquer: sort each smaller list recursively
Sort list (1) recursively
Sort list (2) recursively
Combine: merge the results of (1) and (2)

10. Merge Sort

11. Merge Sort Algorithm MergeSort(A) if A’s size > 1
Divide array A in halves
Call MergeSort on first half.
Call MergeSort on second half.
Merge two results (combine).

12. Merge Sort Algorithm MergeSort(A, l, r) if l < r:
m = (l + r)/2 //Divide
MergeSort(A,l,m) //Conquer MergeSort(A,m+1,r)//Conquer Merge(A,l,m,r)// Combine

13. How to Merge? Merge the sub-problem solutions together:
Compare the sub-array’s first elements
Remove the smallest element and put it into the result array
Continue the process until all elements have been put into the result array 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 98

14. 98 23 45 14 6 67 33 42

15. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42

16. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14

17. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23

18. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23
Merge

19. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 23
Merge

20. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 23 98
Merge

21. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98

22. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98
Merge

23. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14
Merge

24. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45
Merge

25. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45
Merge

26. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45 14
Merge

27. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45 14 23
Merge

28. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45 14 23 45
Merge

29. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 98 23 45 14 23 98 14 45 14 23 45 98
Merge

30. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 23 98 14 45 14 23 45 98

31. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 23 98 14 45 14 23 45 98

32. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 23 98 14 45 14 23 45 98
Merge

33. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 23 98 14 45 6 14 23 45 98
Merge

34. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 23 98 14 45 6 67 14 23 45 98
Merge

35. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 14 23 45 98

36. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 14 23 45 98
Merge

37. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 14 23 45 98
Merge

38. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98
Merge

39. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98
Merge

40. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6
Merge

41. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33
Merge

42. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42
Merge

43. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67
Merge

44. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67
Merge

45. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6
Merge

46. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14
Merge

47. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23
Merge

48. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33
Merge

49. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33 42
Merge

50. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33 42 45
Merge

51. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67
Merge

52. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 98
Merge

53. 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 98 23 45 14 6 67 33 42 23 98 14 45 6 67 33 42 14 23 45 98 6 33 42 67 6 14 23 33 42 45 67 98

54. 98 23 45 14 6 67 33 42 6 14 23 33 42 45 67 98

55. Summary Divide the unsorted collection into two
Until the sub-arrays only contain one element
Then merge the sub-problem solutions together

56. Time Complexity? Recurrence Equation

57. Merge Sort Algorithm
MergeSort(A, l, r) if l < r: m = (l + r)/2 //Divide MergeSort(A,l,m) //Conquer MergeSort(A,m+1,r)//Conquer Merge(A,l,m,r)// Combine

58. Time Complexity Recurrence equation: Assume n is a power of 2
c if n=1
T(n) =
2T(n/2) + c2n if n>1, n=2k

59. Recursion Tree

60. Binary Search Example

61. Merge Sort Apply divide-and-conquer to sorting problem
Problem: Given n elements, sort elements into non-decreasing order
Divide-and-Conquer:
If n=1 terminate (every one-element list is already sorted)
If n>1, partition elements into two or more sub-collections; sort each; combine into a single sorted list
How do we partition?

62. Partitioning - Choice 1
First n-1 elements into set A, last element set B
Sort A using this partitioning scheme recursively
B already sorted
Combine A and B using method Insert() (= insertion into sorted array)
Leads to recursive version of InsertionSort()
Number of comparisons: O(n2)
Best case = n-1
Worst case = O(n2)

63. Partitioning - Choice 2
Put element with largest key in B, remaining elements in A
Sort A recursively
To combine sorted A and B, append B to sorted A
Use Max() to find largest element  recursive SelectionSort()
Use bubbling process to find and move largest element to right-most position  recursive BubbleSort()
All O(n2)

64. Partitioning - Choice 3 Let’s try to achieve balanced partitioning
A gets n/2 elements, B gets rest half
Sort A and B recursively
Combine sorted A and B using a process called merge, which combines two sorted lists into one
O(n log n)

65. Master Theorem