Introduction to Algorithms: Divide-n-Conquer Algorithms

Introduction to Algorithms Divide and Conquer Binary search Powering a number Fibonacci numbers Matrix multiplication Strassen’s algorithm CS 421 - Analysis of Algorithms

Divide-and-Conquer Design Divide the problem (instance) into sub-problems. Conquer the sub-problems by solving them recursively. Combine sub-problem solutions. CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Merge Sort Divide: Trivial. Conquer: Recursively sort 2 sub-arrays. Combine: Linear-time merge. CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Merge Sort Divide: Trivial. Conquer: Recursively sort 2 subarrays. Combine: Linear-time merge. T(n) = 2 T(n/2) + (n) # sub-problems sub-problem size work dividing and combining CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Master Theorem T(n) = a T(n/b) + f (n) Case 1: 𝑎 <𝑏 𝑑 Solution: T(n) ∈ ( 𝑛 𝑑 ) . Case 2: 𝑎 =𝑏 𝑑 Solution: T(n) = ( 𝑛 𝑑 log n) . Case 3: 𝑎 >𝑏 𝑑 Solution: T(n) ∈ ( 𝑛 𝑙𝑜𝑔𝑏𝑎 ) . CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Master Theorem T(n) = a T(n/b) + f (n) Case 1: 𝑎 <𝑏 𝑑 Solution: T(n) ∈ ( 𝑛 𝑑 ) . Case 2: 𝑎 =𝑏 𝑑 Solution: T(n) = ( 𝑛 𝑑 log n) . Case 3: 𝑎 >𝑏 𝑑 Solution: T(n) ∈ ( 𝑛 𝑙𝑜𝑔𝑏𝑎 ) . Merge sort: T(n) = 2 T(n/2) + (n) a = 2, b = 2 , f(n) ∈ (n), d = 1 CASE 2  T(n) = (n1 log n) = (n log n). CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Binary Search Find an element in a sorted array: Divide: Check middle element. Conquer: Recursively search 1 sub-array. Combine: Trivial. Example: Find 9 3 5 7 8 9 12 15 CS 421 - Analysis of Algorithms

Recurrence for Binary Search T(n) = 1 T(n/2) + (1) # sub-problems sub-problem size work dividing and combining CS 421 - Analysis of Algorithms

Recurrence for Binary Search T(n) = 1 T(n/2) + (1) # sub-problems sub-problem size work dividing and combining nlogba = nlog21 = n0 = 1  CASE 2 (k = 0)  T(n) = (lg n) . CS 421 - Analysis of Algorithms

Recurrence for Binary Search T(n) = 1 T(n/2) + (1) # subproblems subproblem size work dividing and combining a = 1, b = 2 , f(n) ∈ (1), d = 0 CASE 2  T(n) = (n0 log n) = (log n). CS 421 - Analysis of Algorithms

Calculating Powers of a Number Problem: Compute an, where n  N. Naive algorithm: (n). CS 421 - Analysis of Algorithms

Calculating Powers of a Number Problem: Compute an, where n  N. Naive algorithm: (n). Divide-and-conquer algorithm: a n/2  a n/2 a (n–1)/2  a (n–1)/2  a if n is even; if n is odd. an = CS 421 - Analysis of Algorithms

Calculating Powers of a Number Problem: Compute an, where n  N. Naive algorithm: (n). Divide-and-conquer algorithm: a n/2  a n/2 a (n–1)/2  a (n–1)/2  a if n is even; if n is odd. an = T(n) = T(n/2) + (1)  T(n) = (lg n) . CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Fibonacci Numbers 0 if n = 0 1 if n = 1 Fn–1 + Fn–2 if n  2. 0 1 1 2 3 5 8 13 21 34 … Recursive definition: Fn = CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Fibonacci Numbers 0 if n = 0 1 if n = 1 Fn–1 + Fn–2 if n  2. 0 1 1 2 3 5 8 13 21 34 … Recursive definition: Fn = Naive recursive algorithm: ( n) (exp. time) where  = (1 is the golden ratio. 5)/ 2 CS 421 - Analysis of Algorithms

Computing Fibonacci Numbers Bottom-up: Compute F0, F1, F2, …, Fn in order, forming each number by summing the two previous. Running time: (n). Naive recursive squaring: Fn =  n/ 5 rounded to the nearest integer. Recursive squaring: (lg n) time. This method is unreliable, since floating-point arithmetic is prone to round-off errors. CS 421 - Analysis of Algorithms

CS 421 - Analysis of Algorithms Conclusion Divide and conquer is just one of several powerful techniques for algorithm design. Divide-and-conquer algorithms can be analyzed using recurrences and the master method (so practice this math). The divide-and-conquer strategy often leads to efficient algorithms. CS 421 - Analysis of Algorithms