Download presentation
Presentation is loading. Please wait.
Published byKory Harmon Modified over 9 years ago
1
1 Computer Algorithms Lecture 7 Master Theorem Some of these slides are courtesy of D. Plaisted, UNC and M. Nicolescu, UNR
2
Recurrence Relations Equation or an inequality that characterizes a function by its values on smaller inputs. Solution Methods (Chapter 4) –Substitution Method. –Recursion-tree Method. –Master Method. Recurrence relations arise when we analyze the running time of iterative or recursive algorithms. –Ex: Divide and Conquer. T(n) = (1)if n c T(n) = a T(n/b) + D(n) + C(n) otherwise 2
3
Master Method Based on the Master theorem. “Cookbook” approach for solving recurrences of the form T(n) = aT(n/b) + f(n) a 1, b > 1 are constants. –a subproblems of size n/b are solved recursively, each in time T(n/b) f(n) is asymptotically positive. –An asymptotically positive function is one that is positive for all –sufciently large n. –f(n) is the cost of dividing the problem and combining the results n/b may not be an integer, but we ignore floors and ceilings. Why? Requires memorization of three cases. Idea: compare f(n) with –f(n) is asymptotically smaller or larger than by a polynomial factor n –f(n) is asymptotically equal with 3
4
Recursion tree view f(n)f(n) af(n/b) a 2 f(n/b 2 ) (n log b a ) Total: There are leaves Split problem into a parts at log b n levels. 4
5
The master method The master method applies to recurrences of the form T(n) = a T(n/b) + f (n), where a 1, b > 1, and f is asymptotically positive. 5
6
Why ? Assume n = b k k = log b n At the end of iterations, i = k: to solve the recurrence, we need only to characterize the dominant term
7
Master Theorem Theorem 4.1 Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b . T(n) can be bounded asymptotically in three cases: 1.If f(n) = O(n log b a– ) for some constant > 0, then T(n) = ( n log b a ). 2.If f(n) = (n log b a ), then T(n) = ( n log b a lg n). 3.If f(n) = (n log b a+ ) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). Theorem 4.1 Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b . T(n) can be bounded asymptotically in three cases: 1.If f(n) = O(n log b a– ) for some constant > 0, then T(n) = ( n log b a ). 2.If f(n) = (n log b a ), then T(n) = ( n log b a lg n). 3.If f(n) = (n log b a+ ) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). Whether the problem cost is dominated by the root or by the leaves 7
8
Master Theorem – What it means? Case 1: If f(n) = O(n log b a– ) for some constant > 0, then T(n) = ( n log b a ). –n log b a = a log b n : Number of leaves in the recursion tree. –f(n) = O(n log b a– ) Sum of the cost of the nodes at each internal level asymptotically smaller than the cost of leaves by a polynomial factor. –Cost of the problem dominated by leaves, hence cost is ( n log b a ). 8
9
Master Theorem – What it means? Case 2: If f(n) = (n log b a ), then T(n) = ( n log b a lg n). –n log b a = a log b n : Number of leaves in the recursion tree. –f(n) = (n log b a ) Sum of the cost of the nodes at each level asymptotically the same as the cost of leaves. –There are (lg n) levels. –Hence, total cost is ( n log b a lg n). 9
10
Master Theorem – What it means? Case 3: If f(n) = (n log b a+ ) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have a·f(n/b) c f(n), then T(n) = (f(n)). –n log b a = a log b n : Number of leaves in the recursion tree. –f(n) = (n log b a+ ) Cost is dominated by the root. Cost of the root is asymptotically larger than the sum of the cost of the leaves by a polynomial factor. –Hence, cost is ( f(n)). 10
11
Summary Assume n = b k k = log b n At the end of iterations, i = k: to solve the recurrence, we need only to characterize the dominant term Case 1: Running time dominated by cost at leaves –If f(n) is dominated by : T(n) = ( ) Case 3: Running evenly distributed throughout the tree –If f(n) dominates : T(n) = (f(n)) Case 2: Running time dominated by cost at root –If f(n) = ( ): T(n) 11
12
Strategy Extract a, b, and f(n) from a given recurrence Determine Compare f(n) and asymptotically Determine appropriate MT case, and apply 12
13
Master Method – Examples T(n) = 16T(n/4)+n –a = 16, b = 4, n log b a = n log 4 16 = n 2. –f(n) = n = O(n log b a- ) = O(n 2- ), where = 1 Case 1. –Hence, T(n) = (n log b a ) = (n 2 ). T(n) = T(3n/7) + 1 – a = 1, b=7/3, and n log b a = n log 7/3 1 = n 0 = 1 – f(n) = 1 = (n log b a ) Case 2. –Therefore, T(n) = (n log b a lg n) = (lg n) 13
14
Master Method – Examples T(n) = 3T(n/4) + n lg n – a = 3, b=4, thus n log b a = n log 4 3 = O(n 0.793 ) – f(n) = n lg n = (n log 4 3 + ) where 0.2 Case 3. –Therefore, T(n) = (f(n)) = (n lg n). T(n) = 2T(n/2) + n lg n – a = 2, b=2, f(n) = n lg n, and n log b a = n log 2 2 = n – f(n) is asymptotically larger than n log b a, but not polynomially larger. The ratio lg n is asymptotically less than n for any positive . Thus, the Master Theorem doesn’t apply here. 14
15
Master Theorem Summarized Given a recurrence of the form The master method cannot solve every recurrence of this form; there is a gap between cases 1 and 2, as well as cases 2 and 3 15
16
f (n/b) Idea of master theorem f (n/b) (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a (1) C ASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. (n log b a ) 16
17
f (n/b) Idea of master theorem f (n/b) (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a (1) C ASE 2: (k = 0) The weight is approximately the same on each of the log b n levels. (n log b a lg n) 17
18
f (n/b) Idea of master theorem f (n/b) (1) … Recursion tree: … f (n)f (n) a f (n/b 2 ) … a h = log b n f (n)f (n) a f (n/b) a 2 f (n/b 2 ) … n log b a (1) C ASE 3: The weight decreases geometrically from the root to the leaves. The root holds a constant fraction of the total weight. ( f (n)) 18
19
More Examples 19
20
Master Method Case 1 T(n) = a T(n/b) + f(n) f(n) = O(n log b a - ) for some >0 T(n) = (n log b a ) T(n) = (n lg 7 ) cn 2 = ? O(n log b a - ) = O(n log 2 7 - ) O(n 2.8 - ) Yes, for any 0.8. T(n) = 7T(n/2) + cn 2 a=7, b=2 20
21
Master Method Case 2 T(n) = a T(n/b) + f(n) f(n) = (n log b a ) T(n) = (n log b a lg n) T(n) = 2T(n/2) + cna=2, b=2 E.g., mergesort. cn = ? (n log b a ) = (n log 2 2 ) = (n) Yes. T(n) = (n lg n) 21
22
Master Method Case 3 T(n) = a T(n/b) + f(n) f(n) = (n log b a + ) for some >0 and a f(n/b) c f(n) for some c<1 and all large enough n T(n) = (f(n)) T(n) = 4 T(n/2) + n 3 a=4, b=2 n 3 = ? (n log b a + ) = (n log 2 4 + ) = (n 2 + ) Yes, for any 1. 4(n/2) 3 = ½ n 3 ? cn 3 Yes, for any c ½. I.e., is the constant factor shrinking? T(n) = (n 3 ) 22
23
Master Method Case 4 T(n) = a T(n/b) + f(n) None of previous apply. Master method doesn’t help. T(n) = 4T(n/2) + n 2 /lg na=4, b=2 Case 1? n 2 /lg n = ? O(n log b a - ) = O(n log 2 4 - ) = O(n 2 - ) = O(n 2 /n ) No, since lg n is asymptotically less than n . Thus, n 2 /lg n is asymptotically greater than n 2 /n . 23
24
Examples Ex. T(n) = 4T(n/2) + n a = 4, b = 2 n log b a = n 2 ; f (n) = n. CASE 1: f (n) = O(n 2 – ) for = 1. T(n) = (n 2 ). Ex. T(n) = 4T(n/2) + n 2 a = 4, b = 2 n log b a = n 2 ; f (n) = n 2. CASE 2: f (n) = (n 2 lg 0 n), that is, k = 0. T(n) = (n 2 lg n). 24
25
Examples Ex. T(n) = 4T(n/2) + n 3 a = 4, b = 2 n log b a = n 2 ; f (n) = n 3. CASE 3: f (n) = (n 2 + ) for = 1 and 4(n/2) 3 cn 3 (reg. cond.) for c = 1/2. T(n) = (n 3 ). 25
26
Master Method Case 4 T(n) = a T(n/b) + f(n) None of previous apply. Master method doesn’t help. T(n) = 4T(n/2) + n 2 /lg na=4, b=2 Case 2? n 2 /lg n = ? (n log b a ) = (n log 2 4 ) = (n 2 ) No. 26
27
Master Method Case 4 T(n) = a T(n/b) + f(n) None of previous apply. Master method doesn’t help. T(n) = 4T(n/2) + n 2 /lg na=4, b=2 Case 3? n 2 /lg n = ? (n log b a + ) = (n log 2 4 + ) = (n 2 + ) No, since 1/lg n is asymptotically less than n . 27
28
Examples Binary-search(A, p, r, s): q (p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s) Binary-search(A, p, r, s): q (p+r)/2 if A[q]=s then return q else if A[q]>s then Binary-search(A, p, q-1, s) else Binary-search(A, q+1, r, s) 28
29
Examples (2) 29
30
Examples (3) 30
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.