Chapter 4: Solution of recurrence relationships Techniques: Substitution: Guess answer, prove by induction Tree analysis: Good way to develop a guess Master theorem: Recipe when T(n) = aT(n\b) + f(n) with a >1, b >1 and f(n) a specified function Example: use substitution method to find run time for merge sort

When n is a power of 2 The recurrence for merge sort (worst case) is T(n) = 2 n=2 T(n) = 2T(n/2) +n n=2k k>1 Proved by induction on integers (HW1) T(n) = nlgn= 2k lg2k = k2k

Worst case merge sort for any n T(n) = constant n=1 T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1 Look for order of growth instead of analytic solution We can ignore floor and ceiling and replace Q(n) by cn with c > 0 Recurrence becomes T(n) = 2T(n/2) + cn Prove T(n) = Q(nlgn) by substitution method

Based on Theorem 3.1, T(n) = Q(nlgn) iff T(n) = O(nlgn) and T(n) = W(nlgn) Start by proving T(n) = O(nlgn) is a solution of T(n) = 2T(n/2) + cn if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big O

For a more general solution start with the string on inequalities T(n) < dnlg(n)-dn+cn < dnlg(n), which says if T(n) < dnlg(n)-dn+cn and dnlg(n)-dn+cn < dnlg(n) then T(n) < dnlg(n), which is what we need in order to show that T(n)=O(nlg(n)). But T(n) < dnlg(n)-dn+cn < dnlg(n) also put a condition on d by the inequality dnlg(n)-dn+cn < dnlg(n) Solving this inequality for d yields d > c > 0, which says that d = c is OK as the constant in the definition of big O but any d bigger than c is also OK

Finish by proving T(n) = W(nlgn) is a solution of T(n) = 2T(n/2) + cn Assume T(n/2)= W((n/2)lg(n/2)) if -dn + cn = 0; therefore d = c > 0 will work as the constant in the definition of big W

For a more general solution start with the string on inequalities T(n) > dnlg(n)-dn+cn > dnlg(n), which says if T(n) > dnlg(n)-dn+cn and dnlg(n)-dn+cn > dnlg(n) then T(n) > dnlg(n), which is what we need in order to show that T(n)=W(nlg(n)). But T(n) > dnlg(n)-dn+cn > dnlg(n) also put a condition on d by the inequality dnlg(n)-dn+cn > dnlg(n) Solving this inequality for d yields 0 < d < c, which says that d = c is OK as the constant in the definition of big W but any d between 0 and c is also OK

Sometimes, even with correct guess, substitution does not work To prove Big O in these cases, try subtracting a lower-order term in your assumption Example: T(n) = 8T(n/2) + Q(n2) Show T(n) = O(n3)

The string of inequalities T(n) < dn3 + cn2 < dn3 cannot be true because cn2 > 0 The substitution T(n/2) = O((n/2)3 will not work in this case Try “subtract off a lower order term” assume T(n/2) < d[(n/2)3 – (n/2)2] if d = c > 0

if we assume T(n/2) < d[(n/2)3 – (n/2)2] then we most show T(n) < d[n3-n2] < d[n3-n2] implies T(n) < d[n3-n2] and 0 < c < d (i.e. any d > c satisfies the definition of big O) if d = c > 0

Substitution T(n/2) = W((n/2)3) works for proving T(n) = W(n3) is a solution of T(n) = 8T(n/2) + cn2 with c > 0 Assuming T(n/2) = W((n/2)3) means there exist d>0 such that T(n/2) > d(n/2)3 which implies T(n) > 8d(n/2)3 + cn2 The string of inequalities T(n) > dn3 + cn2 > dn3 can be true because cn2 > 0, which means that any d > 0 is OK as the constant in the definition of big W; hence T(n) = W(n3)

Sometimes a change of variable helps Example: T(n) = 2T(n1/2) + lg(n)

Change of variable in substitution method

Quiz Friday, 2-6-15 Evaluate and bound sums prove formulas by induction on integers convert one sum into another bound by induction bound every term bound by integration

CptS 450 Spring 2015 [All problems are from Cormen et al, 3rd Edition]   Homework Assignment 4: due 2/11/15 1. ex 4.4-7 on page 93 (tree analysis and substitution) 2. problem 4-3f on page 108 by tree analysis and substitution

Solution of recurrence by tree analysis

Tree analysis At each node write overhead per recurrence usually a function of level index Cost of leaves = number of leaves Calculate istop Sum level cost for i = 0 to istop -1 Add cost of leaves If tree analysis is ambiguous, verify by substitution Example: T(n) = 3T(|_n/4_|) + Q(n2) ~ 3T(n/4) + cn2

T(n)=3T(n/4)+cn2 cost of level i istop = ? How many leaves?

As indicated in previous slide, levels are indexed from zero ISTOP is the index of leaves At ISTOP divide and conquer has bottomed out with subgroups of size n0 Could evaluate cost of level using finite geometric sum series Bounding by infinite series easier

Test guess from tree analysis by substitution

Test guess from tree analysis by substitution Guess is upper bound only Did not evaluate sum over levels < dn2 solving right-most inequality for d yields d > (16/13)c > 0 for all n > 1 there exists d > 0 such that 0 < T(n) < dn2 all n > 1 hence, T(n) = O(n2) = 1

Binary tree with complicated level cost T(n) = 2T(n/2 +17) +n Expect T(n) = nlgn Why?

Note: sum cannot be bounded by geometric series. Why?

HW 4 due date changed Homework Assignment 4: due 2/13/15 1. ex 4.4-7 on page 93 (tree analysis and substitution) 2. problem 4-3f on page 108 by tree analysis and substitution Quiz Friday 2/13 Substitution method set up (remove floors, ceiling, asymptotic notation) assumption implementation of assumption constants and n0

Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

Imbalanced tree analysis: T(n) = T(n/3) + T(2n/3) + cn

Imbalanced tree analysis continued Test guess by substitution < dnlgn solve < 0 for d d > c/(lg3-2/3) > 0 = all n > 0

Skinny Trees

Skinny tree leaf = no2 no2 no2 See text p1147 no2 + Q(n3)

substitution < cn3 T(n) < cn3 - 6cn2 + 12cn -8c + n2 < cn3 Solve - 6cn2 + 12cn -8c + n2 < 0 for c In this case, -c6n2+12cn-8c+n2 = 0 leads to c = f(n) n2 (1 – 6c) + 12cn - 8c < 0 Look for a combination of c and n0 that works in definition of big O n2 (1 – 6c) will be the dominate term at large n Try c = 1 and see how large n must be -5n2 + 12n - 8 < 0 true for all n>1 c = n0 = 1 work in definition of big O < cn3 = 0

T(n) > cn3 - 6cn2 + 12cn -8c + n2 > cn3 Solve - 6cn2 + 12cn -8c + n2 > 0 for c Look for a combination of c and n0 that works in definition of big W c = 1/6 eliminates n2 term 2n – 4/3 > 0 true for all n>1 c = 1/6 and n0 = 1 work in definition of big W

Master Theorem: (statement on p94) “cook-book” method to solve T(n) = aT(n/b) + f(n) when a > 1, b > 1, and f(n) asymptotically positive Ignore floors and ceiling do not effect validity (section 4.6.2 p103) At large n, compare f(n) to Case 1: is polynomially larger than f(n) at large n Conclusion: T(n) = Q( We can subtract e > 0 from logb(a) and is still an upper bound on f(n) There exist e > 0 such that f(n) = O(

is asymptotically the same as f(n) Case 2: is asymptotically the same as f(n) Conclusion: ) f(n) = Q( lgn) = Q(f(n)lgn) T(n) = Q( Case 3: f(n) is polynomially larger than There exist e > 0 such that f(n) = W( We can add e > 0 from logb(a) and is still a lower bound on f(n) f(n) is “regular” af(n/b) < cf(n) for some c < 1 and n > n0 Conclusion: T(n) = Q(f(n))

Cpt S 450 Spring 2015 [All problems are from Cormen et al, 3rd Edition] Homework Assignment 5: due 2/18/15 Problems 4-1a, c and e on page 107 by master method

When is Master Theorem not applicable 1. T(n) does not have the correct form 2. f(n) falls between cases 1 and 2 (cannot find e > 0 required by case 1) 3. f(n) falls between cases 2 and 3 (cannot find e > 0 required by case 3) 4. Regularity condition fails in case 3

Prove that, in case 3, polynomial f(n) is regular

Prove that, in case 3, polynomial f(n) is regular Given f(n) = nk and f(n) = W( )

Quiz Friday 2-20-15 Tree analysis size of tree (istop) cost of leaves (number of leaves) cost of ith level total cost of levels total cost order of growth

Proof of Master Theorem MT applies to T(n) = aT(n/b) + f(n) with a>1 and b>1 Analyze master recurrence for n = bk (no floors or ceilings) 1. tree analysis → solution as a sum 2. bound sum → 3 cases B. Extend solution to any n > 0 Apply techniques related to floors and ceilings

Tree analysis master theorem T(n) = aT(n/b) + f(n) bi bi

T(n) = aT(n/b) + f(n) bi bi Case 1: cost of leaves dominates Case 2: cost evenly distributed over levels and leaves Case 3: cost of root dominates: T(n) = Q(f(n))

T(n) = aT(n/b) + f(n) When n is not a power of b recurrence becomes T(n) = constant n=1 T(n) = T(floor(n/2)) + T(ceiling(n/2)) + Q(n) n>1 Section 4.6.2 show that floors and ceiling do not affect the tree analysis

Quiz Friday, 2-28-14 Master Theorem Which case applies Relationship between f(n) and nlogb(a) in asymptotic notation Acceptable values of e, cases 1 and 3 Regularity of f(n), case 3

Iteration on recurrence T(n)=4T(n/2)+cn T(n) = cn + 4T(n/2) T(n) = cn + 4(cn/2 + 4T(n/4)) T(n) = cn +2cn +16(cn/4 + 4T(n/8)) T(n) = cn +2cn + 4cn +64(cn/8+4T(n/16)) T(n) = cn +2cn + 4cn + 8cn +256(cn/16 + 4T(n/32)) Level = 0 1 2 3 … Cost of level k is 2kcn T(n)=Sk=0 to istop 2kcn=cn(2lg(n)+1-1)/(2-1)=cn(2nlg(2)-1) Compare to solution by master’s theorem

More iteration examples T(n)=T(n-2)+n2 T(n)=T(n/2)+T(n/4)+T(n/8)