Chap. 7 (c) , Michael P. Frank1 Chapter 7: Advanced Counting Techniques

Chap. 7 (c) , Michael P. Frank2 §7.1: Recurrence Relations A recurrence relation (R.R., or just recurrence) for a sequence {a n } is an equation that expresses a n in terms of one or more previous elements a 0, …, a n−1 of the sequence, for all n≥n 0.A recurrence relation (R.R., or just recurrence) for a sequence {a n } is an equation that expresses a n in terms of one or more previous elements a 0, …, a n−1 of the sequence, for all n≥n 0. –A recursive definition, without the base cases. A particular sequence (described non-recursively) is said to solve the given recurrence relation if it is consistent with the definition of the recurrence.A particular sequence (described non-recursively) is said to solve the given recurrence relation if it is consistent with the definition of the recurrence. –A given recurrence relation may have many solutions. § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank3 Recurrence Relation Example Consider the recurrence relationConsider the recurrence relation a n = 2a n−1 − a n−2 (n≥2). Which of the following are solutions? a n = 3n a n = 2 nWhich of the following are solutions? a n = 3n a n = 2 n a n = 5 § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank4 Example Applications Recurrence relation for growth of a bank account with P% interest per given period:Recurrence relation for growth of a bank account with P% interest per given period: M n = M n−1 + (P/100)M n−1 Growth of a population in which each organism yields 1 new one every period starting 2 periods after its birth.Growth of a population in which each organism yields 1 new one every period starting 2 periods after its birth. P n = P n−1 + P n−2 (Fibonacci relation) § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank5 Solving Compound Interest RR M n = M n−1 + (P/100)M n−1M n = M n−1 + (P/100)M n−1 = (1 + P/100) M n−1 = r M n−1 (let r = 1 + P/100) § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank6 Tower of Hanoi Example Problem: Get all disks from peg 1 to peg 2.Problem: Get all disks from peg 1 to peg 2. –Only move 1 disk at a time. –Never set a larger disk on a smaller one. Peg #1Peg #2Peg #3 § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank7 Hanoi Recurrence Relation Let H n = # moves for a stack of n disks.Let H n = # moves for a stack of n disks. Optimal strategy:Optimal strategy: –Move top n−1 disks to spare peg. (H n−1 moves) –Move bottom disk. (1 move) –Move top n−1 to bottom disk. (H n−1 moves) Note: H n = 2H n−1 + 1Note: H n = 2H n−1 + 1 § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank8 Solving Tower of Hanoi RR H n = 2 H n−1 + 1 § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank9 Finding Recurrence Relation Ex: Find a recurrence relation and give initial conditions for the number of bit strings of length n that do not have two consecutive 0s. How many such bit strings are there of length 5? § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank10 Codeword Enumeration Ex:Consider a string of decimal digits a valid codeword if it contains an even number of 0 digits. For example, is valid, whereas is not valid. Let a be the number of valid n-digit codewords. Find a recurrence relation for a. Ex:Consider a string of decimal digits a valid codeword if it contains an even number of 0 digits. For example, is valid, whereas is not valid. Let a n be the number of valid n-digit codewords. Find a recurrence relation for a n. § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank11 Catalan Numbers Ex: Find a recurrence relation for C, the number of ways to parenthesize the product of n+1 numbers, x, x,…, x, to specify the order of multiplication. For example, C = 5. Ex: Find a recurrence relation for C n, the number of ways to parenthesize the product of n+1 numbers, x 0, x 1,…, x n, to specify the order of multiplication. For example, C 3 = 5. § 7.1 – Recurrence Relations

Chap. 7 (c) , Michael P. Frank12 §7.2: Solving Recurrences A linear homogeneous recurrence of degree k with constant coefficients (“k- LiHoReCoCo”) is a recurrence of the form a n = c 1 a n−1 + … + c k a n−k, where the c i are all real, and c k ≠ 0.A linear homogeneous recurrence of degree k with constant coefficients (“k- LiHoReCoCo”) is a recurrence of the form a n = c 1 a n−1 + … + c k a n−k, where the c i are all real, and c k ≠ 0. The solution is uniquely determined if k initial conditions a 0 …a k−1 are provided.The solution is uniquely determined if k initial conditions a 0 …a k−1 are provided. General Solution Schemas § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank13 Solving LiHoReCoCos Basic idea: Look for solutions of the form a n = r n, where r is a constant.Basic idea: Look for solutions of the form a n = r n, where r is a constant. This requires the characteristic equation: r n = c 1 r n−1 + … + c k r n−k, i.e., r k − c 1 r k−1 − … − c k = 0This requires the characteristic equation: r n = c 1 r n−1 + … + c k r n−k, i.e., r k − c 1 r k−1 − … − c k = 0 The solutions (characteristic roots) can yield an explicit formula for the sequence.The solutions (characteristic roots) can yield an explicit formula for the sequence. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank14 Solving 2-LiHoReCoCos Consider an arbitrary 2-LiHoReCoCo: a n = c 1 a n−1 + c 2 a n−2Consider an arbitrary 2-LiHoReCoCo: a n = c 1 a n−1 + c 2 a n−2 It has the characteristic equation (C.E.): r 2 − c 1 r − c 2 = 0It has the characteristic equation (C.E.): r 2 − c 1 r − c 2 = 0 Thm. 1: If this CE has 2 roots r 1 ≠r 2, then a n = α 1 r 1 n + α 2 r 2 n for n≥0 for some constants α 1, α 2.Thm. 1: If this CE has 2 roots r 1 ≠r 2, then a n = α 1 r 1 n + α 2 r 2 n for n≥0 for some constants α 1, α 2. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank15 Example Solve the recurrence a n = a n−1 + 2a n−2 given the initial conditions a 0 = 2, a 1 = 7.Solve the recurrence a n = a n−1 + 2a n−2 given the initial conditions a 0 = 2, a 1 = 7. Solution:Solution: § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank16 Example Continued… To find α 1 and α 2, solve the equations for the initial conditions a 0 and a 1 :To find α 1 and α 2, solve the equations for the initial conditions a 0 and a 1 : Simplifying, we have the pair of equations: which we can solve easily by substitution: Final answer:Final answer: Check: {a n≥0 } = 2, 7, 11, 25, 47, 97 … § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank17 Example Find an explicit formula for the Fibonacci numbers.Find an explicit formula for the Fibonacci numbers. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank18 The Case of Degenerate Roots Now, what if the C.E. r 2 − c 1 r − c 2 = 0 has only 1 root r 0 ?Now, what if the C.E. r 2 − c 1 r − c 2 = 0 has only 1 root r 0 ? Theorem 2: Then, a n = α 1 r 0 n + α 2 nr 0 n, for all n≥0, for some constants α 1, α 2.Theorem 2: Then, a n = α 1 r 0 n + α 2 nr 0 n, for all n≥0, for some constants α 1, α 2. Ex:Ex: § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank19 k-LiHoReCoCos Consider a k-LiHoReCoCo:Consider a k-LiHoReCoCo: It’s C.E. is:It’s C.E. is: Thm.3: If this has k distinct roots r i, then the solutions to the recurrence are of the form:Thm.3: If this has k distinct roots r i, then the solutions to the recurrence are of the form: for all n≥0, where the α i are constants. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank20 Example Ex:Ex: § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank21 Degenerate k-LiHoReCoCos Suppose there are t roots r 1,…,r t with multiplicities m 1,…,m t. Then:Suppose there are t roots r 1,…,r t with multiplicities m 1,…,m t. Then: for all n≥0, where all the α are constants. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank22 Example Ex:Ex: § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank23 LiNoReCoCos Linear nonhomogeneous RRs with constant coefficients may (unlike LiHoReCoCos) contain some terms F(n) that depend only on n (and not on any a i ’s). General form:Linear nonhomogeneous RRs with constant coefficients may (unlike LiHoReCoCos) contain some terms F(n) that depend only on n (and not on any a i ’s). General form: a n = c 1 a n−1 + … + c k a n−k + F(n) The associated homogeneous recurrence relation (associated LiHoReCoCo). § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank24 Solutions of LiNoReCoCos A useful theorem about LiNoReCoCos:A useful theorem about LiNoReCoCos: –If a n = p(n) is any particular solution to the LiNoReCoCo –Then all its solutions are of the form: a n = p(n) + h(n), where a n = h(n) is any solution to the associated homogeneous RR § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank25 Example Find all solutions to a n = 3a n−1 +2n. Which solution has a 1 = 3?Find all solutions to a n = 3a n−1 +2n. Which solution has a 1 = 3? –Notice this is a 1-LiNoReCoCo. Its associated 1-LiHoReCoCo is a n = 3a n−1, whose solutions are all of the form a n = α3 n. Thus the solutions to the original problem are all of the form a n = p(n) + α3 n. So, all we need to do is find one p(n) that works. § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank26 Trial Solutions If the extra terms F(n) are a degree-t polynomial in n, you should try a degree-t polynomial as the particular solution p(n).If the extra terms F(n) are a degree-t polynomial in n, you should try a degree-t polynomial as the particular solution p(n). This case: F(n) is linear so try a n = cn + d.This case: F(n) is linear so try a n = cn + d. (for all n) (collect terms) So (for all n) (collect terms) So So is a solution. Check: a n≥1 = {−5/2, −7/2, −9/2, … }Check: a n≥1 = {−5/2, −7/2, −9/2, … } § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank27 Finding a Desired Solution From the previous, we know that all general solutions to our example are of the form:From the previous, we know that all general solutions to our example are of the form: Solve this for α for the given case, a 1 = 3: The answer isThe answer is § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank28 Example Ex: § 7.2 – Solving Recurrences

Chap. 7 (c) , Michael P. Frank29 §7.3: Divide & Conquer R.R.s Main points so far: Many types of problems are solvable by reducing a problem of size n into some number a of independent subproblems, each of size  n/b , where a  1 and b>1.Many types of problems are solvable by reducing a problem of size n into some number a of independent subproblems, each of size  n/b , where a  1 and b>1. The time complexity to solve such problems is given by a recurrence relation:The time complexity to solve such problems is given by a recurrence relation: –T(n) = a·T(  n/b  ) + g(n) § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank30 Divide+Conquer Examples Binary search: Break list into 1 sub- problem (smaller list) (so a=1) of size  n/2  (so b=2).Binary search: Break list into 1 sub- problem (smaller list) (so a=1) of size  n/2  (so b=2). –So T(n) = T(  n/2  )+c (g(n)=c constant) Merge sort: Break list of length n into 2 sublists (a=2), each of size  n/2  (so b=2), then merge them, in g(n) = Θ(n) time.Merge sort: Break list of length n into 2 sublists (a=2), each of size  n/2  (so b=2), then merge them, in g(n) = Θ(n) time. –So T(n) = 2T(  n/2  ) + cn (roughly, for some c) § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank31 Divide+Conquer Examples Finding the Maximum and Minimum: Break list into 2 sub-problem (smaller list) (so a=2) of size  n/2  (so b=2).Finding the Maximum and Minimum: Break list into 2 sub-problem (smaller list) (so a=2) of size  n/2  (so b=2). –So T(n) = 2T(  n/2  )+2 (g(n)=2 constant) § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank32 Fast Multiplication Example The ordinary grade-school algorithm takes Θ(n 2 ) steps to multiply two n-digit numbers.The ordinary grade-school algorithm takes Θ(n 2 ) steps to multiply two n-digit numbers. –This seems like too much work! So, let’s find an asymptotically faster multiplication algorithm!So, let’s find an asymptotically faster multiplication algorithm! To find the product cd of two 2n-digit base-b numbers, c=(c 2n-1 c 2n-2 …c 0 ) b and d=(d 2n-1 d 2n-2 …d 0 ) b,To find the product cd of two 2n-digit base-b numbers, c=(c 2n-1 c 2n-2 …c 0 ) b and d=(d 2n-1 d 2n-2 …d 0 ) b, First, we break c and d in half: First, we break c and d in half: c=b n C 1 +C 0, d=b n D 1 +D 0, and then... (see next slide) c=b n C 1 +C 0, d=b n D 1 +D 0, and then... (see next slide) § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank33 Derivation of Fast Multiplication Zero (Multiply out polynomials) (Factor last polynomial) § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank34 Recurrence Rel. for Fast Mult. Notice that the time complexity T(n) of the fast multiplication algorithm obeys the recurrence: T(2n)=3T(n)+  (n) i.e.,T(2n)=3T(n)+  (n) i.e., T(n)=3T(n/2)+  (n)T(n)=3T(n/2)+  (n) So a=3, b=2. Time to do the needed adds & subtracts of n-digit and 2n-digit numbers § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank35 The Master Theorem Consider a function f(n) that, for all n=b k for all k  Z +,, satisfies the recurrence relation: f(n) = a f (n/b) + cn d with a≥1, integer b>1, real c>0, d≥0. Then: § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank36 Examples Consider a function f(n) that, for all n=2 k for all k  Z +,, satisfies the recurrence relation: f(n) = 5f(n/2) + 3. Then: Complexity of Merge Sort: § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank37 Example Recall that complexity of fast multiply was:Recall that complexity of fast multiply was: T(n)=3T(n/2)+  (n) Thus, a=3, b=2, d=1. So a > b d, so case 3 of the master theorem applies, so:Thus, a=3, b=2, d=1. So a > b d, so case 3 of the master theorem applies, so: which is  (n 1.58… ), so the new algorithm is strictly faster than ordinary Θ(n 2 ) multiply! § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank38 Example The Closest-Pair Problem:a set of n points,The Closest-Pair Problem:a set of n points, How can this closest pair of points be found in an efficient way?How can this closest pair of points be found in an efficient way? T(n)=2T(n/2)+7n § 7.3 – D-C Recurrence Relations

Chap. 7 (c) , Michael P. Frank39 §7.4: Generating Functions Definition:generating function for the sequence of real numbers is the infinite seriesDefinition:generating function for the sequence of real numbers is the infinite series § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank40 Examples What is the generating function of the sequence 1,1,1,1,1,1?What is the generating function of the sequence 1,1,1,1,1,1? What is the generating function of the sequence ?What is the generating function of the sequence ? § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank41 Examples The function f(x)=1/(1  x) is the generating function of the sequence 1,1,1,…for |x|<1.The function f(x)=1/(1  x) is the generating function of the sequence 1,1,1,…for |x|<1. The function f(x)=1/(1  ax) is the generating function of the sequence 1,a,a 2,…for |ax|<1.The function f(x)=1/(1  ax) is the generating function of the sequence 1,a,a 2,…for |ax|<1. § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank42 Theorem Convolution of a k and b k § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank43 Example What sequence has the generating function What sequence has the generating function f(x)=1/(1  x) 2 ? f(x)=1/(1  x) 2 ? § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank44 Extended Binomial Coefficient Examples: Note: u positive integer, § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank45 Extended Binomial Theorem Can be proved using Maclaurin series. Examples: § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank46 Example Find the number of solutions of Find the number of solutions of Sol: Find the coefficient of x 17, The answer is § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank47 Example Solve the recurrence relation: Solve the recurrence relation: Sol: Let G(x) be the generating function of {a}, Sol: Let G(x) be the generating function of {a k }, § 7.4 – Generating Functions

Chap. 7 (c) , Michael P. Frank48 Example(Cont’d) § 7.4 – Generating Functions