Recursion and Recurrence Relations
Recurrence Relations Definition: A recurrence relation for a sequence {an} is an equation that expresses an in terms of one or more of the previous terms in the sequence: a0, a1, a2, …, an-1 for all integers nn0 where n0 is a nonnegative integer. OR simply A recurrence relation is an equation which is defined in terms of itself. A sequence is called a solution of a recurrence if its terms satisfy the recurrence relation
CMSC 203 - Discrete Structures Recurrence Relations In other words, a recurrence relation is like a recursively defined sequence, but without specifying any initial values (initial conditions). Therefore, the same recurrence relation can have (and usually has) multiple solutions. If both the initial conditions and the recurrence relation are specified, then the sequence is uniquely determined. Fall 2002 CMSC 203 - Discrete Structures
CMSC 203 - Discrete Structures Recurrence Relations Example: Consider the recurrence relation an = 2an-1 – an-2 for n = 2, 3, 4, … Is the sequence {an} with an=3n a solution of this recurrence relation? For n 2 we see that 2an-1 – an-2 = 2(3(n – 1)) – 3(n – 2) = 3n = an. Therefore, {an} with an=3n is a solution of the recurrence relation. Fall 2002 CMSC 203 - Discrete Structures
CMSC 203 - Discrete Structures Recurrence Relations Is the sequence {an} with an= 5 a solution of the same recurrence relation? For n 2 we see that 2an-1 – an-2 = 25 - 5 = 5 = an. Therefore, {an} with an=5 is also a solution of the recurrence relation. Fall 2002 CMSC 203 - Discrete Structures
More examples of recurrence relations Initial condition a0 = 1 Recursive formula: a n = 1 + 2a n-1 for n > 1 First few terms are: 1, 3, 7, 15, 31, 63, … Example 2: Initial conditions a0 = 1, a1 = 2 Recursive formula: a n = 3(a n-1 + a n-2) for n > 2 First few terms are: 1, 2, 9, 33, 126, 477, 1809, 6858, 26001,…
More examples of recurrence relations The Fibonacci numbers are defined by the recurrence relation F(n) = F(n-1) +F(n-2) F(1) = 1 F(0) = 0 The solution to the Fibonacci recurrence is fn = 1/5 ((1+5)/2)n – 1/5 ((1-5)/2)n
Recurrence Relations: Initial Conditions The initial conditions specify the value of the first few necessary terms in the sequence In the Fibonacci numbers, we needed two initial conditions: F(0)=0 and F(1)=1 because F(n) is defined by the two previous terms in the sequence Initial conditions are also known as boundary conditions (as opposed to general conditions) From now on, we will use the subscript notation, so the Fibonacci numbers are: fn = fn-1 + fn-2 f1 = 1 f0 = 0
Find the Fibonacci numbers f2, f3, f4, f5, and f6. Solution: The recurrence relation for the Fibonacci sequence tells us that we find successive terms by adding the previous two terms. Because the initial conditions tell us that f0 = 0 and f1 = 1, using the recurrence relation in the definition we find that f2 = f1 + f0 = 1 + 0 = 1, f3 = f2 + f1 = 1 + 1 = 2, f4 = f3 + f2 = 2 + 1 = 3, f5 = f4 + f3 = 3 + 2 = 5, f6 = f5 + f4 = 5 + 3 = 8.
Fibonacci sequence in nature 13 8 5 3 2 1
Reproducing rabbits You have one pair of rabbits on an island The rabbits repeat the following: Get pregnant one month Give birth (to another pair) the next month This process repeats indefinitely (no deaths) Rabbits get pregnant the month they are born How many rabbits are there after 10 months?
Reproducing rabbits First month: 1 pair Second month: 1 pair The original pair Second month: 1 pair The original (and now pregnant) pair Third month: 2 pairs The child pair (which is pregnant) and the parent pair (recovering) Fourth month: 3 pairs “Grandchildren”: Children from the baby pair (now pregnant) Child pair (recovering) Parent pair (pregnant) Fifth month: 5 pairs Both the grandchildren and the parents reproduced 3 pairs are pregnant (child and the two new born rabbits)
Reproducing rabbits Sixth month: 8 pairs Seventh month: 13 pairs All 3 new rabbit pairs are pregnant, as well as those not pregnant in the last month (2) Seventh month: 13 pairs All 5 new rabbit pairs are pregnant, as well as those not pregnant in the last month (3) Eighth month: 21 pairs All 8 new rabbit pairs are pregnant, as well as those not pregnant in the last month (5) Ninth month: 34 pairs All 13 new rabbit pairs are pregnant, as well as those not pregnant in the last month (8) Tenth month: 55 pairs All 21 new rabbit pairs are pregnant, as well as those not pregnant in the last month (13)
The Fibonacci numbers appear in many other places in nature, including the number of petals on flowers and the number of spirals on seedheads etc.
Fibonacci sequence As the terms increase, the ratio between successive terms approaches 1.618 This is called the “golden ratio” Ratio of human leg length to arm length Ratio of successive layers in a conch shell Reference: http://en.wikipedia.org/wiki/Golden_ratio
Recurrence Relations: Terms Recurrence relations have two parts: recursive terms and non-recursive terms T(n) = 2T(n-2) + n2 -10 Recursive terms come from when an algorithms calls itself Non-recursive terms correspond to the non-recursive cost of the algorithm: work the algorithm performs within a function We will see examples later. First, we need to know how to solve recurrences.
Solving Recurrences There are several methods for solving recurrences Characteristic Equations Forward Substitution Backward Substitution Recurrence Trees … Maple! However, in this study we will only be concerned with solution by means of characteristic equations.
Method 1: Iteration Problem: Given a recursive expression with initial conditions a0, a1 try to express an without dependence on previous terms. Example: an = 2an-1 for n > 1, with initial condition a0 = 1 Solution: an = 2n
More on the iteration method Example: Deer Population growth Deer population dn at time n Initial condition: d0 = 1000 Increase from time n-1 to time n is 10%. Therefore the recursive function is dn – dn-1 = 0.1dn-1 dn = 1.1dn-1 Solution: dn = 1000(1.1)n
Linear Homogeneous Recurrences Definition: A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of the form an = c1an-1 + c2an-2 + … + ckan-k with c1, c2, …, ckR, ck 0. Linear: RHS is a sum of multiples of previous terms of the sequence each multiplied by a function of n (linear combination of previous terms). The coefficients are all constants (not functions depending on n) Homogeneous: no terms occur that are not multiples of aj’s Degree k: an is expressed in terms of (n-k)th term of the sequence Linear homogeneous recurrence relations are studied for two reasons. First, they often occur in modeling of problems. Second, they can be systematically solved.
Linear homogeneous recurrences-examples EXAMPLE 1 The recurrence relation Pn = (1.11)Pn−1 is a linear homogeneous recurrence relation of degree one. The recurrence relation fn = fn−1 + fn−2 is a linear homogeneous recurrence relation of degree two. The recurrence relation an = an-5 is a linear homogeneous recurrence relation of degree five. EXAMPLE 2 The recurrence relation an = an−1 + a2n−2 is not linear. The recurrence relation Hn = 2Hn−1 + 1 is not homogeneous. The recurrence relation Bn = nBn−1 does not have constant coefficients.
Linear Homogeneous Recurrences: Examples So are the following relations: an = 4an-1 + 5an-2 + 7an-3 an = 2an-2 + 4an-4 + 8an-8 How many initial conditions do we need to specify for these relations? So, how do solve linear homogeneous recurrences? As many as the degree k: k=3, 8 respectively
Solving Linear Homogeneous Recurrences We want a solution of the form an=rn where r is some real constant We observe that an=rn is a solution to a linear homogeneous recurrence if and only if rn = c1rn-1 + c2rn-2 + … + ckrn-k We can now divide both sides by rn-k, collect terms and we get a k-degree polynomial rk - c1rk-1 - c2rk-2 - … - ck = 0 This equation is called the characteristic equation of the recurrence relation The roots of this polynomial are called the characteristics roots of the recurrence relation. They can be used to find the solutions (if they exist) to the recurrence relation. We will consider several cases.
Second Order Linear Homogeneous Recurrences A second order (k=2) linear homogeneous recurrence is a recurrence of the form an = c1an-1+ c2an-2 Theorem 1: Let c1, c2R and suppose that r2-c1r-c2=0 is the characteristic polynomial of a 2nd order linear homogeneous recurrence that has two distinct* roots r1,r2, then {an} is a solution if and only if an= 1r1n + 2r2n for n=0,1,2,… where 1, 2 are constants dependent upon the initial conditions * We discuss single root later
Proof: We must do two things to prove the theorem Proof: We must do two things to prove the theorem. First, it must be shown that if r1 and r2 are the roots of the characteristic equation, and α1 and α2 are constants, then the sequence {an} with an = α1r1n+ α2r2n is a solution of the recurrence relation. Second, it must be shown that if the sequence {an} is a solution, then an = α1r1n+ α2r2n for some constants α1 and α2. Now we will show that if an = α1r1n+ α2r2n , then the sequence {an} is a solution of the recurrence relation. Because r1 and r2 are roots of r2 − c1r − c2 = 0, it follows that r12= c1r1 + c2, r22= c1r2 + c2.
From these equations, we see that c1an−1 + c2an−2 = c1 (α1r1n-1+ α2r2n-1 ) + c2 (α1r1n-2+ α2r2n-2 ) = α1r1n-2 (c1r1 + c2 ) + α2r2n-2 (c1r2 + c2 ) = α1r1n-2r12 + α2r2n-2 r22 = α1r1n + α2r2n = an This shows that the sequence {an} with an = α1r1n + α2r2n is a solution of the recurrence relation.
To show that every solution {an} of the recurrence relation an = c1 an-1 + c2 an -2 has an = α1r1n + α2r2n for n = 0, 1, 2, . . . , for some constants α1 and α2, suppose that {an} is a solution of the recurrence relation, and the initial conditions a0 = C0 and a1 = C1 hold. It will be shown that there are constants α1 and α2 such that the sequence {an} with an = α1r1n + α2r2n satisfies these same initial conditions. This requires that a0 = C0 = α1 + α2, a1 = C1 = α1r1 + α2r2.
We can solve these two equations for α1 and α2 We can solve these two equations for α1 and α2. From the first equation it follows that α2 = C0 − α1. Inserting this expression into the second equation gives C1 = α1 r1 + (C0 − α1) r2. Hence, C1 = α1(r1 − r2) + C0 r2. This shows that α1 = C1−C0r2 r1 − r2 and α2 = C0 − α1 = C0 − C1−C0r2 r1 − r2 = C0r1−C1 r1 − r2
We can now conclude that {an} and {α1r1n + α2r2n } are both solutions of the recurrence relation an = c1 an-1 + c2 an -2 and both satisfy the initial conditions when n = 0 and n = 1. Because there is a unique solution of a linear homogeneous recurrence relation of degree two with two initial conditions, it follows that the two solutions are the same, that is, an = α1r1n + α2r2n for all nonnegative integers n. We have completed the proof by showing that a solution of the linear homogeneous recurrence relation with constant coefficients of degree two must be of the form an = α1r1n + α2r2n , where α1 and α2 are constants.
Second Order Linear Homogeneous Recurrences: Example A (1) Find a solution to an = 5an-1 - 6an-2 with initial conditions a0=1, a1=4 The characteristic equation is r2 - 5r + 6 = 0 The roots are r1=2, r2=3 r2 - 5r + 6 = (r-2)(r-3) Using the 2nd order theorem we have a solution an = 12n + 23n
Second Order Linear Homogeneous Recurrences: Example A (2) Given the solution an = 12n + 23n We plug in the two initial conditions to get a system of linear equations a0 = 120 + 230 a1 = 121 + 231 Thus: 1 = 1 + 2 4 = 21 + 32
Second Order Linear Homogeneous Recurrences: Example A (3) 1 = 1 + 2 4 = 21 + 32 Solving for 1 = (1 - 2), we get 4 = 2(1-2) + 32 4 = 2 - 22 + 32 2 = 2 Substituting for 1: 1 = -1 Putting it back together, we have an = 12n + 23n an = -12n + 23n
Second Order Linear Homogeneous Recurrences: Example B (1) Solve the recurrence an = -2an-1 + 15an-2 with initial conditions a0= 0, a1= 1 If we did it right, we have an = 1/8 (3)n - 1/8 (-5)n To check ourselves, we verify a0, a1, we compute a3 with both equations, then maybe a4, etc.
Single Root Case We can apply the theorem if the roots are distincts, i.e. r1r2 If the roots are not distinct (r1=r2), we say that one characteristic root has multiplicity two. In this case, we apply a different theorem Theorem 2: Let c1, c2R and suppose that r2 - c1r - c2 = 0 has only one distinct root, r0, then {an} is a solution to an = c1an-1+ c2an-2 if and only if an= 1r0n + 2nr0n for n=0,1,2,… where 1, 2 are constants depending upon the initial conditions
Single Root Case: Example (1) What is the solution to the recurrence relation an = 8an-1 - 16an-2 with initial conditions a0= 1, a1= 7? The characteristic equation is: r2 – 8r + 16 = 0 Factoring gives us: r2 – 8r + 16 = (r-4)(r-4), so r0=4 Applying the theorem we have the solution: an= 1(4)n + 2n(4)n
Single Root Case: Example (2) Given: an= 1(4)n + 2n(4)n Using the initial conditions, we get: a0= 1 = 1(4)0 + 20(4)0 = 1 a1= 7 = 1(4) + 21(4)1 = 41 + 42 Thus: = 1 = 1, 2 = 3/4 The solution is an= (4)n + ¾ n (4)n Always check yourself…
General Linear Homogeneous Recurrences There is a straightforward generalization of these cases to higher-order linear homogeneous recurrences Essentially, we simply define higher degree polynomials The roots of these polynomials lead to a general solution The general solution contains coefficients that depend only on the initial conditions In the general case, the coefficients form a system of linear equalities
General Linear Homogeneous Recurrences: Distinct Roots Theorem 3: Let c1,c2,..,ck R and suppose that the characteristic equation rk - c1rk-1 - c2rk-2 - … - ck = 0 has k distinct roots r1,r2, …,rk. Then a sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 + … + ckan-k if and only if an = 1r1n + 2r2n + … + krkn for n=0,1,2,… where 1,2,…,k are constants depending upon the initial conditions
General Linear Homogeneous Recurrences: Any Multiplicity Theorem 4: Let c1,c2,..,ck R and suppose that the characteristic equation rk - c1rk-1 - c2rk-2 - … - ck = 0 has t roots with multiplicities m1,m2, …,mt. Then a sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 + … + ckan-k if and only if an = (1,0 + 1,1n + … + 1,m1-1nm1-1) r1n + (2,0 + 2,1n + … + 2,m2-1nm2-1) r2n + ... (t,0 + t,1n + … + t,mt-1nmt-1) rtn for n=0,1,2,… where i,j are constants for 1 i t and 0 jmi-1 depending upon the initial conditions
Modeling with Recurrence Relations Example: Someone deposits $10,000 in a savings account at a bank yielding 5% per year with interest compounded annually. How much money will be in the account after 30 years? Solution: Let Pn denote the amount in the account after n years. How can we determine Pn on the basis of Pn-1? Fall 2002 CMSC 203 - Discrete Structures
Modeling with Recurrence Relations We can derive the following recurrence relation: Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1. The initial condition is P0 = 10,000. Then we have: P1 = 1.05P0 P2 = 1.05P1 = (1.05)2P0 P3 = 1.05P2 = (1.05)3P0 … Pn = 1.05Pn-1 = (1.05)nP0 We now have a formula to calculate Pn for any natural number n and can avoid the iteration. Fall 2002 CMSC 203 - Discrete Structures
Modeling with Recurrence Relations Let us use this formula to find P30 under the initial condition P0 = 10,000: P30 = (1.05)3010,000 = 43,219.42 After 30 years, the account contains $43,219.42. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations In general, we would prefer to have an explicit formula to compute the value of an rather than conducting n iterations. For one class of recurrence relations, we can obtain such formulas in a systematic way. Those are the recurrence relations that express the terms of a sequence as linear combinations of previous terms. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Definition: A linear homogeneous recurrence relation of degree k with constant coefficients is a recurrence relation of the form: an = c1an-1 + c2an-2 + … + ckan-k, Where c1, c2, …, ck are real numbers, and ck 0. A sequence satisfying such a recurrence relation is uniquely determined by the recurrence relation and the k initial conditions a0 = C0, a1 = C1, a2 = C2, …, ak-1 = Ck-1. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Examples: The recurrence relation Pn = (1.05)Pn-1 is a linear homogeneous recurrence relation of degree one. The recurrence relation fn = fn-1 + fn-2 is a linear homogeneous recurrence relation of degree two. The recurrence relation an = an-5 is a linear homogeneous recurrence relation of degree five. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Basically, when solving such recurrence relations, we try to find solutions of the form an = rn, where r is a constant. an = rn is a solution of the recurrence relation an = c1an-1 + c2an-2 + … + ckan-k if and only if rn = c1rn-1 + c2rn-2 + … + ckrn-k. Divide this equation by rn-k and subtract the right-hand side from the left: rk - c1rk-1 - c2rk-2 - … - ck-1r - ck = 0 This is called the characteristic equation of the recurrence relation. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations The solutions of this equation are called the characteristic roots of the recurrence relation. Let us consider linear homogeneous recurrence relations of degree two. Theorem: Let c1 and c2 be real numbers. Suppose that r2 – c1r – c2 = 0 has two distinct roots r1 and r2. Then the sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 if and only if an = 1r1n + 2r2n for n = 0, 1, 2, …, where 1 and 2 are constants. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Example: What is the solution of the recurrence relation an = an-1 + 2an-2 with a0 = 2 and a1 = 7 ? Solution: The characteristic equation of the recurrence relation is r2 – r – 2 = 0. Its roots are r = 2 and r = -1. Hence, the sequence {an} is a solution to the recurrence relation if and only if: an = 12n + 2(-1)n for some constants 1 and 2. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Given the equation an = 12n + 2(-1)n and the initial conditions a0 = 2 and a1 = 7, it follows that a0 = 2 = 1 + 2 a1 = 7 = 12 + 2 (-1) Solving these two equations gives us 1 = 3 and 2 = -1. Therefore, the solution to the recurrence relation and initial conditions is the sequence {an} with an = 32n – (-1)n. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Example: Give an explicit formula for the Fibonacci numbers. Solution: The Fibonacci numbers satisfy the recurrence relation fn = fn-1 + fn-2 with initial conditions f0 = 0 and f1 = 1. The characteristic equation is r2 – r – 1 = 0. Its roots are Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Therefore, the Fibonacci numbers are given by for some constants 1 and 2. We can determine values for these constants so that the sequence meets the conditions f0 = 0 and f1 = 1: Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations The unique solution to this system of two equations and two variables is So finally we obtained an explicit formula for the Fibonacci numbers: Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations But what happens if the characteristic equation has only one root? How can we then match our equation with the initial conditions a0 and a1 ? Theorem: Let c1 and c2 be real numbers with c2 0. Suppose that r2 – c1r – c2 = 0 has only one root r0. A sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 if and only if an = 1r0n + 2nr0n, for n = 0, 1, 2, …, where 1 and 2 are constants. Fall 2002 CMSC 203 - Discrete Structures
Solving Recurrence Relations Example: What is the solution of the recurrence relation an = 6an-1 – 9an-2 with a0 = 1 and a1 = 6? Solution: The only root of r2 – 6r + 9 = 0 is r0 = 3. Hence, the solution to the recurrence relation is an = 13n + 2n3n for some constants 1 and 2. To match the initial condition, we need a0 = 1 = 1 a1 = 6 = 13 + 23 Solving these equations yields 1 = 1 and 2 = 1. Consequently, the overall solution is given by an = 3n + n3n. Fall 2002 CMSC 203 - Discrete Structures
Compound interest Given P = initial amount (principal) n = number of years r = annual interest rate A = amount of money at the end of n years At the end of: 1 year: A = P + rP = P(1+r) 2 years: A = P + rP(1+r) = P(1+r)2 3 years: A = P + rP(1+r)2 = P(1+r)3 … Obtain the formula A = P (1 + r) n
Eugene Catalan Catalan numbers are generated by the formula: Belgian mathematician, 1814-1894 Catalan numbers are generated by the formula: Cn = C(2n,n) / (n+1) for n > 0 The first few Catalan numbers are: n 1 2 3 4 5 6 7 8 9 10 11 Cn 14 42 132 429 1430 4862 16796 58786
Catalan Numbers: applications The number of ways in which a polygon with n+2 sides can be cut into n triangles The number of ways in which parentheses can be placed in a sequence of numbers, to be multiplied two at a time The number of rooted trivalent trees with n+1 nodes The number of paths of length 2n through an n by n grid that do not rise above the main diagonal The number of nonisomorphic binary trees with n vertices
Towers of Hanoi Start with three pegs numbered 1, 2 and 3 mounted on a board, n disks of different sizes with holes in their centers, placed in order of increasing size from top to bottom. Object of the game: find the minimum number of moves needed to have all n disks stacked in the same order in peg number 3.
Rules of the game: Hanoi towers Start with all disks stacked in peg 1 with the smallest at the top and the largest at the bottom Use peg number 2 for intermediate steps Only a disk of smaller diameter can be placed on top of another disk
End of game: Hanoi towers Game ends when all disks are stacked in peg number 3 in the same order they were stored at the start in peg number 1. Verify that the minimum number of moves needed is the Catalan number C3 = 5. Start End
How to Cheat with Maple (1) Maple and other math tools are great resources. However, they are no substitutes for knowing how to solve recurrences yourself As such, you should only use Maple to check you answers Recurrence relations can be solved using the rsolve command and giving Maple the proper parameters The arguments are essentially a comma-delimited list of equations General and boundary conditions Followed by the ‘name’ and variables of the function
How to Cheat with Maple (2) > rsolve({T(n)= T(n-1)+2*n,T(1)=5},T(n)); 1+2(n+1)(1/2n+1)-2n You can clean up Maple’s answer a bit by encapsulating it in the simplify command > simplify(rsolve({T(n)= T(n-1) + 2*n, T(1) = 5}, T(n))); 3 + n2 + n