1. UNIT-6 Recurrence Relations
Yen (NTUEE) Discrete Mathematics 2012

2. Recurrence Relations Definition
A recurrence relation for {an} is an equation that expresses an in terms of a0, , an−1. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.
For instance, the relation fn = fn−1 + fn−2 of Fibonacci numbers is a recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012

3. Tower of Hanoi
Example
(The Tower of Hanoi) Consider moving a stack of disks with different sizes with three pegs. Initially, all disks are sorted and placed on the first peg. One can only move the topmost disk from a peg to another peg. At any time, disks on each peg must be sorted as well. How many steps does it take to move all disks from one peg to another?
Solution: Let Hn be the number of steps to solve the problem of n disks. Clearly, H1 = 1.
Now consider moving n disks from peg 1 to peg 2. If we can move the topmost n − 1 disks from peg 1 to peg 3, we can solve the puzzle by moving the bottom disk from peg 1 to peg 2 and then the n − 1 disks from peg 3 to peg 2. In other words, Hn = Hn− Hn−1. We have Hn = 2Hn−1 + 1.
Yen (NTUEE) Discrete Mathematics 2012

4. Tower of Hanoi Hence Hn = 2Hn−1 + 1 = 2(2Hn−2 + 1) + 1 =
· · ·
= 2n−1H1 + 2n−2 + · · ·
n−1
X 2i
i =0 =
2n − 1 =
= 2n − 1
2 − 1
Yen (NTUEE) Discrete Mathematics 2012

5. = P Catalan numbers Example
Find the number of ways to parenthesize the product of n + 1 numbers
x0, x1, , xn.
Solution: Let Cn denote the number of ways to parenthesize the product of n + 1 numbers. For instance, C3 = 5 because
x0 · (x1 · (x2 · x3)) x0 · ((x1 · x2) · x3) (x0 · (x1 · x2)) · x3 ((x0 · x1) · x2) · x3
are all the possible ways of multiplying x0 · x1 · · · · · x3.
Clearly C0 = 1. Consider x0 · x1 · · · · · xn. We can compute it by (x0 · · · · · xk ) · (xk+1 · · · · · xn) for any k = 0, , n. Therefore
Cn = C0Cn−1 + C1Cn−2 + · · · + Cn−1C0.
The sequence {C }, C
= P
n−1
n n
C C 1, is called Catalan numbers.
k=0
k n−k−
We’ll give a closed form for Catalan numbers later.
Yen (NTUEE) Discrete Mathematics 2012

6. Homogeneous Linear Recurrence Relations with Constant Coefficients
Definition
A linear homogeneous recurrence relation of degree k with constant coefficients is of the form
an = c1an−1 + c2an−2 + · · · + ck an−k ,
where c1, c2, , ck ∈ R with ck =6 0.
For instance, the recurrence for Fibonacci numbers fn = fn−1 + fn−2 is a linear homogeneous recurrence relation of degree two.
Theorem
Let c1, c2 ∈ R. Suppose x 2 = c1x + c2 has two distinct roots r1 and r2.
Then {an} is a solution of the recurrence relation an = c1an−1 + c2an−2 if
and only if a = α r + α r for n ∈ N and some constants α , α .
n n
n 1
1 2
2 1 2
Yen (NTUEE) Discrete Mathematics 2012

7. Solving Linear Recurrence Relations
Proof.
(⇐) Suppose a = α r + α r . We want to verify a = c a n n
n 1 1 2 2
n 1 n 1
− 2 n−2
+ c a .
c1an−1 c2an−2
c1an−1 + c2an−2
= c1α1r1 + c1α2r2
= c2α1r1 + c2α2r2 =
= an
n−1 n−1
n−2 n−2
α1r1 (c1r1 + c2) + α2r2 (c1r2 + c2)
n−2 n−2
α1rn−2 2
r1 + α2r2 r2
n−2 2 1
α1rn + α2rn 1 2
(Cont’d on Next Page)
Yen (NTUEE) Discrete Mathematics 2012

8. Solving Linear Recurrence Relations
Proof.
(Cont’d) (⇒) From the first part of proof, we know a = α r + α r n n
n 1 1 2 2
satisfies the recurrence relation
an = c1an−1 2 n−2
+ c a . It remains to show
an = α1rn + α2rn satisfies initial conditions for some α1, α2. The theorem
1 2
then follows from the uniqueness of solution to linear homogeneous recurrence relation.
To see an = α1rn + α2rn satisfies the initial conditions for some α1, α2.
1 2
Consider a1 = α1r1 + α2r2 and a0 = α1 + α2.
This is a linear system of two variables α1 and α2. The solutions are
a1 − a0r2
α1 = ,
r − r
1 2
and
a r − a
α2 =
r1 − r2
Yen (NTUEE) Discrete Mathematics 2012

9. 1 + √5 !n √ n √ n Fibonacci numbers Example 5 , we
Recall the recurrence relation for Fibonacci numbers fn = fn−1 + fn−2 with
f0 = 0 and f1 = 1. Find an explicit formula for the Fibonacci numbers.
Solution: The solutions to x 2 = x + 1 are 1± Hence √ 2
1 + √5 !n 1
√ n
− 5 !
fn = α1
+ α2
2 2 √ √
for some α1 and α2. Solving 0 = α1 + α2, and 1 = α1 1+
5 + α2 1−
5 , we 2 2
have α1 = and α = − . Therefore
1 1 √5 2 √5
√5 !n − 1
√ n 5 !
fn = √5 . 2
− √5 2
Yen (NTUEE) Discrete Mathematics 2012

10. Characteristic Equations with Multiple Roots
Theorem
Let c1, c2 ∈ R with c2 =6 0. Suppose x 2 = c1x + c2 has only one root r .
Then {an} is a solution of the recurrence relation an = c1an−1 + c2an−2 if
and only if an = α1rn + α2nrn for n ∈ N and some constants α1, α2.
Proof.
(⇐) 2r = c1 and c1r + 2c2 = 0. Let an = α1rn + α2nrn. Then
c1an−1 c2an−2
∴ c1an−1 + c2an−2
= c1α1r + c1α2(n − 1)r
= c2α1r + c2α2(n − 2)r =
n−1
n−1 n−2
n−2
α1r (c1r + c2) + α2r (c1(n − 1)r + c2(n − 2))
n−2 n−2
α1rn−2r 2 + α2rn−2(c1nr − c1r + c2n − 2c2)
α1rn + α2rn−2(c1nr + c2n) α1rn + α2rn−2n(c1r + c2) α1rn + α2nrn−2r 2 =
Yen (NTUEE) Discrete Mathematics 2012
α rn + α nrn

11. Characteristic Equations with Multiple Roots
Proof.
(Cont’d)
(⇒) It remains to show there are α1 and α2 satisfying the initial conditions. We have a0 = α1. Therefore
a1 − a0r r
α2 = .
Yen (NTUEE) Discrete Mathematics 2012

12. An Example Example What is the solution of the recurrence relation
an = 6an−1 − 9an−2
with a0 = 1 and a1 = 6?
Solution: Since 3 is the multiple root of x 2 = 6x − 9, we have
an = α13n + α2n3n. Moreover, α1 = a0 = 1 and α2 = (6 − 1 · 3)/3 = 1. We have an = 3n + n3n.
Yen (NTUEE) Discrete Mathematics 2012

13. Characteristic Equations with Distinct Roots
Theorem
Let c1, c2, , ck ∈ R. Suppose the characteristic equation
rk − c1rk−1 − · · · − ck = 0
has k distinct roots r1, , rk . Then {an} is a solution of the recurrence relation
an = c1an−1 + c2an−2 + · · · + ck an−k
if and only if
a = α r + α r + · · · + α r n
k k
n n
n 1
for n ∈ N and constants α1, α2, , αk .
Yen (NTUEE) Discrete Mathematics 2012

14. Characteristic Equations with Multiple Roots
Theorem
Let c1, c2, , ck ∈ R. Suppose the characteristic equation
rk − c1rk−1 − · · · − ck = 0
has t distinct roots r1, , rt with multiplicities m1, , mt respectively such that mi ≥ 1 and m1 + m2 + · · · + mk = k. Then {an} is a solution of the recurrence relation an = c1an−1 + c2an−2 + · · · + ck an−k if and only if
an = (α1,0 + α1,1n + · · · + α1,m1−1n )r1
+ (α2,0 + α2,1n + · · · + α2,m2−1n )r2
+ · · ·
m1−1 n
m2−1 n
+ (αt,0 + αt,1n + · · · + αt,mt −1n )rt
for n ∈ N and constants αi ,j .
mt −1 n
Yen (NTUEE) Discrete Mathematics 2012

15. Solving Nonhomogeneous Linear Recurrence Relations
Definition
A linear nonhomogeneous recurrence relation with constant coefficients is of the form
an = c1an−1 + c2an−2 + · · · + ck an−k + F (n), where c1, c2, , ck ∈ R and F : N → Z. The recurrence relation
an = c1an−1 + c2an−2 + · · · + ck an−k
is called the associated homogeneous recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012

16. Solving Linear Recurrence Relations
Let {an } be a particular solution of
an = c1an−1 + c2an−2 + · · · + ck an−k + F (n),
Then every solution is of the form {an an }, where {an } is a solution
Theorem
(p)
(p) (h) (h) of
an = c1an−1 + c2an−2 + · · · + ck an−k .
(Cont’d on Next Page)
Yen (NTUEE) Discrete Mathematics 2012

17. Solving Nonhomogeneous Linear Recurrence Relations
Proof.
Suppose {bn} be any solution of the nonhomogeneous recurrence relation such that
bn = c1bn−1 + c2bn−2 + · · · + ck bn−k + F (n).
Then
Since {an } is a particular solution, we have
(p)
a(p)
= c1a(p) (p) (p)
n−1 + c2an−2 + · · · + ck an−k + F (n). n
bn − an = c1(bn−1 − an−1) + c2(bn−2 − an−2) + · · · + ck (bn−k − an−k ).
That is, {an − bn} is a solution of the associated homogeneous
(p) (p) (p) (p)
(p)
recurrence relation. Thus, bn = a(p) + a(h).
n n
Yen (NTUEE) Discrete Mathematics 2012

18. An Example
Example
Find all solutions of an = 3an−1 + 2n with a1 = 3.
Solution: Let us guess a(p) = cn + d . Then n
cn + d = 3(c(n − 1) + d ) + 2n. We have cn + d = (3c + 2)n + (3d − 3c).
Solve c = 3c + 2 and d = 3d − 3c. We obtain c = −1 and d = − . 3 2
Therefore a = −n − is a particular solution.
(p) 3 n 2
By Theorem 9, we have a(h) = α3n as solutions to the associated n
homogeneous recurrence relation. Therefore all solutions are of the form
a = −n − + α · 3 where α is a constant.
3 n n 2
Finally, 3 = a = −1 − + 3α. α = . Therefore
3 11 1 2 6
3 11 n
an = −n −
Yen (NTUEE) Discrete Mathematics 2012

19. a(p) = 49 n 7n An Example Example 20 · 7 is a particular solution.
Find solutions of an = 5an−1 − 6an−
(p) n
Solution: Let us guess a = c · 7 . Then n n
c · 7 = 5c · 7 − 6c · Hence 49c = 35c − 6c + 49, c = . n
n−1
n−2 n 49 20
a(p)
= 49 n n
20 · 7 is a particular solution.
By Theorem 9, we have a(h) = α 2n + α 3n as solutions of the associated
n 1 2
homogeneous recurrence relation. Therefore
an = α12n + α23n + 49 20 7n
are all solutions.
Yen (NTUEE) Discrete Mathematics 2012

20. Solving Particular Solutions
Theorem
Suppose {an} satisfies an = c1an−1 + c2an−2 + · · · + ck an−k + F (n), where c1, c2, , ck ∈ R and
F (n) = (btnt + bt n + · · · + b n + b )s ,
t−1 n −1
1 0
where s, b0, b1, , bt ∈ R.
When s is not a root of the characteristic equation of the associated linear homogeneous recurrence relation, there is a particular solution of the form
(ptnt + pt n + · · · + p n + p )s .
t−1 n −1
1 0
When s is a root of the characteristic equation and its multiplicity is m, there is a particular solution of the form
nm(ptnt + pt n + · · · + p n + p )s .
t−1 n −1
1 0
Yen (NTUEE) Discrete Mathematics 2012

21. Solving Nonhomogeneous Linear Recurrence Relations
Example
Solve an = an−1 + 3n − 3n + 1. with a0 = 0. 2
Solution: Since an = an−1 + 3n − 3n + 1, F (n) = (3n − 3n + 1)1 .
2 2 n
By Theorem 9, we have a = α · 1 = α as homogeneous solutions.
(h) n n
By Theorem 15, we let a(p) = n(p n2 + p n + p ) as a particular solution. n
Substitute a(p) in the recurrence relation, we have n
n(p2n2 + p1n + p0) = (n − 1)(p2(n − 1)2 + p1(n − 1) + p0) + 3n2 − 3n + 1.
Simplify both sides of the equation and compare their coefficients: p2 =
p1 p0 0
−3p2 + p p2 − 2p1 + p0 − 3
−p2 + p1 − p0 + 1
Yen (NTUEE) Discrete Mathematics 2012

22. Solving Nonhomogeneous Linear Recurrence Relations
(Cont’d)
Solve the linear system and get p0 = 0, p1 = 0, p2 = 1. Hence
a(p)
= n(1 · n2 + 0 · n + 0) = n3 is a particular solution. n
Thus, an = n3 + α are all solutions. Since a0 = 0, we have α = 0. And we conclude an = n3.
Notice that n3 − (n − 1)3 = 3n2 − 3n + 1. Hence Pn
3k − 3k + 1 = n . Alternatively, note that n
an = X 3k2 − 3k + 1,
k=1
2 3
k=1
you can solve it by computing the summation.
Yen (NTUEE) Discrete Mathematics 2012

23. Solving Recurrence Relations by Domain Transformation
Example
T (n) = 2T (n/2) + nlog2n, n > 1 where n is a power of 2.
Solutions: Replace n by 2k , we have
T (2k ) = 2T (2k−1) + k2k
Let tk = T (2k ). The above can be rewritten as
tk = 2tk−1 + k2
Hence, k
tk = c12k + c2k2k + c3k22k
T (n) = c1n + c2nlogn + c3nlog 2n
Yen (NTUEE) Discrete Mathematics 2012

24. Solving Recurrence Relations by Range Transformation
Example
an = 3a2 subject to a0 = 1
n−1
Solution: Let bn = log an. Then the above can be rewritten as
bn = 2bn−1 + log 3 subject to b0 = 0.
Hence
bm = (2n − 1)log 3
an = 2(2 −1)log 3 = 32 −1
n n
Yen (NTUEE) Discrete Mathematics 2012

25. Divide-and-Conquer Algorithms and Recurrence Relations
Recall the merge sort algorithm. The algorithm divides the sequence in halves, solves each half recursively, and then merges sorted results to get the solution. This type of algorithms are called divide-and-conquer algorithms. In the complexity analysis of divide-and-conquer algorithms, we obtain the recurrence relation of the form n
f (n) = af ( ) + g (n), b
where f (n) is the number of steps needed in solving an instance of size n. We will prove a theorem to help us solve the recurrence relation.
Yen (NTUEE) Discrete Mathematics 2012

26. Divide-and-Conquer Algorithms and Recurrence Relations
Theorem
Let f be a non-decreasing function satisfying n
f (n) = af ( ) + c b
whenever b|n, where a ≥ 1, b ∈ Z+, b > 1, and c ∈ R+. Then
logb a
f (n) = O(n
) if a > 1
if a = 1
O(lg n)
Furthermore, when n = bk and a > 1, k ∈ Z+,
f (n) = C1nlogb a + C2, where C1 = c/(a − 1) + f (1) and C2 = −c/(a − 1).
Yen (NTUEE) Discrete Mathematics 2012

27. Divide-and-Conquer Algorithms and Recurrence Relations
Proof: We first consider n = bk . Then bk
f (n) = af ( ) + c b
= af (bk−1) + c
= a(af (bk−2) + c) + c
= a2f (bk−2) + ac + c
= · · ·
= ak f (1) + ak−1c + ak−2c + · · · + ac + c
k−1
= ak f (1) + c X ai
i =0
When a = 1, we have f (n) = f (1) + ck = f (1) + c logb n. Furthermore, if
bk < n < bk+1, we have f (bk ) ≤ f (n) ≤ f (bk+1) and bk+1 < nb. Thus,
f (n) ≤ f (bk+1) = f (1)+c(k +1) < f (1)+c(1+logb n) = f (1)+c +c logb n. Therefore f (n) = O(lg n) when a = 1.
Yen (NTUEE) Discrete Mathematics 2012

28. Divide-and-Conquer Algorithms and Recurrence Relations
(Cont’d)
When a > 1 and n = bk , then nlogb a = bk logb a = ak . We have k
f (n) = ak f (1) + c a
− 1
a − 1
= ak f (1) + c c
a − 1 − a − 1 =
For bk < n < bk+1, we have
nlogb aC1 + C2
f (n)
≤ f (bk+1)
= C1ak+1 + C2
= aC1ak + C2
≤ aC1nlogb a + C2
Hence, f (n) = O(nlogb a).
Yen (NTUEE) Discrete Mathematics 2012

29. An Example Example Let f (n) = 5f (n/2) + 3. Estimate f (n).
Solution: By Theorem 19, we have f (n) = O(nlog2 5) = O(nlg 5) since
a = 5 and b = 2.
For merge sort algorithm, we have T (n) = 2T (n/2) + Θ(n). Theorem 19 is not applicable. The following theorem will be proved in your algorithm class:
Yen (NTUEE) Discrete Mathematics 2012

30. Divide-and-Conquer Algorithms and Recurrence Relations
Theorem
Let f be a non-decreasing function satisfying n
f (n) = af ( ) + cnd b
whenever n = bk , k ∈ Z+, b ∈ Z+ with a ≥ 1, b > 1, and c, d ∈ R with c > 0, d ≥ 0. Then
 O(nd )
f (n) =  O(nd lg n)
 O(nlogb a)
if a < bd if a = bd if a > bd
Therefore, the complexity of merge sort algorithm is O(n lg n).
Yen (NTUEE) Discrete Mathematics 2012