Recurrence Relations

Outline Recurrence relationsSolving recurrence relationsRecurrence and Divide-and-conquer algorithmsGenerating functions Recurrence Relation 2

RECURRENCE RELATIONS Recurrence Relation 3

What is a recurrence relation? A recurrence relation for the sequence {a n } is an equation that expresses an in terms of one or more of the previous terms of the sequence, namely a 0, a 1,…, a n-1, for all integers n  n 0, where n 0 is a non-negative integer. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation. Example: Given the recurrence relation a n =2a n-1 -a n-2. 3, 5, 7, 9, 11, … satisfies the recurrence relation. 2, 3, 4, 5, 6, … also satisfies the recurrence relation Recurrence Relation 4

Compound interest : Example Let P n be the amount of money in the account after n years. If you initially deposit P 0 =1000 in your account and the bank gives k % per year with interest compounded annually. P n = (1+k/100) P n-1 P 1 = (1+k/100) P 0 = (1+k/100) P 0 P 2 = (1+k/100) P 1 = (1+k/100) 2 P 0 P 3 = (1+k/100) P 2 = (1+k/100) 3 P 0 P n = (1+k/100) P n-1 = (1+k/100) n P Recurrence Relation 5

Binary strings with no “00” : Example Recurrence Relation 6 Strings with length nStarting with 0 Starting with 00 Starting with 01Starting with All strings of length n-1 with no 00 All strings of length n-2 with no 00 none

Binary strings with no “00” : Example Let a n be the number of n-bit binary strings containing no “00”. a n = a n-1 + a n-2 a 0 = 1, a 1 = 2, a 2 = Recurrence Relation 7 Strings with length nStarting with 0 Starting with 00 Starting with 01Starting with 1 All strings of length n-1 with no 00 All strings of length n-2 with no 00 none

Rabbit breeding & Fibonacci Numbers A pair of rabbits is placed on an island. After 2 months old, a pair of rabbits produces another pair each month. How many rabbits are there after n months? Recurrence Relation 8 f n =f n-1 + f n-2

Tower of Hanoi Recurrence Relation 9 Let H(n) be the number of move to solve tower of Hanoi with n discs. H(n) = 2H(n-1) + 1H(1) = 1

SOLVING RECURRENCE RELATIONS Recurrence Relation 10

Verify solutions of recurrence relations Recurrence Relation 11 an = (-4)n an = 1 an = 3nan = 2n + 1 an = 4an-2 – 3 a n- 1 (-4) n = 4(-4) n-2 – 3(-4) n-1 ? 4(-4) n-2 +12(-4) n-2 = 16(-4) n-2 4(-4) n-2 +12(-4) n-2 = (-4) n 3n = 4(3(n-2)) – 3 (3(n-1)) ? 12n – 24 – 9n +9 = 3n – 15  3n 1= 4 (1) – 3(1) ? 1= 1 2n+1=4(2(n-2)+1) –3(2(n-1)+1)? 8n – 12 – 6n +3 = 2n – 9  2n +1 If no initial condition is given, there can be more than one solution of a recurrence relation.

Types of recurrence relations Recurrence Relation 12

Example: Recurrence relations linear, homogeneous, degree 1, with constant coefficient an = 1.2 an-1 linear, homogeneous, degree 3, with constant coefficient an = 3an-3 Non-linear, homogeneous, degree 2, with constant coefficient an = a2 n -1 + a n -2 Linear, homogeneous, degree 2, with variable coefficient an = nan-1 - 2an Recurrence Relation 13 Linear, non-homogeneous, degree 1, with constant coefficient an = an-1 + 2n Linear, homogeneous, degree 1, with constant coefficient hn = 2hn Non-linear, non-homogeneous, degree 1, constant coefficient an = a2 n n Linear, non-homogeneous, degree 1, variable coefficient an = n2 a n-1 + n

How to solve recurrence relations Guess and verifyUnfoldVariable transformationUsing formulaGenerating function Recurrence Relation 14

Guess and verify Find the closed form formula of a n = 2a n-1 +1 for n >0, a 0 =0 = Guess: a n = 2 n -1, for n  0. Verify: a 0 = = 0  a n = 2a n-1 +1 = 2(2 n-1 -1) +1 = 2 n = 2 n – 1  So, our guess is correct. a n = 2 n -1, for n  Recurrence Relation 15 Difficult to make a correct guess

Guess and Verify Find the closed form formula of a n = a n-1 +n for n >0, a 0 =1 a 0 = 1, a 1 = 1+1, a 2 = 1+1+2, a 3 = , a 4 = , a 5 = Guess: a n = 1+  0  i  n i, for n  0. a n = 1+n(n+1)/2 = (n 2 +n+2)/2 Verify: a 0 = ( )/2 = 1  a n = a n-1 +n =[(n-1) 2 +(n-1)+2]/2 +n = (n 2 +n+2)/2  So, our guess is correct. a n = (n 2 +n+2)/2, for n  Recurrence Relation 16

Unfold Find the closed form formula of a n = a n-1 +n for n >0, a 0 =1 a n = a n-1 +n = (a n-2 + n-1)+n = ((a n-3 + n-2) + n-1)+n … = ((…(a 0 +1)… + n-2) + n-1)+n = 1+1… + n-2 + n-1+n =1+n(n+1)/2 = (n 2 +n+2)/ Recurrence Relation 17

Recurrence in Tower of Hanoi H(1) = 1 H(n) = 2H(n-1) + 1 = 2(2H(n-2) +1) +1 = 4H(n-2) = 4(2(H(n-3) + 1) = 8H(n-3) = 8(2H(n-4) +1) = 16H(n-4) = 2 n n n-3 + … + 1 = 2 n - 1 Let x = 2 n n-2 + … x + 1=(2 n n-2 + … )+1 x + 1 = 2 n n-2 + … x + 1 = 2 n n-2 + … x + 1 = 2 n n-2 + … + 8 x + 1 = 2 n x = 2 n Recurrence Relation 18

Variable Transformation Find the closed form formula of a(n) = a(n/2) +1 for n =2 k, k  0 a 1 =1 Let b(k) = a(2 k ). From a(1)=1 a(n) = a(n/2) +1 We get b(0)=1 b(k) = b(k-1) +1 The closed form formula of b(k) is 1+ k. b(k) = 1+k = 1+ log 2 n= a(n). Thus, a(n) = 1+ log 2 n Recurrence Relation 19

Variable Transformation Find the closed form formula of a n = na n-1 +n! for n >0, a 0 =2. Divide the recurrence relation by n!, and we have a n /n! = a n-1 /(n-1)! + 1. Let b n = a n /n!. Then, b 0 =2b n = b n The closed form formula of b n is 2+ n. b n = a n /n! = 2+n. Thus, a n = n!(2 + n) Recurrence Relation 20

Variable Transformation Find the closed form formula of a n = (a n-1 ) 2 /a n-2 for n >1, a 0 =1, a 1 =2. Divide the recurrence relation by a n-1, and we have a n /a n-1 = a n-1 /a n-2. Let b n = a n /a n-1. Then, b n = b n-1 b 1 = a 1 /a 0 =2. The closed form formula of b n is 2. b n = a n /a n-1 = 2. Thus, a n = 2a n-1, and a n = 2 n Recurrence Relation 21

Using formula Linear homogeneous recurrence relation The sequence { a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 iff a n =  1 r n 1 +  2 r n 2, where : – r 1 and r 2 are two distinct roots of r 2 - c 1 r - c 2 = 0, – a 0 =  1 +  2 and a 1 =  1 r 1 +  2 r 2 (or –  1 = (a 1 – a 0 r 2 )/(r 1 - r 2 ) and  2 = (a 0 r 1 –a 1 )/(r 1 - r 2 ) ) Recurrence Relation 22

Where the formula comes from Let r 1 and r 2 be two distinct roots of r 2 – c 1 r – c 2 = 0. Let   and   be constants. Since r 1 and r 2 are roots of r 2 - c 1 r - c 2 = 0, r c 1 r 1 - c 2 = 0 (r 1 2 = c 1 r 1 + c 2 ) and r c 1 r 2 - c 2 = 0 (r 2 2 = c 1 r 2 + c 2 ). c 1 a n-1 + c 2 a n-2 = c 1 (  1 r 1 n-1 +  2 r 2 n-1 ) + c 2 (  1 r 1 n-2 +  2 r 2 n-2 ) = c 1  1 r 1 n-1 + c 1  2 r 2 n-1 + c 2  1 r 1 n-2 + c 2  2 r 2 n-2 = c 1  1 r 1 n-1 + c 2  1 r 1 n-2 +c 1  2 r 2 n-1 + c 2  2 r 2 n-2 =  1 r 1 n-2 (c 1 r 1 + c 2 ) +  2 r 2 n-2 (c 1 r 2 + c 2 ) =  1 r 1 n-2 r  2 r 2 n-2 r 2 2 =  1 r 1 n +  2 r 2 n = a n Recurrence Relation 23

Using formula Linear homogeneous recurrence relation The sequence { a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 +… + c k a n-k iff a n =  1 r n 1 +  2 r n 2 +…+  k r n k, where : – r 1, r 2,…, r k are k distinct roots of r k -c 1 r k-1 - c 2 r k-2 -…- c k = 0, – a 0 =  1 +  2 +…+  k – a 1 =  1 r 1 +  2 r 2 +…+  k r k – a 2 =  1 r  2 r 2 2 +…+  k r 2 k – … Recurrence Relation 24

Example Find the solution of a n = a n-1 + a n-2 with a 0 =0 a 1 =1 ? We have c 1 = 1 and c 2 = 1. The characteristic equation is r 2 - r - 1 = 0. The roots r 1 = (1+  5)/2 and r 2 = (1-  5)/2. From a 0 =  1 +  2 and a 1 =  1 r 1 +  2 r 2 0 =  1 +  2 and 1 =  1 r 1 +  2 r 2 1 =  1 r 1 -  1 r 2 =  1 (r 1 - r 2 ) =  1  5  1 = 1/  5  2 = -  1 = -1/  5   2 = -1/  5 a n =  1 r n 1 +  2 r n 2 = ((1+  5)/2) n /  5 - ((1-  5)/2) n /  5 a n = ( (1+  5) n - (1-  5) n )/(  5  2 n ) Recurrence Relation 25

Example What is the solution of a n = a n-1 + 2a n-2 with a 0 =2, a 1 =7 ? We have c 1 = 1 and c 2 = 2. The characteristic equation is r 2 - r - 2 = 0. The roots r 1 = 2 and r 2 = -1.  1 = (a 1 – a 0 r 2 )/(r 1 - r 2 ) = (7+2)/(2+1) = 3  2 = (a 0 r 1 – a 1 )/(r 1 - r 2 ) = (4-7)/(2+1) = -1 a n =  1 r n 1 +  2 r n 2 = 3  2 n - (-1) n Recurrence Relation 26

Example What is the solution of a n =6a n-1 -11a n-2 +6a n-3 with a 0 =2, a 1 =5, a 2 =15 ? The characteristic equation is r 3 - 6r r - 6 = 0, with roots r 1 = 1, r 2 = 2 and r 3 = 3. a 0 =  1 +  2 +  3 a 1 =  1 r 1 +  2 r 2 +  3 r 3 =    3 a 2 =  1 r  2 r  3 r 3 2 =    3  1 = 1,  2 = -1,  3 = 2 a n = 1 – 2 n + 2  3 n Recurrence Relation 27

Using formula Linear homogeneous recurrence relation The sequence { a n } is a solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 +… + c k a n-k iff a n = (  1,0 +  1,1 n + …+  1,m 1 n m 1 -1 ) r n 1 + (  2,0 +  2,1 n + …+  2,m 2 n m 2 -1 ) r n 2 +…+ (  t,0 +  t,1 n + …+  t,m t n m t -1 ) r n t, where : r 1, r 2,…, r t are distinct roots of r k -c 1 r k-1 - c 2 r k-2 -…- c k = 0, with multiplicities m 1, m 2,…, m t Recurrence Relation 28

Example What is the solution of a n = 6a n-1 -9a n-2 with a 0 =1, a 1 =6 ? We have c 1 = 6 and c 2 = -9. The characteristic equation is r 2 - 6r + 9 = 0. The root r 0 = 3.  1 = a 0 = 1  2 = a 1 /r 0 – a 0 = 6/3-1 = 1 a n =  1 r n 0 +  2 nr n 0 = 3 n + n 3 n = 3 n (1+ n) Recurrence Relation 29

Using formula Linear non-homogeneous recurrence relation If the sequence { a n (p) } is a particular solution of the recurrence relation a n = c 1 a n-1 + c 2 a n-2 +…+ c k a n-k + F(n) then every solution is of the form {a n (p) + a n (h) }, where { a n (h) } is a solution of the associated homogeneous recurrence relation a n = c 1 a n-1 + c 2 a n-2 +…+ c k a n-k Recurrence Relation 30

Example What are the solutions of a n = 3a n-1 + 2n with a 1 =3 ? We have c 1 = 3. The associated characteristic eq n is r-3=0, with root r = 3. Then, a n (h) =  3 n. Let a n (p) = cn +d. Then, from a n = 3a n-1 + 2n, cn +d = 3(c(n-1)+d ) + 2n. = (3cn + 2n) - 3c + 3d = (3c + 2)n - 3c + 3d c = 3c + 2, d = -3c + 3d. Thus, c = -1 and d = -3/2. That is, a n (p) = -n -3/2 a n = a n (h) + a n (p) =  3 n -n -3/2. From a 1 = 3 = 3  /2,  = 11/6. Solution: a n = (11/6)  3 n -n -3/ Recurrence Relation 31

Theorem Let { a n } be a solution of the linear non-homogeneous recurrence relation a n = c 1 a n-1 + c 2 a n-2 +… + c k a n-k + F(n), where c 1, c 2, …, c k are real numbers, and F(n) = (b t n t + b t-1 n t -1 +…+ b 1 n + b 0 ) s n, where b 0, b 1,…, b t and s are real numbers. When s is not a root of the characteristic equation of the associated linear homogeneous recurrence relation, there is a particular solution of the form (p t n t + p t-1 n t -1 +…+ p 1 n + p 0 ) s n Recurrence Relation 32

Theorem Let { a n } be a solution of the linear non-homogeneous recurrence relation a n = c 1 a n-1 + c 2 a n-2 +… + c k a n-k + F(n), where c 1, c 2, …, c k are real numbers, and F(n) = (b t n t + b t-1 n t -1 +…+ b 1 n + b 0 ) s n, where b 0, b 1,…, b t and s are real numbers. When s is a root of the characteristic equation with multiplicity m, there is a particular solution of the form n m (p t n t + p t-1 n t -1 +…+ p 1 n + p 0 ) s n Recurrence Relation 33

Example: Summation What are the solutions of a n = a n-1 + n with a 1 =1 ? We have c 1 = 1. The associated characteristic eq n is r-1=0, with root r = 1. Then, a n (h) =  1 n = . Let a n (p) = n(cn +d) = cn 2 +dn. Then, from a n = a n-1 + n, cn 2 +dn = c(n-1) 2 +d (n-1) + n. Thus, cn 2 +dn = cn 2 -2cn+c +d n - d + n, c-d + n(1-2c)=0. c-d =0 and 1-2c=0. That is, c=d=1/2. a n = a n (h) + a n (p) =  + n(n+1)/2. From a 1 = 1 =  +1,  = 0. The solution is a n = n(n+1)/ Recurrence Relation 34

DIVIDE-AND-CONQUER ALGORITHMS RECURRENCE AND Recurrence Relation 35

Divide-and-conquer and Recurrence Let f ( n ) be the number of operations required to solve a problem of size n. f (n) = a f (n/b) + g(n) Recurrence Relation 36 X X1X1 X2X2 X3X3 XaXa... n/b n elements

Binary Search To search for an element a from a sorted list X of size n elements. – If a = m, then stop – If a < m, then search for a from X 1. – If a > m, then search for a from X 2. Let f (n) be the number of comparisons in binary search within n elements. f (n) = f (  n/2  ) Recurrence Relation 37 X X1X1 X2X2 m

Master Method Let f be an eventually non-decreasing function that satisfies the recurrence relation f (n) = a f (n/b) + cn d whenever n  n 0, a  1, b > 1, c > 0, d  0, n = n 0 b k for any integer k > 0. Let n = n 0 b k. Then, f (n 0 b k )= a f (n 0 b k-1 ) + cn 0 d b kd. Let h i = f (n 0 b i ). Then, h i = a h i-1 + c (b d ) i, where c =cn 0 d. h i ( h) =  a i When a  b d : h i ( p) = p b di When a=b d : h i ( p) = pi a i Recurrence Relation 38

Master Method When a  b d : h i ( p) = p b di p b di = a p b d(i-1) + c (b d ) i p = a p/b di + c p (1 – a/b d ) = c p = c /(1 – a/b d ) h i = h i ( h) + h i ( p) =  a i + c /(1 – a/b d ) b di f (n) = f (n 0 b i ) = h i (i=log b n/n 0 ) =  a i + c /(1 – a/b d ) b di (a i = (n/n 0 ) log b a, b di = (n/n 0 ) d ) =  (n/n 0 ) log b a +c /(1–a/b d ) (n/n 0 ) d = c 1 n log b a + (c/(1–a/b d )) n d When a=b d : h i ( p) = pi a i pi a i = a p(i-1) a (i-1) + c a i p a i = c a i p = c = cn 0 d. h i = h i ( h) + h i ( p) =  a i + c i a i f (n) = f (n 0 b i ) = h i (i=log b n/n 0 ) = (  + c i)a i (a i = b d log b n/n0 = (n/n 0 ) d ) = (  + c i )(n/n 0 ) d = c 1 n d + cn d log b (n/n 0 ) Recurrence Relation 39

Master Method Let f be an eventually non-decreasing function that satisfies the recurrence relation f (n) = a f (n/b) + cn d whenever n  n 0, a  1, b > 1, c > 0, d  0, n = n 0 b k for any positive integer k. When a=b d, f(n)  c 1 n d log b n. When a>b d, f(n)  c 2 n log b a When a<b d, f(n)  c 3 n d Recurrence Relation 40

Binary Search If a = m, then stop If a < m, then search for a from X 1. If a > m, then search for a from X 2. Let f ( n ) be the # of comparison in binary search within n elements. f ( n ) = f ( n/2 ) + 2. Because a=1, f(n)  k log n Recurrence Relation 41

Merge sort procedure msort(L: list of a 1, a 2, …, a n ) if n>1 then { m :=  n/2  L1 := list of a 1, a 2, …, a m L2 := list of a m+1, a m+2, …, a n merge(msort(L1), msort(L2)) } f ( n ) = 2f ( n/2 ) + n. a = 2, b = 2, c = 1, d = 1 Since a=b d, f(n)  kn log n. procedure merge(A, B: list of n elements) i := 1j := 1k := 1 while i<n & j<n { if A[i]<B[j] then C[k++] := A[i++] else C[k++] := B[j++] } Recurrence Relation 42

Fast Multiplication for Large Integers a = 2 n A 1 + A 0. b = 2 n B 1 + B 0. A 1 A 0 B 1 B 0 a b = (2 2n + 2 n ) A 1 B n (A 1 -A 0 )(B 0 – B 1 ) + (2 n +1) A 0 B 0. Let f ( n ) be the number of operations needed to multiply two n -bit integers. f ( 2n ) = 3f ( n ) + Cn. a = 3, b = 2, c = C, d = 1. Because a>b d, f(n)  k n log Recurrence Relation

GENERATING FUNCTION Recurrence Relation 44

Generating function: Definition The generating function for the sequence a 0, a 1, …, a k, … of real numbers is the infinite series G(x) = a 0 + a 1 x + … + a k x k + … =   k =0 a k x k Examples: The generating function for 1, 2, 4,,…2 k x k, … is 1 + 2x + 4x 2 + … + 2 k x k + … The generating function for 1, -1, 1,,…(-1) k x k, … is 1 - x + x 2 - … + x 2k - x 2k+1 +… Recurrence Relation 45

Power Series Let f (x) =   k =0 a k x k and g(x) =   k =0 b k x k. Then, f (x) + g(x) =   k =0 (a k + b k )x k and f (x) g(x) =   k =0 (  k j=0 a j b k-j ) x k Recurrence Relation 46

Extended Binomial Theorem Let u be a real number and k be a nonnegative integer. The extended binomial coefficient C(u, k) or ( u k ) is  u(u-1)…(u-k+1)/k! if k > 0.   1 if k = 0. Let x be a real number with | x |<1 and u be a real number. Then, (1+x) u = x n ( u k ) x k Recurrence Relation 47

Useful Generating Functions G(x)G(x)akak (1 + x) n = 1 + C(n,1)x + C(n,2)x 2 + …+ x n. C(n, k) (1 + ax) n = 1 + C(n,1)ax + C(n,2)a 2 x 2 + …+ a n x n C(n, k) a k (1 + x r ) n = 1 + C(n,1)x r + C(n,2)x 2r + …+ x nr C(n, k/r) if r | k 0 otherwise 1/(1-x) =1 + x + x 2 + ….1 1/(1-ax) =1 + ax + a 2 x 2 + ….akak (1-x n+1 )/(1-x) =1 + x + x 2 + …+ x n. 1 if k  n 0 otherwise Recurrence Relation 48

Example Solve the recurrence relation a k = 3a k-1 with a 0 =2. Let G(x) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + … be the generating function for the sequence {a k }. G(x) = a 0 + a 1 x + a 2 x 2 + a 3 x 3 + … 3x G(x) = 3a 0 x + 3a 1 x 2 + 3a 2 x 3 + … (1-3x)G(x) = a 0 +(a 1 -3a 0 )x+(a 2 -3a 1 )x 2 +(a 3 -3a 2 ) x (1-3x)G(x) = a 0 = 2, and G(x) = 2/(1-3x) From 1/(1-ax) =1 + ax + a 2 x 2 +…, G(x)/2 = 1/(1-3x). Thus, a = 3, and a k = 2(3 k ) Recurrence Relation 49

Example Solve the recurrence relation a n = 8a n n-1 with a 0 =1. Multiply the recurrence relation by x n, and get a n x n = 8a n-1 x n + 10 n-1 x n Let G(x) =   n=0 a n x n be the generating function for the sequence {a n }. G(x) =   n=0 a n x n = a 0 +   n=1 a n x n G(x) - 1 = 8   n=1 a n-1 x n +   n=1 10 n-1 x n Recurrence Relation 50

Example G(x) - 1 = 8   n=1 a n-1 x n +   n=1 10 n-1 x n G(x) - 1 = 8x   n=1 a n-1 x n-1 + x   n=1 10 n-1 x n-1 G(x) - 1 = 8x   n=0 a n x n + x   n=0 10 n x n G(x) - 1 = 8x G(x) + x /(1 - 10x) (1 - 8x) G(x) = 1+ x /(1 - 10x) (1 - 8x) G(x) = (1 - 9x) /(1 - 10x) G(x) = (1 - 9x) /((1 - 10x) (1 - 8x)) Recurrence Relation 51

Example G(x) = (1 - 9x) /((1 - 10x) (1 - 8x)) Using partial fraction, we get G(x) = ½(1 /(1 - 10x) + 1/(1 - 8x)) From the known power series, G(x) = ½(   n=0 10 n x n +   n=0 8 n x n ) G(x) =   n=0 ½(10 n + 8 n ) x n Thus, we know that a n = ½(10 n + 8 n ) Recurrence Relation 52

Recurrence Relation 53 Aren’t you glad it’s done?