Solving Recurrence Lecture 19: Nov 25

Some Recursive Programming (Optional?)

Test int hello(int n) { if (n==0) return 0; else printf(“Hello World %d\n”,n); hello(n-1); } 1.What would the program do if I call hello(10)? 2.What if I call hello(-1)? 3.What if the order of printf() and hello() is reversed?

Computing Sum of Arithmetic Progression int AP(int n) { if (n==0) return 0; else return (n+AP(n-1)); } Many programs can be written in a recursive way. The way of thinking is different. Always try to reduce it to smaller problems.

Computing Exponential Function int EX(int n) { if (n==0) return 1; else return (EX(n-1)+EX(n-1)); } How many function calls if I run EX(n)? This function is to compute 2 n. 2 n times. If we replace the last line by return 2EX(n-1), then the program will compute the same thing, but there will be only n function calls.

Move 1,2 (n)::= Move 1,3 (n-1); biggest disk 1  2; Move 3,2 (n-1) Tower of Hanoi To move the biggest disk, we must first move the disks on top to another post.

Tower of Hanoi T11(a,b) { T10(a,c); printf(“Move Disk 11 from a to b\n”); T10(c,b); } Suppose we already have a function T10(a,b) to move 10 disks from pole a to pole b. It is then easy to write a program for T11(a,b) In general you can write exactly the same program for Tn, if we are given a program for Tn-1. Instead of writing one program for each n, we can take n as an input and write a recursive program.

Tower of Hanoi Tower_of_Hanoi(int origin, int destination, int buffer, int number) { if (n==0) return; Tower_of_Hanoi(origin, buffer, destination, number-1); printf(“Move Disk #%d from %d to %d\n”, number, origin, destination); Tower_of_Hanoi(buffer, destination, origin, number-1); } This is the power of recursive thinking. The program is very short, yet it is not easy to write it without recursion

Tower of Hanoi T(A,C,B,3) Tower_of_Hanoi(origin, destination, buffer, number) Move 3 from A to C. T(A,B,C,2) T(B,C,A,2) move 2 from A to B move 2 from B to C T(A,C,B,1) T(C,B,A,1) T(B,A,C,1) T(A,C,B,1) move 1 from A to C move 1 from C to B move 1 from B to A move 1 from A to C

Generate Strings Can you write a program to generate all n-bit strings with k ones? There are many ways to do it. (1)Generate all n-bit strings and output those strings with k ones. - Too slow (2) Write k for-loops. - That’s okay if k is known, but what if k is part of the input? (3) Write a program to write a program with k for loops. - That’s okay, but can we do it more elegantly? (4) Write a recursive program for it.

Generate Strings The idea of the recursive program is simple. First we generate all strings which begin with 1, and then generate all strings which begin with 0. Can you write a program to generate all n-bit strings with k ones? For strings which begin with 1, we still have n-1 bits and k-1 ones left. For strings which begin with 0, we still have n-1 bits and k ones left. Let’s say the program is called gen(n,k). gen(n-1,k-1) gen(n-1,k) Sounds familiar?

Generate Strings Can you write a program to generate all N-bit strings with K ones? int solution[N]; (an array holding an N-bit string, from solution[0] to solution[n-1]) gen(int n, int k) (n is # bits left, and k is # ones left) { if (n==0) (no more bits left) print solution; (write a for loop to print out the N-bits in solution) return; if (k > 0) solution[N-n] = 1; (generate the strings beginning with one) gen(n-1,k-1); if (n > k) (do it only if there are enough places for the ones) solution[N-n]=0; (generate the strings beginning with zero) gen(n-1,k); }

Programming Exercises Can you write a recursive program to generate all permutations of n elements? Can you write a recursive program for merge-sort?

Solving Recurrence

Warm Up a 0 =1, a k = a k a 1 = a a 2 = a = (a 0 + 2) + 2 = a a 3 = a = (a 0 + 4) + 2 = a a 4 = a = (a 0 + 6) + 2 = a See the pattern is a k = a 0 + 2k = 2k+1 You can verify by induction.

Solving Hanoi Sequence a 1 =1, a k = 2a k a 2 = 2a = 3 a 3 = 2a = 2(2a 1 + 1) + 1 = 4a = 7 a 4 = 2a = 2(4a 1 + 3) + 1 = 8a = 15 a 5 = 2a = 2(8a 1 + 7) + 1 = 16a = 31 a 6 = 2a = 2(16a ) + 1 = 32a = 63 Guess the pattern is a k = 2 k -1 You can verify by induction.

Solving Merge Sort Recurrence T 2k = 2T k + 2k If we could guess that T k is k log 2 k, then we can prove that T 2k is 2k log 2 (2k). This is because T 2k = 2T k + 2k = 2klog 2 k + 2k = 2k(log 2 k + 1) = 2k(log 2 k + log 2 2) = 2klog 2 2k

Solving Merge Sort Recurrence T 2k = 2T k + 2k How could we guess T k = k log 2 k in the first place? If there are n numbers there are log 2 n levels. In each level i we need to solve 2 i-1 merge problems. Each merge problem in level i has two subseqences of n/2 i numbers, and so can be solved in n/2 i-1 steps. So each level requires a total of (2 i-1 )(n/2 i-1 )=n steps. Since there are log 2 n levels, the total number of steps is at most nlog 2 n. Level 3 Level 1 Level 2

Solving Fibonacci Sequence a 0 =0, a 1 =1, a k = a k-1 + a k-2 a 2 = a 1 + a 0 = 1 a 3 = a 2 + a 1 = 2a 1 + a 0 = 2 a 4 = a 3 + a 2 = 2a 2 + a 1 = 3a 1 + 2a 0 = 3 a 5 = a 4 + a 3 = 2a 3 + a 2 = 3a 2 + 2a 1 = 5a 1 + 3a 0 = 5 a 6 = a 5 + a 4 = 2a 4 + a 3 = 3a 3 + 2a 2 = 5a 2 + 3a 1 = 8a 1 + 5a 0 = 8 a 7 = a 6 + a 5 = 2a 5 + a 4 = 3a 4 + 2a 3 = 5a 3 + 3a 2 = 8a 2 + 5a 1 = 13a 1 + 8a 0 = 13 See the pattern a n = a n-k a k+1 + a n-k-1 a k but this does not give a formula for computing a n

Second Order Recurrence Relation In the textbook it is called “second-order linear homogeneous recurrence relation with constant coefficients”. a k = Aa k-1 + Ba k-2 A and B are real numbers and B≠0 For example, Fibonacci sequence is when A=B=1. To solve this equation, first we will find some solutions, and later show that they “generate” all possible solutions.

Geometric-Sequence Solution a k = Aa k-1 + Ba k-2 Find solutions of the form (1, t, t 2, t 3, t 4, …, t n, …) t k = At k-1 + Bt k-2 That is, suppose a k =t k t 2 = At + B t 2 - At – B = 0 So t is a root of the quadratic equation t 2 - At – B = 0.

Example a k = a k-1 + 2a k-2 Find solutions of the form (1, t, t 2, t 3, t 4, …, t n, …) So t must be a root of the quadratic equation t 2 - t – 2 = 0. This implies that t=2 or t=-1. So solutions of the form (1, t, t 2, t 3, t 4, …, t n, …) are: (i) (1,2,4,8,16,32,64,…) (ii) (1,-1,1,-1,1,-1,…)

Example a k = a k-1 + 2a k-2 So solutions of the form (1, t, t 2, t 3, t 4, …, t n, …) are: (i) (1,2,4,8,16,32,64,…) (ii) (1,-1,1,-1,1,-1,1,…) Are there other solutions? Try (2,1,5,7,17,31,65,…) (0,3,3,9,15,33,63,…) (4,5,13,23,49,95,193,…) How to obtain these solutions?

Linear Combination of Two Solutions If (r 0,r 1,r 2,r 3,…) and (s 0,s 1,s 2,s 3,…) are solutions to a k = Aa k-1 + Ba k-2, then the sequence (a 0,a 1,a 2,a 3,…) defined by the formula a k = Cr k + Ds k also satisfies the same recurrence relation for any C and D. This says that any linear combination of two solutions for the recurrence relation is also a solution for the recurrence. (This is easy to check.)

Linear Combination of Two Solutions Are there other solutions than a k = Cr k + Ds k ? Any solution to a k = Aa k-1 + Ba k-2 is determined by the first two terms a 0 and a 1 Suppose we already have two solutions (r 0,r 1,r 2,r 3,…) and (s 0,s 1,s 2,s 3,…). If (r 0,r 1 ) and (s 0,s 1 ) are linearly independent, then any possible (a 0,a 1 ) is a linear combination of (r 0,r 1 ) and (s 0,s 1 ), and thus any solution to the recurrence relation a k = Aa k-1 + Ba k-2 is a linear combination of (r 0,r 1,r 2,r 3,…) and (s 0,s 1,s 2,s 3,…).

Distinct-Roots Theorem Suppose a sequence (a 0,a 1,a 2,a 3,…) satisfies a recurrence relation a k = Aa k-1 + Ba k-2 If t 2 - At – B = 0 has two distinct roots r and s, then a n = Cr n + Ds n for some C and D. If we are given a 0 and a 1, then C and D are uniquely determined. The theorem says that all the solutions of the recurrence relation are a linear combination of the two series (1,r,r 2,r 3,r 4,…,r n,…) and (1,s,s 2,s 3,s 4,…,s n,…) defined by the distinct roots of t 2 - At – B = 0, because the two series are linearly independent.

Solving Fibonacci Sequence a 0 =0, a 1 =1, a k = a k-1 + a k-2 First solve the quadratic equation t 2 - t – 1 = 0. So the distinct roots are:

Solving Fibonacci Sequence a 0 =0, a 1 =1, a k = a k-1 + a k-2 By the distinct-roots theorem, the solutions satisfy the formula: To figure out C and D, we substitute the value of a 0 and a 1 :

Solving these two equations, we get: Therefore: Solving Fibonacci Sequence

Quick Summary Recursion is a very useful and powerful technique in computer science. It is very important to learn to think recursively, by reducing the problem into smaller problems. This is a necessary skill to acquire to become a professional programmer. Make sure you have more practices in setting up recurrence relations.