1. An Example in the Design and Analysis of an Algorithm Demonstrates: -- Recurrence Relations -- Design Techniques -- The fact that analysis provide useful information Problem to be solved: BUNNY PROBLEM When a pair of rabbits is less than one month old, it cannot reproduce, but during the second and each succeeding month of life, each pair of rabbits gives rise to one new pair of rabbits. Given that you (Fibonacci) just purchased a pair of new born rabbits, how many pairs will you have n months from now?

2. Illustration

3. Observation MonthPairs of newborn rabbits at month’s end (S(n)) Pairs of mature rabbits at month’s end (B(n)) Total pairs of rabbits at month’s end (f(n)) 01234567…01234567… 10112358…10112358… 0 1 2 3 5 8 13 … 1 2 3 5 8 13 21 …

4. The Recurrence Relation i  0 1 2 3 4 5 6 7 8 9... f(i)  1 1 2 3 5 8 13 21 34 55... f(i) satisfies the following recurrence relation f(i) = f(i-1) + f(i-2) for I ≥ 2, f(0) = f(1) = 1 B(n+1) = B(n) + S(n) = f(n) S(n+1) = B(n) = f(n-1) f(n+1) = B(n+1) + S(n+1) = f(n) + f(n-1)

5. Solution #1 1.Function f(n: integer): integer; 2. begin 3.if n ≤ 1 then return(1) 4. else return(f(n-1)+f(n-2)) 5. end. Analysis: Two factors -- # of function calls and # of additions. Let FC(n) = # of function calls needed to evaluate f(n) ADD(n) = # of additions needed

6. Consider a tree of function calls (showing the value of n at each node). Analysis

7. Thus, we have the recurrence relations FC(n) = FC(n-1) + FC(n-2) + 1, if n≥2 FC(0) = FC(1) = 1 ADD(n) = ADD(n-1) + ADD(n-2) + 1, if n≥2 ADD(0) = ADD(1) = 0 Later, we will learn how to solve such equations. For now f(n) = FC(n) = 2f(n)-1, ADD(n) = f(n)-1 Derivation

8. Since Hence, f(n) = = ADD(n) and FC(n) = O(1.618 n ) Exponential!! Asymptotic Performance

9. Avoid Redundant Computations We can use the dynamic programming technique. Idea: Solving the problem (calculating f(n)) consists of solving two smaller problems (calculating f(n-1) and f(n- 2)) and "merging" (i.e., adding) the two smaller solutions to obtain a solution to the original problem. Also, we will have to solve the same problem more than once. When a problem has these characteristics, build up a TABLE of SOLUTIONS to the smaller problems, starting with the smallest, and proceed upwards to the largest (original) problem.

10. Solution #2 1.Program FIB 2. var f: array[0..n] of integer; i: integer; 3.begin 4.f[0] := 1; 5.f[1] := 1; 6.for i := 2 to n do f[i] := f[i-1] + f[i-2]; 7. end. The answer is stored in f[n].

11. Analysis Clearly n-1 operations (additions) are required -- time is O(n). Space required is also O(n). (Space can be reduce to O(1).) This method usually trades space for time (though not in this case). LINEAR!! a vast improvement.

12. Modified Solution #2 1.Program FIB 2. var a, b, i: integer; 3.begin 4.a := 1; 5.b := 1; 6.for i := 2 to n do 7. begin b:= a + b; a := b – a end; 8. end. The answer is stored in the variable “b”.

13. Interlude This still doesn’t seem very good. Do we need to calculate all the f(i)? Can't we skip over some? Try substituting recurrence into itself. f(n) = f(n-1) + f(n-2) = f(n-2) + f(n-3) + f(n-2) = 2f(n-2) + f(n-3) = 3f(n-3) + 2f(n-4) = 5f(n-4) + 3f(n-5) = 8f(n-5) + 5f(n-6) (these #s look familiar ) Conjecture: f(n) = f(a)f(n-a) + f(a-1)f(n-a-1) where n≥2; n-1 ≥ a ≥1.

14. 1.Function f(x : integer) : integer; 2.var i, j: integer; 3.begin 4.if x ≤1 then return(1) 5. else begin 6. j :=  x/2  ; 7. i :=  x/2  ; 8. return(f(i)*f(j)+f(i-1)*f(j-1)); 9. end; 10.end. Solution #3

15. Analysis Why did we choose i and j  x/2? Another design heuristic -- BALANCING: divide problem sizes as evenly as possible. "Divide and Conquer". Let n = 100 Look at "tree" of function calls (notice that FC(n) = 4FC(n/2)+1, FC(1)=1).

16. Analysis

17. What Happened? But obviously the above can be improved. f(100) calls f(50) twice! Another important technique -- (subtree isomorphism) Eliminate REDUNDANT Computation

18. A Possible Improvement Consider 2 cases: –x is even then i = j we want to return f(i) 2 +f(i-1) 2. –x is odd then i-1 = j we want to return f(i-1)*(f(i)+f(i-2)). What does the tree look like now? –An even node has 1 even son and 1 odd son. –Some odd nodes have 2 odd and 1 even sons (5, 9, 13,..., i.e.  1 mod 4). –The rest have 1 odd and 2 even sons (  3 mod 4). The tree has between O(n 1.271 ) and O(n 1.385 ) nodes. Using the relation f(i) = f(i-1) + f(i-2), we have f(i-2) = f(i) - f(i-1), the "odd" return value can be further simplified: f(i-1)(f(i)+f(i-2)) = f(i-1)(2f(i)-f(i-1))

19. Solution #4 1.Function f(x : integer) : integer; 2.var i, FI, FIM1: integer; 3.begin 4.if x ≤1 then return(1) 5. else begin 6. i :=  x/2  ; 7. FI := f(i); 8. FIM1 := f(i-1); 9. if x is even 10. then return(FI**2+FIM1**2) 11. else return(FIM1*(2*FI-FIM1)) 12. end; 13.end.

20. Analysis Tree of calls is now completely binary. Take n = 32, Height of tree is about log n, hence it has about n nodes, i.e., FC(n) = O(n).

21. Are You Satisfied? Obviously there still is some redundant computation. Can all the calls at one level be done only once? Then we would have:

22. Solution #5 Procedure calc(x : integer; var fx, fxm1: integer); var i, fi, fim1, fim2: integer; begin if x ≤ 1 then begin fx := 1; fxm1 := 1 end else begin i :=  x/2  ; calc(i, fi, fim1) if x is even then begin fx := fi**2+fim1**2; fxm1 := fim1*(2*fi-fim1)) end else begin fim2 := fi-fim1; fx := fim1*(2*fi-fim1)); fxm1 := fim1**2+fim2**2 end; end. Analysis: Tree obviously has  (log n) nodes!

23. Solution #6

24. Repeated Squaring X(n) = A  X(n-1) = A  (A  X(n-2)) = … = A n-1  X(1). Use this relation, we can design a  (log n) algorithm for calculating Fibonacci numbers. Hint: Calculate A, A 2, A 4, …, A 2  log n , then compute A n-1 using these items. Questions: Is there a better way for calculating A n-1 ?

25. Exercise Implement the above solutions 1, 2 and 6, and compare their running time for calculating f(50), f(100), and f(150). Can your programs calculate f(500)? How about f(1000)? Now, you should realize that real implementation sometimes take a lot of efforts even you understand the underlying algorithm.