Prof. Amr Goneid, AUC1 Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 9. Intermezzo

Prof. Amr Goneid, AUC2

3 1. Fibonacci Numbers 2. Computing Binomial Coefficients 3. The Sum of Subset Problem 4. Summary

Prof. Amr Goneid, AUC4 1. Fibonacci Numbers Fibonacci numbers represent the sequence: 0,1,1,2,3,5,8,13,21,34,55,89,144,233,377,610,... Introduced by Leonardo Fibonacci (1202) Can be computed by the recurrence relation: F(n) = F(n-2) + F(n-1) n ≥ 2 with F(0)=F(1)=1

Prof. Amr Goneid, AUC5 Fibonacci Numbers Fibonacci numbers are closely related to the Golden Ratio φ since:

Prof. Amr Goneid, AUC6 The Golden Ratio

Prof. Amr Goneid, AUC7 In Nature

Prof. Amr Goneid, AUC8 In Nature

Prof. Amr Goneid, AUC9 In Architecture & Design

Prof. Amr Goneid, AUC10 In ART

Prof. Amr Goneid, AUC11 In Art

Prof. Amr Goneid, AUC12 Divide & Conquer Algorithm Divide & Conquer Algorithm implemented as a recursive function: int Fib(int n) { if (n < 2) return 1; else return Fib(n-2) + Fib(n-1); }

Prof. Amr Goneid, AUC13 Divide & Conquer Algorithm Analysis Tracing for n = n12345 Calls135915

Prof. Amr Goneid, AUC14 Divide & Conquer Algorithm This solution is an example of Overlapping Subproblems. It means that any recursive algorithm solving the problem should solve the same subproblems over and over, rather than generating new subproblems..

Prof. Amr Goneid, AUC15 Analysis The number of calls T(n) = T(n-1) + T(n-2) + 1 with T(0)=T(1)=1 The cost is very high because we keep computing the same instance over and over again.

Prof. Amr Goneid, AUC16 2. Computing Binomial Coefficients A binomial is an expression like (x + y). Algebraic expansion of powers of a binomial is done through the Binomial Theorem:

Prof. Amr Goneid, AUC17 Computing Binomial Coefficients The coefficients are represented by the Pascal Triangle, named after the French mathematician Blaise Pascal (1665).

Prof. Amr Goneid, AUC18 Computing Binomial Coefficients Previously, the triangle has been investigated by Omar Khayyam ( ) Awake! for Morning in the Bowl of Night Has flung the Stone that puts the Stars to Flight; And Lo! the Hunter of the East has caught The Sultan ’ s Turret in a Noose of Light! [from Fitzgerald’s translation of the Rubaiyat of Omar Khayyam]

Prof. Amr Goneid, AUC19 Computing Binomial Coefficients The coefficients are related to the Fibonacci numbers. The sums of the “hidden“ diagonals give the Fibonacci sequence.

Prof. Amr Goneid, AUC20 Computing Binomial Coefficients The coefficients also represent the number of combinations of n items taken m at a time. Given m <= n, to choose m things out of n, either: Choose the first item. Then we must choose the remaining m−1 items from the other n−1 items. Or Don’t choose the first item. Then we must choose the m items from the other n − 1 items.

Prof. Amr Goneid, AUC21 Computing Binomial Coefficients This leads to the following recurrence relation: Again, this suggests a Divide & Conquer algorithm

Prof. Amr Goneid, AUC22 Divide & Conquer Algorithm int comb (int n, int m) { if ((m == 0) || (m == n)) return1; else return comb (n − 1, m − 1) + comb (n − 1, m); } Analysis Let T(n) = worst case running time over all values of m. Then, T(n) = 2T(n − 1) + 1 for n > 1 with T(1) = 1 Hence T(n) = 2 n - 1 = O(2 n ) = exponential Time

Prof. Amr Goneid, AUC23 Counting Combinations Again the cost is very high because we keep computing the same instance over and over again, i.e., an overlapping subproblem. 5,3 4,24,3 3,13,23,3 2,12,2 1,01,1 3,2 2,12,2 1,01,1

Prof. Amr Goneid, AUC24 3. The Sum of Subset Problem Given a set of positive integers W = {w 1,w 2...w n } The problem: is there a subset of W that sums exactly to m? i.e, is SumSub (w,n,m) true? Example: W = { 11, 13, 27, 7}, m = 31 A possible subset that sums exactly to 31 is {11, 13, 7} Hence, SumSub (w,4,31) is true w1w1 w2w2....wnwn m

Prof. Amr Goneid, AUC25 Divide & Conquer Approach Consider the partial problem SumSub (w,i,j) SumSub (w, i, j) is true if: w i is not needed, {w 1,..,w i-1 } has a subset that sums to (j), i.e., SumSub (w, i-1, j) is true, OR w i is needed to fill the rest of (j), i.e., {w 1,..,w i-1 } has a subset that sums to (j-w i ) If there are no elements, i.e. (i = 0) then SumSub (w, 0, j) is true if (j = 0) and false otherwise w1w1 w2w2....wiwi j

Prof. Amr Goneid, AUC26 Divide & Conquer Approach Algorithm: bool SumSub (w, i, j) { if (i == 0) return (j == 0); else if (SumSub (w, i-1, j)) return true; else if ((j - w i ) >= 0) return SumSub (w, i-1, j - w i ); else return false; }

Prof. Amr Goneid, AUC27 Divide & Conquer Approach Analysis: T(n) = no. of calls to SumSub (w, n, m): For n = 0, one main call, T(0) = 1 For n > 0, one main call plus two calls each with n-1 The recurrence relation is: T(n) = 2T(n-1) + 1 for n > 0 with T(0) = 1 Hence T(n) = 2 n+1 -1 = O(2 n ) = exponential time Can We Do Better?

Prof. Amr Goneid, AUC28 4. Summary The recursive Fibonacci algorithm, recursive combination counting and sum of subsets are examples of problems where: Problem is solved by divide & conquer methods Same sub-problems are solved repeatedly This produces an exponential time Conclusion: Sometimes, the divide and conquer approach seems appropriate but fails to produce an efficient algorithm.