1. LEVEL II, TERM II CSE – 243 MD. MONJUR-UL-HASAN LECTURER DEPT OF CSE, CUET EMAIL: MAIL@MONJUR-UL-HASAN.INFO Recursive & Dynamic Programming

2. Introduction to Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 2 "Normally", we have methods that call other methods.  For example, the main() method calls the square () method. Recursive Method:  A recursive method is a method that calls itself. main() square() compute()

3. Why Recursive Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 3 In computer science, some problems are more easily solved by using recursive methods. In this course, will see many of examples of this. For example:  Traversing through directories of a file system.  Traversing through a tree of search results.  Some sorting algorithms recursively sort data For today, we will focus on the basic structure of using recursive methods.

4. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 4 LIST Consider the following list of numbers: 24, 88, 40, 37 Such a list can be defined as follows: A LIST is a: number or a: number comma LIST That is, a LIST is defined to be a single number, or a number followed by a comma followed by a LIST The concept of a LIST is used to define itself

5. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 5 The recursive part of the LIST definition is used several times, terminating with the non-recursive part: number comma LIST 24, 88, 40, 37 number comma LIST 88, 40, 37 number comma LIST 40, 37 number 37

6. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 6 GCD gcd(a,b) = a; when b =gcd(b,a%b); otherwise gcd(4032, 1272) = gcd(1272, 216) = gcd(216, 192) = gcd(192, 24) = gcd(24, 0) = 24.

7. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 7 Finding Power The power function, p(x,n)=x n, can be defined recursively: This leads to an power function that runs in O(n) time (for we make n recursive calls). We can do better than this, however.

8. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 8 Recursive Squaring We can derive a more efficient linearly recursive algorithm by using repeated squaring: For example, 2 4 = 2 ( 4 / 2 ) 2 = ( 2 4 / 2 ) 2 = ( 2 2 ) 2 = 4 2 = 16 2 5 = 2 1 +( 4 / 2 ) 2 = 2 ( 2 4 / 2 ) 2 = 2 ( 2 2 ) 2 = 2 ( 4 2 ) = 32 2 6 = 2 ( 6 / 2)2 = ( 2 6 / 2 ) 2 = ( 2 3 ) 2 = 8 2 = 64 2 7 = 2 1 +( 6 / 2 ) 2 = 2 ( 2 6 / 2 ) 2 = 2 ( 2 3 ) 2 = 2 ( 8 2 ) = 128. Better then this solution is also available For Practice Try to solve (b n %P) in this same way for large number of n

9. Recursive Situation Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 9 Reversing an Array Algorithm ReverseArray(A, i, j): Input: An array A and nonnegative integer indices i and j Output: The reversal of the elements in A starting at index i and ending at j if i < j then Swap A[i] and A[ j] ReverseArray(A, i + 1, j - 1) return

10. Recursive Definition Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 10 Base case: You must always have some base case which can be solved without recursion Making Progress: For cases that are to be solved recursively, the recursive call must always be a case that makes progress toward the base case. Design Rule: Assume that the recursive calls work. Compound Interest Rule: Never duplicate work by solving the same instance of a problem in separate recursive calls.

11. Visualizing Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 11 Recursion trace A box for each recursive call An arrow from each caller to callee An arrow from each callee to caller showing return value Example recursion trace: recursiveFactorial(3) (2) (1) (0) Return1 call return1*1=1 2* 1 =2 3*2=6 final answer

12. Visualizing Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 12 Time 2: Push: fact(3) main() fact(3) Time 3: Push: fact(2) Inside findFactorial(3): if (number <= 1) return 1; else return (3 * factorial (2)); main() fact(3) fact(2) Time 4: Push: fact(1) Inside findFactorial(2): if (number <= 1) return 1; else return (2 * factorial (1)); main() fact(3) fact(2) fact(1) Inside findFactorial(1): if (number <= 1) return 1; else return (1 * factorial (0)); Time 5: Pop: fact(1) returns 1. main() fact(3) fact(2) 1 Time 6: Pop: fact(2) returns 2. main() fact(3) 2 Time 7: Pop: fact(3) returns 6. main() 6

13. Several Types of Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 13 Liner Recursion  May be more than one recursive method available but for each recursion it calls only one among them Tail Recursion  The recursive call will be the last statement of the method Chain Recursion  Function A will call Function B. function B will call Function C ……. … last function will call function A again. Multiple Recursion  f(x) = a when x = 0 = b when x = 1 = a*f(x-1) + b*f(x-2) otherwise

14. Defining Arguments for Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 14 In creating recursive methods, it is important to define the methods in ways that facilitate recursion. This sometimes requires we define additional paramaters that are passed to the method. For example, we defined the array reversal method as ReverseArray(A, i, j), not ReverseArray(A). To verify that a recursive definition works :  convince yourself that the base case(s) are handled correctly  ASSUME RECURSIVE CALLS WORK ON SMALLER PROBLEMS, then convince yourself that the results from the recursive calls are combined to solve the whole

15. When recursion? Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 15  when it is the most natural way of thinking about & implementing a solution  can solve problem by breaking into smaller instances, solve, combine solutions  when it is roughly equivalent in efficiency to an iterative solution  OR  when the problems to be solved are so small that efficiency doesn't matter  think only one level deep  make sure the recursion handles the base case(s) correctly  assume recursive calls work correctly on smaller problems  make sure solutions to the recursive problems are combined correctly  avoid infinite recursion  make sure there is at least one base case & each recursive call gets closer

16. Alternative of Recursive Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 16 any recursive method/class can be rewritten iteratively (i.e., using a loop)  but sometimes, a recursive definition is MUCH clearer e.g., PermutationGenerator would be very difficult to conceptualize & implement without recursion Use stack to model the recursive into iterative method. [ use the data structure course concept] When overhead is low it is always better to use recursive but in other case it is better to use iterative solution.

17. Recursion pros and cons Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 17 All recursive solutions can be implemented without recursion. Recursion is "expensive". The expense of recursion lies in the fact that we have multiple activation frames and the fact that there is overhead involved with calling a method. If both of the above statements are true, why would we ever use recursion? In many cases, the extra "expense" of recursion is far outweighed by a simpler, clearer algorithm which leads to an implementation that is easier to code. Ultimately, the recursion is eliminated when the compiler creates assembly language (it does this by implementing the stack).

18. Think Your Own Recursion Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 18 Try to solve the problem for base case Think in the middle of the problem with following criteria:  Define that with same problem but with different domain  The change in domain should leads that to the base case. You need not to think all the problem together. Just think one step from middle and the base case.

19. Generating Permutation Systematically Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 19 numerous applications require systematically generating permutations (orderings)  e.g., word games, debugging concurrent systems, tournament pairings,...  want to be able to take some sequence of items (say a String of characters) and generate every possible arrangement without duplicates  "123"  "123", "132", "213", "231", "312", "321"  "tape"  "tape", "taep", "tpae", "tpea", "teap", "tepa",  "atpe", "atep", "apte", "apet", "aetp", "aept",  "ptae", "ptea", "pate", "paet", "peta", "peat",  "etap", "etpa", "eatp", "eapt", "epta", "epat“ how do we generate permutations systematically? must maintain initial ordering must get all permutations, with no duplicates

20. Generating Permutation Systematically Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 20 Generate all permutations that start with ‘t', then 'a' then ‘p‘ then ‘e’ To generate permutations starting with ‘t', we need to find all permutations of "ape" This is the same problem with simpler inputs. Use recursion To get your permutation go: http://home.att.net/~srschmitt/script_permutatio ns.html http://home.att.net/~srschmitt/script_permutatio ns.html

21. All Permutation Algorithm Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 21 consider P is a global array contain the string exch (int i, int j) { int t = p[i]; p[i] = p[j]; p[j] = t; } generate(int N) { int c; if (N == 1) doit(); for (c = 1; c <= N; c++) { exch(c, N); generate(N-1); exch(c, N); } } Invoke by calling generate(N);

22. Challenge to Students (75 Marks) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 22 1. Redesign the algorithm with no global variable 2. Redesign the algorithm that u develop in 1 to generate permutation which contain k element from the n length string 3. Redesign the algorithm of describe in 1 to generate all combination from a string by taking k element each time? 4. Design a correct algorithm to generate permutation when the string contain repetitive character in the given string 5. Design a correct Algorithm to generate Combination of k element among n element where the string contain repetitive character.

23. Marks Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 23  Fail to submit: -15 from your Lab work  Duplicate submission : add absolute 0 with your lab work  Correct solution will add marks as follows  Q1: 5  Q2 & Q2: 10+10 = 20  Q420  Q530  Total75 This mark will add to you lab result.

24. Recursive to Iterative Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 24 What is the problems with this recursion? Is there any direct solution available of this recursive function? How can we test and find if any such available.

25. Guess-and-Test Method Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 25 In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction: Guess: T(n) < cn log n. Wrong: we cannot make this last line be less than cnlogn

26. Guess-and-Test Method, Part 2 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 26 Recall the recurrence equation: Guess #2: T(n) < cn log 2 n.  if c > b. So, T(n) is O(n log 2 n). In general, to use this method, you need to have a good guess and you need to be good at induction proofs.

27. Master Method Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 27 Many divide-and-conquer recurrence equations have the form: The Master Theorem:

28. Master Method, Example 1 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 28 The form: The Master Theorem: Example: Solution: log b a=2, so case 1 says T(n) is O(n 2 ).

29. Master Method, Example 2 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 29 The form: The Master Theorem: Example: Solution: log b a=1, so case 2 says T(n) is O(n log 2 n).

30. Master Method, Example 3 Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 30 The form: The Master Theorem: Example: Solution: log b a=0, so case 3 says T(n) is O(n log n). As f(n) = nlogn is asymptotically larger then n log b a =n

31. Divide & Conquer Algorithm Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 31  Divide problem into sub-problems  Conquer by solving sub-problems recursively. If the sub-problems are small enough, solve them in brute force fashion  Combine the solutions of sub-problems into a solution of the original problem (tricky part) Example MergeSort Finding the middle point in the alignment matrix in linear space Linear space sequence alignment Block Alignment Four-Russians speedup Constructing LCS in sub-quadratic time

32. Dynamic Programming Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 32 fibonacci(5) fibonacci(4) + fibonacci(3) fibonacci(3) + fibonacci(2)fibonacci(2) + fibonacci(1) fibonacci(2) + fibonacci(1) Dynamic Programming is an algorithm design technique for optimization problems: often minimizing or maximizing. Like divide and conquer, DP solves problems by combining solutions to subproblems. Unlike divide and conquer, subproblems are not independent.  Subproblems may share subsubproblems,  However, solution to one subproblem may not affect the solutions to other subproblems of the same problem. (More on this later.)

33. Dynamic Programming: How to think? Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 33 DP reduces computation by  Solving subproblems in a bottom-up fashion.  Storing solution to a subproblem the first time it is solved.  Looking up the solution when subproblem is encountered again. Key: determine structure of optimal solutions Elements of Dynamic Programming Optimal substructure Overlapping subproblems Memorization

34. Optimizing Substructure Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 34 Show that a solution to a problem consists of making a choice, which leaves one or more subproblems to solve. Suppose that you are given this last choice that leads to an optimal solution. Given this choice, determine which subproblems arise and how to characterize the resulting space of subproblems. Show that the solutions to the subproblems used within the optimal solution must themselves be optimal. Usually use cut-and-paste. Need to ensure that a wide enough range of choices and subproblems are considered.

35. Optimal SubStructure Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 35 Optimal substructure varies across problem domains:  1. How many subproblems are used in an optimal solution.  2. How many choices in determining which subproblem(s) to use. Informally, running time depends on (# of subproblems overall)  (# of choices). How many subproblems and choices do the examples considered contain? Dynamic programming uses optimal substructure bottom up.  First find optimal solutions to subproblems.  Then choose which to use in optimal solution to the problem. Pieces of larger problem have a sequential dependency

36. Overlapping Subproblems Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 36 The space of subproblems must be “small”. The total number of distinct subproblems is a polynomial in the input size.  A recursive algorithm is exponential because it solves the same problems repeatedly.  If divide-and-conquer is applicable, then each problem solved will be brand new.

37. DP Approach Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 37 Forward Approach Backward Approach  Note that if the recurrence relations are formulated using the forward approach then the relations are solved backwards. i.e., beginning with the last decision  On the other hand if the relations are formulated using the backward approach, they are solved forwards. You can also represent the problem by a multistage graph

38. A Simple DP Approach Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 38 Fibonacci Numbers Int fib(int n) { static int knownFib[MAXFIB]; int x; if(knownFib[n]==0) { if(n==1 || n==2) knownFib[n] = 1; else knownFib[n] = fib(n-1) + fib(n-2); }

39. Matrix-chain multiplication Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 39 What will be the number of scalar multiplication in a matrix multiplication of a 3X5 and 5X 10 matrix? What will be new matrix’s direction? What is the number of scalar multiplication of the following matrices A 1 X A 2 X A 3 X A 4 Where, the damnations are 3x5, 5x4, 4x2 and 2x5 We have the following choice: ((A 1  A 2 )  A 3 )  A 4, # of scalar multiplications: 3 * 5 * 4 + 3 * 4 * 2 + 3 * 2 * 5 = 114 (A 1  (A 2  A 3 ))  A 4, # of scalar multiplications: 3 * 5 * 2 + 5 * 4 * 2 + 3 * 2 * 5 = 100 (A 1  A 2 )  (A 3  A 4 ), # of scalar multiplications: 3 * 5 * 4 + 3 * 4 * 5 + 4 * 2 * 5 = 160

40. Recursive Function Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 40 8 -40 Let m(i, j) denote the minimum cost for computing A i  A i+1  …  A j Computation sequence : Time complexity : O(n 3 )

41. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 41 Since subproblems overlap, we don’t use recursion. Instead, we construct optimal subproblems “bottom-up.” N i,i ’s are easy, so start with them. Then do length 2,3,… subproblems,and so on. Running time: O(n3)

42. Matrix Chan Multiplication Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 42 Algorithm matrixChain(S): Input: sequence S of n matrices to be multiplied Output: number of operations in an optimal paranethization of S 1. for i ← 1 to n-1 do 1. Ni,i ← 0 2. for b ← 1 to n-1 do 1. for i ← 0 to n-b-1 do 1. j ← i+b 2. Ni,j ← +infinity 3. for k ← i to j-1 do 1. Ni,j ← min{Ni,j, Ni,k +Nk+1,j +di dk+1 dj+1} 2. Si,j ← k //for which Ni,j Minimum

43. Matrix Chan Multiplication Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 43 Print the Optimal Structure OPTIMAL-MAT(s,i,j) { if(i==j) then print “A”I else print “(“ OPTIMAL-MAT(s,i,s[i,j) OPTIMAL-MAT(s,s[i,j]+1, j) print “)”; }

44. Longest Common Subsequence (LCS) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 44 Problem: Given 2 sequences, X =  x 1,...,x m  and Y =  y 1,...,y n , find a common subsequence whose length is maximum. springtimencaa tournamentbasketball printingnorth carolinakrzyzewski Subsequence need not be consecutive, but must be in order.

45. LCS (Native Algorithm) Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 45 For every subsequence of X, check whether it’s a subsequence of Y. Time: Θ(n2 m ).  2 m subsequences of X to check.  Each subsequence takes Θ(n) time to check: scan Y for first letter, for second, and so on.

46. LCS: Optimal Substructure Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 46 Notation: prefix X i =  x 1,...,x i  is the first i letters of X. This says what any longest common subsequence must look like; do you believe it? Theorem Let Z =  z 1,..., z k  be any LCS of X and Y. 1. If x m = y n, then z k = x m = y n and Z k-1 is an LCS of X m-1 and Y n-1. 2. If x m  y n, then either z k  x m and Z is an LCS of X m-1 and Y. 3. or z k  y n and Z is an LCS of X and Y n-1. Theorem Let Z =  z 1,..., z k  be any LCS of X and Y. 1. If x m = y n, then z k = x m = y n and Z k-1 is an LCS of X m-1 and Y n-1. 2. If x m  y n, then either z k  x m and Z is an LCS of X m-1 and Y. 3. or z k  y n and Z is an LCS of X and Y n-1.

47. Recursive Solution Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 47 c[springtime, printing] c[springtim, printing] c[springtime, printin] [springti, printing] [springtim, printin] [springtim, printin] [springtime, printi] [springt, printing] [springti, printin] [springtim, printi] [springtime, print]

48. Memorization Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 48 printing s p r i n g t i m e Keep track of c[  ] in a table of nm entries: top/down bottom/up

49. LCS: Length Finding Algorithm Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 49 LCS-LENGTH (X, Y) 1. m ← length[X] 2. n ← length[Y] 3. for i ← 1 to m 4. do c[i, 0] ← 0 5. for j ← 0 to n 6. do c[0, j ] ← 0 7. for i ← 1 to m 8. do for j ← 1 to n 9. do if x i = y j 10. then c[i, j ] ← c[i  1, j  1] + 1 11. b[i, j ] ← “ ” 12. else if c[i  1, j ] ≥ c[i, j  1] 13. then c[i, j ] ← c[i  1, j ] 14. b[i, j ] ← “ ↑ ” 15. else c[i, j ] ← c[i, j  1] 16. b[i, j ] ← “ ← ” 17. return c and b

50. Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 50 printing s P r i n g t i m e printing 000000000 s000000000 p011111111 r012222222 i012333333 n012344444 g012344445 t012345555 i012345666 m012345666 e012345666

51. LCS: Constructing Md. Monjur-ul-hasan. Lecturer, Dept. of CSE, CUET 51 PRINT-LCS (b, X, i, j) 1. if i = 0 or j = 0 2. then return 3. if b[i, j ] = “ ” 4. then PRINT-LCS(b, X, i  1, j  1) 5. print x i 6. elseif b[i, j ] = “ ↑ ” 7. then PRINT-LCS(b, X, i  1, j) 8. else PRINT-LCS(b, X, i, j  1)