1. Dynamic Programming Lecture 9 Asst. Prof. Dr. İlker Kocabaş 1

2. Dynamic Programming History Bellman. Pioneered the systematic study of dynamic programming in the 1950s. Etymology. –Dynamic programming = planning over time. –Secretary of Defense was hostile to mathematical research. –Bellman sought an impressive name to avoid confrontation. "it's impossible to use dynamic in a pejorative sense" "something not even a Congressman could object to" Reference: Bellman, R. E. Eye of the Hurricane, An Autobiography. 2

3. Introduction Dynamic Programming(DP) applies to optimiza- tion problems in which a set of choices must be made in order to arrive at an optimal solution. As choices are made, subproblems of the same form arise. DP is effective when a given problem may arise from more than one partial set of choices. The key technique is to store the solution to each subproblem in case it should appear 3

4. Introduction (cont.) Divide and Conquer algorithms partition the problem into independent subproblems. Dynamic Programming is applicable when the subproblems are not independent.(In this case DP algorithm does more work than necessary) Dynamic Programming algorithm solves every subproblem just once and then saves its answer in a table. 4

5. Dynamic Programming Applications Areas. –Bioinformatics. –Control theory. –Information theory. –Operations research. –Computer science: theory, graphics, AI, systems, …. Some famous dynamic programming algorithms. –Viterbi for hidden Markov models. –Unix diff for comparing two files. –Smith-Waterman for sequence alignment. –Bellman-Ford for shortest path routing in networks. –Cocke-Kasami-Younger for parsing context free grammars. 5

6. Assembly-line Scheduling 6

7. If we are given a list of which stations to use in line 1 and which to use in line 2, then T ( n)=Θ(n). There are 2 n possible ways to choose stations. Thus, the solution is obtained by enumarating all posible ways and computing how long each takes would take Ω(2 n ). 7

8. The steps of a dynamic programming Characterize the structure of an optimal solution Recursively define the value of an optimal solution Compute the value of an optimal solution in a bottom-up fashion Construct an optimal solution from computed information 8

9. Step 1:The structure of the fastest way through the factory Consider the fastest possible way(fpw) to get from the starting point through station S 1,j. If j=1, there is only one way that the chassis could have gone. For j=2, 3,..., n there are two choices: –The chassis could have come from S 1, j-1 and then directly to station S 1, j. –The chassis could have come from S 2, j-1 and then directly to station S 1, j-1. 9

10. Step 1:The structure of the fastest way through the factory (cont.) First, suppose that the fastest way(fw) through station S 1, j is through station S 1, j-1. The key observation is that the chassis must have taken a fw from the starting point through station S 1, j-1. 10

11. Step 1:The structure of the fastest way through the factory (cont.) Similarly, suppose that the fw through station S 2, j is through station S 2, j-1. We observe that the chassis must have taken a fw from tha starting point through station S 2, j-1. 11

12. Step 1:The structure of the fastest way through the factory (cont.) We can say that for assembly-line scheduling, an optimal solution to a problem contains within it an optimal solution to sub-problems (finding the fw through either S 1, j-1 or S 2, j-1 ). We refer to this property as optimal substructure. 12

13. The fw through stations S 1, j The fw through station S 1, j-1 and then directly through station S 1, j. or The fw through station S 2, j-1 and then through station S 1, j. 13

14. The fw through station S 2, j The fw through station S 2, j-1 and then directly through station S 2, j. or The fw through station S 1, j-1 and then through station S 2, j. 14

15. Step 1:The structure of the fastest way through the factory (cont.) To solve the problem of finding the fw through station j of either line, we solve the subproblems of finding the fw through station j-1 on both lines. Thus, we can build an optimal solution to an instance of the assemly line scheduling problem by building on optimal solutions to subproblems 15

16. Step 2: A recursive solution We define the value of an optimal solution recursively in tems of the optimal solutions to subproblems. Let f i [j] denote the fastest possible time to get a chassis from the starting point throuhg station S i, j. Let f i * denote the fastest time to get a chassis all the way through the factory. 16

17. Step 2: A recursive solution (cont.) 17

18. Step 2: A recursive solution (cont.) 18

19. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 4 2 7 93 8 5 4 3 2 3 1 34 4 21 5 2 1 8 65 4 7 2 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 19

20. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 4 9 93 8 5 4 3 2 3 1 34 21 5 2 1 12 65 4 7 2 9 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 20 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6

21. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6 4 9 183 8 5 4 3 3 1 34 5 2 1 12 6 16 4 7 2 18 9 1612 1 1 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 21 1

22. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6 4 9 1820 8 5 4 3 1 34 5 2 1 12 2216 4 7 2 18 920 161222 2 1 2 1 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 22

23. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6 24 9 1820 8 5 4 3 34 2 1 12 2216 25 7 2 18 92024 16122225 2 11 2 11 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 23

24. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 24 9 1820 32 30 4 3 4 1 12 2216 25 7 2 18 9202432 1612222530 2 111 2 112 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 24 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6

25. Step 2: A recursive solution: An example ( Correction !!! ) Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 24 9 1820 32 30 4 3 4 1 12 2216 25 7 2 18 9202432 1612222530 2 111 2 112 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 25 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6

26. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6 9 1820 35 3 12 2216 37 2 18 9202432 35 161222253037 2 1111 2 1122 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=0l * =0 Line 1 Line 2 Chassis Enters 26 32 24 20 30 22 25

27. Step 2: A recursive solution: An example Station S 1,1 Station S 1,2 Station S 1,3 Station S 1,4 Station S 1,5 Station S 1,6 Station S 2,1 Station S 2,2 Station S 2,3 Station S 2,4 Station S 2,5 Station S 2,6 9 1820 32 35 3 12 2216 37 2 Line 1 Line 2 Chassis Enters 27 18 9202432 35+3 161222253037+2 2 1 1 1 2 2 1 1 2 2 j 2 3 4 5 6 l1[j]l1[j] l2[j]l2[j] j 1 2 3 4 5 6 f1[j]f1[j] f2[j]f2[j] f*=3 8 l * =1 24 20 30 22 25

28. Step 2: A recursive solution (cont.) f i [j] values give the values of optimal solutions to subproblems. Let l i [j] to be the line number 1 or 2, whose station j -1 is used in a fw throgh station S i, j. We also define l * to be the line whose station n is used in a fw through the entire factory. 28

29. Step 2: A recursive solution (cont.) The fw through the entire factory: 29

30. Step 3: Computing the fastest times We can write a simple recursive algorithm based on the equations for f 1 [j] and f 2 [j] to compute the fw through the factory. However its running time is exponential. Let r i ( j ) be the number of references made to f i [ j ] in a recursive algorithm: 30

31. Step 3: Computing the fastest times(cont.) We observe that for each value of f i [j] depends only on the values of f 1 [j-1] and f 2 [j-1]. By computing the f i [j] values in order of increasing station numbers we can compute the fw through the factory in time. 31

32. The fastest way procedure 32

33. The fastest way procedure (cont.) line 1, station 6 line 2, station 5 line 2, station 4 line 1, station 3 line 2, station 2 line 1, station 1 33

34. Matrix-Chain multiplication We are given a sequence And we wish to compute 34

35. Matrix-Chain multiplication (cont.) Matrix multiplication is assosiative, and so all parenthesizations yield the same product. For example, if the chain of matrices is then the product can be fully paranthesized in five distinct way: 35

36. Matrix-Chain multiplication MATRIX-MULTIPLY (A,B) if columns [A] ≠ rows [B] then error “incompatible dimensions” else for i←1 to rows [A] do for j←1 to columns [B] do C[i, j]←0 for k←1 to columns [A] do C[ i, j ]← C[ i, j] +A[ i, k]*B[ k, j] return C 36

37. Matrix-Chain multiplication (cont.) Cost of the matrix multiplication: An example: 37

38. Matrix-Chain multiplication (cont.) 38

39. Matrix-Chain multiplication (cont.) The problem: Given a chain of n matrices, where matrix A i has dimension p i-1 x p i, fully paranthesize the product in a way that minimizes the number of scalar multiplications. 39

40. Matrix-Chain multiplication (cont.) Counting the number of alternative paranthesization : b n 40

41. Matrix-Chain multiplication (cont.) Based on the above defination we have 41

42. Matrix-Chain multiplication (cont.) Substituting the initial value for b we obtain 42

43. Matrix-Chain multiplication (cont.) Let B(x) be the generating function 43

44. Matrix-Chain multiplication (cont.) Substituting in B(x) we can write that 44 B(x)B(x)

45. Matrix-Chain multiplication (cont.) Summing over the both sides 45

46. Matrix-Chain multiplication (cont.) One way to express B(x) in closed form is Expanding into a Taylor series about x=0 we get Comparing the coefficients of x we finally show that 46

47. Matrix-Chain multiplication (cont.) Step 1: The structure of an optimal paranthesization(op) Find the optimal substructure and then use it to construct an optimal solution to the problem from optimal solutions to subproblems. Let A i...j where i ≤ j, denote the matrix product A i A i+1... A j Any parenthesization of A i A i+1... A j must split the product between A k and A k+1 for i ≤ k < j. 47

48. Matrix-Chain multiplication (cont.) The optimal substructure of the problem: Suppose that an op of A i A i+1... A j splits the product between A k and A k+1 then the paranthesization of the subchain A i A i+1... A k within this parantesization of A i A i+1... A j must be an op of A i A i+1... A k 48

49. Matrix-Chain multiplication (cont.) Step 2: A recursive solution: Let m[i,j] be the minimum number of scalar multiplications needed to compute the matrix A i...j where 1≤ i ≤ j ≤ n. Thus, the cost of a cheapest way to compute A 1...n would be m[1,n]. Assume that the op splits the product A i...j between A k and A k+1. where i ≤ k <j. Then m[i,j] = The minimum cost for computing A i...k and A k+1...j + the cost of multiplying these two matrices. 49

50. Matrix-Chain multiplication (cont.) Recursive defination for the minimum cost of paranthesization: 50

51. Matrix-Chain multiplication (cont.) To help us keep track of how to constrct an optimal solution we define s[ i,j] to be a value of k at which we can split the product A i...j to obtain an optimal paranthesization. That is s[ i,j] equals a value k such that 51

52. Matrix-Chain multiplication (cont.) Step 3: Computing the optimal costs It is easy to write a recursive algorithm based on recurrence for computing m[i,j]. But the running time will be exponential!... 52

53. Matrix-Chain multiplication (cont.) Step 3: Computing the optimal costs We compute the optimal cost by using a tabular, bottom-up approach. 53

54. Matrix-Chain multiplication ( Contd. ) MATRIX-CHAIN-ORDER(p) n←length[p]-1 for i←1 to n do m[i,i]←0 for l←2 to n do for i←1 to n-l+1 do j←i+l-1 m[i,j]← ∞ for k←i to j-1 do q←m[i,k] + m[k+1,j]+p i-1 p k p j if q < m[i,j] then m[i,j] ←q s[i,j] ←k return m and s 54

55. Matrix-Chain multiplication (cont.) An example : matrix dimension A 1 30 x 35 A 2 35 x 15 A 3 15 x 5 A 4 5 x 10 A 5 10 x 20 A 6 20 x 25 55

56. Matrix-Chain multiplication (cont.) 15750 0 9375 7875 15125 11875 4375 2625 10500 7125 0 0 0 575 2500 0 5000 1000 3500 0 750 3 3 3 3 3 3 3 3 1 5 2 1 3 4 5 4 5 3 6 2 3 4 5 6 1 2 5 4 3 2 3 5 4 6 2 1 1 i i j j A1A1 A2A2 A3A3 A4A4 A5A5 A6A6 m s 1000 2500 56

57. Matrix-Chain multiplication (cont.) Step 4: Constructing an optimal solution An optimal solution can be constructed from the computed information stored in the table s[1...n, 1...n]. We know that the final matrix multiplication is The earlier matrix multiplication can be computed recursively. 57

58. Matrix-Chain multiplication ( Contd. ) PRINT-OPTIMAL-PARENS (s, i, j) 1 if i=j 2 then print “A i ” 3 else print “ ( “ 4 PRINT-OPTIMAL-PARENS (s, i, s[i,j]) 5 PRINT-OPTIMAL-PARENS (s, s[i,j]+1, j) 6 Print “ ) ” 58

59. Matrix-Chain multiplication ( Contd. ) RUNNING TIME: Recursive solution takes exponential time. Matrix-chain order yields a running time of O(n 3 ) 59

60. Elements of dynamic programming When should we apply the method of Dynamic Programming? Two key ingredients: - Optimal substructure - Overlapping subproblems 60

61. Elements of dynamic programming (cont.) Optimal substructure (os): A problem exhibits os if an optimal solution to the problem contains within it optimal solutions to subproblems. Whenever a problem exhibits os, it is a good clue that dynamic programming might apply. In dynamic programming, we build an optimal solution to the problem from optimal solutions to subproblems. Dynamic programming uses optimal substructure in a bottom-up fashion. 61

62. Elements of dynamic programming (cont.) Overlapping subproblems: When a recursive algorithm revisits the same problem over and over again, we say that the optimization problem has overlapping subproblems. In contrast, a divide-and-conquer approach is suitable usually generates brand new problems at each step of recursion. Dynamic programming algorithms take advantage of overlapping subproblems by solving each subproblem once and then storing the solution in a table where it can be looked up when needed. 62

63. Elements of dynamic programming (cont.) Overlapping subproblems: (cont.) 1..4 1..13..42..41..2 2..2 1..3 1..1 4..4 3..4 3..3 2..3 4..4 2..2 1..12..2 4..4 3..3 2..31..23..3 1..12..23..3 The recursion tree of RECURSIVE-MATRIX-CHAIN ( p, 1, 4). The computations performed in a shaded subtree are replaced by a single table lookup in MEMOIZED-MATRIX-CHAIN ( p, 1, 4). 63

64. Matrix-Chain multiplication ( Contd. ) RECURSIVE-MATRIX-CHAIN (p, i, j) 1 if i = j 2 then return 0 3 m[i,j] ←∞ 4 for k←i to j-1 5 do q←RECURSIVE-MATRIX-CHAIN (p, i, k) + RECURSIVE-MATRIX-CHAIN (p, k+1, j)+ p i-1 p k p j 6 if q < m[i,j] 7 then m[i,j] ←q 8 return m[i,j] 64

65. Elements of dynamic programming (cont.) Overlapping subproblems: (cont.) Time to compute m[ i,j] by RECURSIVE-MATRIX-CHAIN: We assume that the execution of lines 1-2 and 6-7 take at least unit time. 65

66. Elements of dynamic programming (cont.) Overlapping subproblems: (cont.) WE guess that Using the substitution method with 66

67. Elements of dynamic programming (cont.) Memoization There is a variation of dynamic programming that often offers the efficiency of the usual dynamic-programming approach while maintaining a top-down strategy. The idea is to memoize the the natural, but inefficient, recursive algorithm. We maintain a table with subproblem solutions, but the control structure for filling in the table is more like the recursive algorithm. 67

68. Elements of dynamic programming (cont.) Memoization (cont.) An entry in a table for the solution to each subproblem is maintained. Eech table entry initially contains a special value to indicate that the entry has yet to be filled. When the subproblem is first encountered during the execution of the recursive algorithm, its solution is computed and then stored in the table. Each subsequent time that the problem is encountered, the value stored in the table is simply looked up and returned. 68

69. Elements of dynamic programming (cont.) 1 MEMOIZED-MATRIX-CHAIN(p) 2 n←length[p]-1 3 for i←1 to n 4 do for j←i to n do m[i,j] ←∞ return LOOKUP-CHAIN(p,1,n) 69

70. Elements of dynamic programming (cont.) Memoization (cont.) LOOKUP-CHAIN(p,1,n) 1 if m[i,j] < ∞ 2 then return m[i,j] 3 if i=j 4 then m[i,j] ←0 5 else for k←1 to j-1 6 do q← LOOKUP-CHAIN(p,i,k) + LOOKUP-CHAIN(p,k+1,j) + p i-1 p k p j 7 if q < m[i,j] 8 then m[i,j] ←q 9 return m[i,j] 70

71. Longest Common Subsequence 71

72. Elements of dynamic programming (cont.) 72

73. Elements of dynamic programming (cont.) 73

74. Elements of dynamic programming (cont.) 74

75. Elements of dynamic programming (cont.) 75

76. Elements of dynamic programming (cont.) 76

77. Elements of dynamic programming (cont.) 77

78. Elements of dynamic programming (cont.) 78

79. Elements of dynamic programming (cont.) 79

80. Elements of dynamic programming (cont.) 80

81. Elements of dynamic programming (cont.) 81

82. Elements of dynamic programming (cont.) 82