Dynamic Programming Dr. Yingwu Zhu Chapter 15

Dynamic Programming (DP) It is used, when the solution can be recursively described in terms of solutions to subproblems (optimal substructure) Algorithm finds solutions to subproblems and stores them in memory for later use More efficient than “brute-force methods”, which solve the same subproblems over and over again

DP An algorithm design technique for optimization problems: often minimizing or maximizing. Like divide-and-conquer, DP solves problems by combining solutions to subproblems. Unlike divide-and-conquer, subproblems are not independent. Subproblems may share subsubproblems, However, solution to one subproblem may not affect the solutions to other subproblems of the same problem. (More on this later.) DP reduces computation by Solving subproblems in a bottom-up fashion. Storing solution to a subproblem the first time it is solved. Looking up the solution when subproblem is encountered again. Key: determine structure of optimal solutions

Steps in DP Characterize structure of an optimal solution. Define value of optimal solution recursively. Compute optimal solution values either top-down with caching or bottom-up in a table. Construct an optimal solution from computed values. We’ll study these with the help of examples.

Computing a binomial coefficient by DP Binomial coefficients are coefficients of the binomial formula: (a + b)n = C(n,0)anb0 + . . . + C(n,k)an-kbk + . . . + C(n,n)a0bn Recurrence: C(n,k) = C(n-1,k) + C(n-1,k-1) for n > k > 0 C(n,0) = 1, C(n,n) = 1 for n  0

Computing a binomial coefficient by DP Binomial coefficients are coefficients of the binomial formula: (a + b)n = C(n,0)anb0 + . . . + C(n,k)an-kbk + . . . + C(n,n)a0bn Recurrence: C(n,k) = C(n-1,k) + C(n-1,k-1) for n > k > 0 C(n,0) = 1, C(n,n) = 1 for n  0 Value of C(n,k) can be computed by filling a table: 0 1 2 . . . k-1 k 0 1 1 1 1 . n-1 C(n-1,k-1) C(n-1,k) n C(n,k)

Computing C(n,k): pseudocode and analysis Time efficiency: Θ(nk) Space efficiency: Θ(nk)

More Space Efficient We can achieve Θ(k)

Properties of a problem that can be solved with DP Simple Subproblems We should be able to break the original problem to smaller subproblems that have the same structure Optimal Substructure of the problems The solution to the problem must be a composition of subproblem solutions Subproblem Overlap Optimal solutions to unrelated problems can contain subproblems in common

0-1 Knapsack problem Given a knapsack with maximum capacity W, and a set S consisting of n items Each item i has some weight wi and benefit value bi (all wi , bi and W are integer values) Problem: How to pack the knapsack to achieve maximum total value of packed items?

0-1 Knapsack problem: a picture Weight Benefit value wi bi Items 3 2 This is a knapsack Max weight: W = 20 4 3 5 4 W = 20 8 5 9 10

0-1 Knapsack problem Problem, in other words, is to find The problem is called a “0-1” problem, because each item must be entirely accepted or rejected. Just another version of this problem is the “Fractional Knapsack Problem”, where we can take fractions of items. 12/7/2018

0-1 Knapsack problem: brute-force approach Let’s first solve this problem with a straightforward algorithm Since there are n items, there are 2n possible combinations of items. We go through all combinations and find the one with the most total value and with total weight less or equal to W Running time will be O(2n) 12/7/2018

0-1 Knapsack problem: brute-force approach Can we do better? Yes, with an algorithm based on dynamic programming We need to carefully identify the subproblems Let’s try this: If items are labeled 1..n, then a subproblem would be to find an optimal solution for Sk = {items labeled 1, 2, .. k} 12/7/2018

Defining a Subproblem If items are labeled 1..n, then a subproblem would be to find an optimal solution for Sk = {items labeled 1, 2, .. k} This is a valid subproblem definition. The question is: can we describe the final solution (Sn ) in terms of subproblems (Sk)? Unfortunately, we can’t do that. Explanation follows…. 12/7/2018

? Defining a Subproblem wi bi Weight Benefit wi bi Item # ? 1 2 3 Max weight: W = 20 For S4: Total weight: 14; total benefit: 20 S4 2 3 4 S5 3 4 5 4 5 8 5 9 10 w1 =2 b1 =3 w2 =4 b2 =5 w3 =5 b3 =8 w5 =9 b5 =10 Solution for S4 is not part of the solution for S5!!! For S5: Total weight: 20 total benefit: 26 12/7/2018

Defining a Subproblem (continued) As we have seen, the solution for S4 is not part of the solution for S5 So our definition of a subproblem is flawed and we need another one! Let’s add another parameter: w, which will represent the exact weight for each subset of items The subproblem then will be to compute B[k,w] 12/7/2018

Recursive Formula for subproblems It means, that the best subset of Sk that has total weight w is one of the two: 1) the best subset of Sk-1 that has total weight w, or 2) the best subset of Sk-1 that has total weight w-wk plus the item k 12/7/2018

Recursive Formula The best subset of Sk that has the total weight w, either contains item k or not. First case: wk>w. Item k can’t be part of the solution, since if it was, the total weight would be > w, which is unacceptable Second case: wk <=w. Then the item k can be in the solution, and we choose the case with greater value 12/7/2018

0-1 Knapsack Algorithm for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Remember that the brute-force algorithm Running time for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 < the rest of the code > O(W) Repeat n times O(W) What is the running time of this algorithm? O(n*W) Remember that the brute-force algorithm takes O(2n) 12/7/2018

Example Let’s run our algorithm on the following data: n = 4 (# of elements) W = 5 (max weight) Elements (weight, benefit): (2,3), (3,4), (4,5), (5,6) 12/7/2018

Example (2) i 1 2 3 4 W 1 2 3 4 5 for w = 0 to W B[0,w] = 0 12/7/2018

Example (3) i 1 2 3 4 W 1 2 3 4 5 for i = 0 to n B[i,0] = 0 12/7/2018

Example (4) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=1 bi=3 wi=2 1 2 3 4 W i=1 bi=3 wi=2 w=1 w-wi =-1 1 2 3 4 5 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (5) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=1 bi=3 wi=2 1 2 3 4 W i=1 bi=3 wi=2 w=2 w-wi =0 1 2 3 3 4 5 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (6) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=1 bi=3 wi=2 1 2 3 4 W i=1 bi=3 wi=2 w=3 w-wi=1 1 2 3 3 3 4 5 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (7) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=1 bi=3 wi=2 1 2 3 4 W i=1 bi=3 wi=2 w=4 w-wi=2 1 2 3 3 3 4 3 5 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (8) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=1 bi=3 wi=2 1 2 3 4 W i=1 bi=3 wi=2 w=5 w-wi=2 1 2 3 3 3 4 3 5 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (9) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=2 bi=4 wi=3 1 2 3 4 W i=2 bi=4 wi=3 w=1 w-wi=-2 1 2 3 3 3 4 3 5 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (10) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=2 bi=4 wi=3 1 2 3 4 W i=2 bi=4 wi=3 w=2 w-wi=-1 1 2 3 3 3 3 4 3 5 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (11) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=2 bi=4 wi=3 1 2 3 4 W i=2 bi=4 wi=3 w=3 w-wi=0 1 2 3 3 3 3 4 4 3 5 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (12) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=2 bi=4 wi=3 1 2 3 4 W i=2 bi=4 wi=3 w=4 w-wi=1 1 2 3 3 3 3 4 4 3 4 5 3 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (13) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=2 bi=4 wi=3 1 2 3 4 W i=2 bi=4 wi=3 w=5 w-wi=2 1 2 3 3 3 3 4 4 3 4 5 3 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (14) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=3 bi=5 wi=4 1 2 3 4 W i=3 bi=5 wi=4 w=1..3 1 2 3 3 3 3 3 4 4 4 3 4 5 3 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (15) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=3 bi=5 wi=4 1 2 3 4 W i=3 bi=5 wi=4 w=4 w- wi=0 1 2 3 3 3 3 3 4 4 4 3 4 5 5 3 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (15) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=3 bi=5 wi=4 1 2 3 4 W i=3 bi=5 wi=4 w=5 w- wi=1 1 2 3 3 3 3 3 4 4 4 3 4 5 5 3 7 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (16) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=3 bi=5 wi=4 1 2 3 4 W i=3 bi=5 wi=4 w=1..4 1 2 3 3 3 3 3 3 4 4 4 4 3 4 5 5 5 3 7 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Example (17) Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) i=3 bi=5 wi=4 1 2 3 4 W i=3 bi=5 wi=4 w=5 1 2 3 3 3 3 3 3 4 4 4 4 3 4 5 5 5 3 7 7 7 if wi <= w // item i can be part of the solution if bi + B[i-1,w-wi] > B[i-1,w] B[i,w] = bi + B[i-1,w- wi] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // wi > w 12/7/2018

Knapsack Problem and Memory Functions Bottom-up is classic solutions to DP problems Idea: keep solutions to subproblems in the table, fill the table entries Drawback: some unnecessary entries are filled, inefficient! Memory functions are used to avoid the drawbacks How? Lookup table + top-down solution Memoization

Knapsack Problem and Memory Functions ALGORITHM MFKnapsack(i, j) //W[1..n], V[1..n] //T[0..n, 0..w] is the lookup table, initialized with -1 for each entry except for //row 0 and column 0 initialized with 0s If T[i, j] < 0 if j < W[i] value  MFKnapsack(i-1, j) else value  max (MFKnapsack(i-1,j), MFKnapsack(i-1,j-W[i]) + V[i]) T[i,j]  value return T[i,j]

Comments This algorithm only finds the max possible value that can be carried in the knapsack To know the items that make this maximum value, an addition to this algorithm is necessary We can define another subproblem to solve the problem B(w): the maximum value for a knapsack of capacity w given the n items What’s the optimal substructure? 12/7/2018

Longest Common Subsequence (LCS) Application: comparison of two DNA strings Ex: X= {A B C B D A B }, Y= {B D C A B A} Longest Common Subsequence: X = A B C B D A B Y = B D C A B A Brute force algorithm would compare each subsequence of X with the symbols in Y 12/7/2018

LCS Problem: Given 2 sequences, X = x1,...,xm and Y = y1,...,yn, find a common subsequence whose length is maximum. springtime ncaa tournament basketball printing north carolina snoeyink Subsequence need not be consecutive, but must be in order.

LCS Algorithm if |X| = m, |Y| = n, then there are 2m subsequences of x; we must compare each with Y (n comparisons) So the running time of the brute-force algorithm is O(n 2m) Notice that the LCS problem has optimal substructure: solutions of subproblems are parts of the final solution. Subproblems: “find LCS of pairs of prefixes of X and Y” 12/7/2018

LCS Algorithm First we’ll find the length of LCS. Later we’ll modify the algorithm to find LCS itself. Define Xi, Yj to be the prefixes of X and Y of length i and j respectively Define c[i,j] to be the length of LCS of Xi and Yj Then the length of LCS of X and Y will be c[m,n] 12/7/2018

LCS recursive solution We start with i = j = 0 (empty substrings of x and y) Since X0 and Y0 are empty strings, their LCS is always empty (i.e. c[0,0] = 0) LCS of empty string and any other string is empty, so for every i and j: c[0, j] = c[i,0] = 0 12/7/2018

LCS recursive solution When we calculate c[i,j], we consider two cases: First case: x[i]=y[j]: one more symbol in strings X and Y matches, so the length of LCS Xi and Yj equals to the length of LCS of smaller strings Xi-1 and Yi-1 , plus 1 12/7/2018

LCS recursive solution Second case: x[i] != y[j] As symbols don’t match, our solution is not improved, and the length of LCS(Xi , Yj) is the same as before (i.e. maximum of LCS(Xi, Yj-1) and LCS(Xi-1,Yj) Why not just take the length of LCS(Xi-1, Yj-1) ? 12/7/2018

LCS Length Algorithm LCS-Length(X, Y) 1. m = length(X) // get the # of symbols in X 2. n = length(Y) // get the # of symbols in Y 3. for i = 1 to m c[i,0] = 0 // special case: Y0 4. for j = 1 to n c[0,j] = 0 // special case: X0 5. for i = 1 to m // for all Xi 6. for j = 1 to n // for all Yj 7. if ( Xi == Yj ) 8. c[i,j] = c[i-1,j-1] + 1 9. else c[i,j] = max( c[i-1,j], c[i,j-1] ) 10. return c 12/7/2018

LCS Example We’ll see how LCS algorithm works on the following example: X = ABCB Y = BDCAB What is the Longest Common Subsequence of X and Y? LCS(X, Y) = BCB X = A B C B Y = B D C A B 12/7/2018

LCS Example (0) ABCB BDCAB X = ABCB; m = |X| = 4 j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 B 2 3 C 4 B X = ABCB; m = |X| = 4 Y = BDCAB; n = |Y| = 5 Allocate array c[5,4] 12/7/2018

LCS Example (1) ABCB BDCAB for i = 1 to m c[i,0] = 0 j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 B 2 3 C 4 B for i = 1 to m c[i,0] = 0 for j = 1 to n c[0,j] = 0 12/7/2018

LCS Example (2) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 B 2 3 C A 1 B 2 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (3) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 B 2 3 C A 1 B 2 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (4) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 B 2 3 A 1 1 B 2 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (5) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 A 1 1 1 B 2 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (6) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 A 1 1 1 B 2 1 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (7) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 A 1 1 1 B 2 1 1 1 1 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (8) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 A 1 1 1 B 2 1 1 1 1 2 3 C 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (10) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (11) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (12) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 4 B if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (13) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 4 B 1 if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (14) ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 4 B 1 1 2 2 if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Example (15) 3 ABCB BDCAB j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 3 4 B 1 1 2 2 if ( Xi == Yj ) c[i,j] = c[i-1,j-1] + 1 else c[i,j] = max( c[i-1,j], c[i,j-1] ) 12/7/2018

LCS Algorithm Running Time LCS algorithm calculates the values of each entry of the array c[m,n] So what is the running time? O(m*n) since each c[i,j] is calculated in constant time, and there are m*n elements in the array 12/7/2018

How to find actual LCS So far, we have just found the length of LCS, but not LCS itself. We want to modify this algorithm to make it output Longest Common Subsequence of X and Y Each c[i,j] depends on c[i-1,j] and c[i,j-1] or c[i-1, j-1] For each c[i,j] we can say how it was acquired: For example, here c[i,j] = c[i-1,j-1] +1 = 2+1=3 2 2 2 3 12/7/2018

How to find actual LCS - continued Remember that So we can start from c[m,n] and go backwards Whenever c[i,j] = c[i-1, j-1]+1, remember x[i] (because x[i] is a part of LCS) When i=0 or j=0 (i.e. we reached the beginning), output remembered letters in reverse order 12/7/2018

Finding LCS 3 j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 1 1 1 1 2 3 A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 3 4 B 1 1 2 2 12/7/2018

Finding LCS (2) 3 B C B LCS (reversed order): B C B j 0 1 2 3 4 5 i Yj B D C A B Xi A 1 1 1 B 2 1 1 1 1 2 3 C 1 1 2 2 2 3 4 B 1 1 2 2 B C B LCS (reversed order): B C B (this string turned out to be a palindrome) LCS (straight order): 12/7/2018