Dynamic Programming1 Modified by: Daniel Gomez-Prado, University of Massachusetts Amherst

Dynamic Programming2 Outline and Reading Fuzzy Explanation of Dynamic Programming Introductory example: Matrix Chain-Product (§5.3.1) The General Technique (§5.3.2) A very good example: 0-1 Knapsack Problem (§5.3.3)

Dynamic Programming3 Matrix Chain-Products Dynamic Programming is a general algorithm design paradigm. Rather than give the general structure, let us first give a motivating example: Matrix Chain-Products Review: Matrix Multiplication. C = A*B A is d × e and B is e × f O(def ) time AC B dd f e f e i j i,j

Dynamic Programming4 Matrix Chain-Products Matrix Chain-Product: Compute A=A 0 *A 1 *…*A n-1 A i is d i × d i+1 Problem: How to parenthesize? Example B is 3 × 100 C is 100 × 5 D is 5 × 5 (B*C)*D takes = 1575 ops B*(C*D) takes = 4000 ops

Dynamic Programming5 An Enumeration Approach Matrix Chain-Product Alg.: Try all possible ways to parenthesize A=A 0 *A 1 *…*A n-1 Calculate number of ops for each one Pick the one that is best Running time: The number of paranethesizations is equal to the number of binary trees with n nodes This is exponential! It is called the Catalan number, and it is almost 4 n. This is a terrible algorithm!

Dynamic Programming6 A Greedy Approach Idea #1: repeatedly select the product that uses (up) the most operations. Counter-example: A is 10 × 5 B is 5 × 10 C is 10 × 5 D is 5 × 10 Greedy idea #1 gives (A*B)*(C*D), which takes = 2000 ops But A*((B*C)*D) takes = 1000 ops

Dynamic Programming7 Another Greedy Approach Idea #2: repeatedly select the product that uses the fewest operations. Counter-example: A is 101 × 11 B is 11 × 9 C is 9 × 100 D is 100 × 99 Greedy idea #2 gives A*((B*C)*D)), which takes = ops (A*B)*(C*D) takes = ops The greedy approach is not giving us the optimal value.

Dynamic Programming8 A “Recursive” Approach Define subproblems: Find the best parenthesization of A i *A i+1 *…*A j. Let N i,j denote the number of operations done by this subproblem. The optimal solution for the whole problem is N 0,n-1. Subproblem optimality: The optimal solution can be defined in terms of optimal subproblems There has to be a final multiplication (root of the expression tree) for the optimal solution. Say, the final multiply is at index i: (A 0 *…*A i )*(A i+1 *…*A n-1 ). Then the optimal solution N 0,n-1 is the sum of two optimal subproblems, N 0,i and N i+1,n-1 plus the time for the last multiply. If the global optimum did not have these optimal subproblems, we could define an even better “optimal” solution.

Dynamic Programming9 A Characterizing Equation The global optimal has to be defined in terms of optimal subproblems, depending on where the final multiply is at. Let us consider all possible places for that final multiply: Recall that A i is a d i × d i+1 dimensional matrix. So, a characterizing equation for N i,j is the following: Note that subproblems are not independent--the subproblems overlap.

Dynamic Programming10 The characterization equation means: Divide and ConquerDynamic Programming≠ Recursive problemOverlapped subproblems

Dynamic Programming11 A Dynamic Programming Algorithm Since subproblems overlap, we don’t use recursion. Instead, we construct optimal subproblems “bottom-up.” N i,i ’s are easy, so start with them Then do length 2,3,… subproblems, and so on. Running time: O(n 3 ) Algorithm matrixChain(S): Input: sequence S of n matrices to be multiplied Output: number of operations in an optimal paranethization of S for i  1 to n-1 do N i,i  0 for b  1 to n-1 do for i  0 to n-b-1 do j  i+b N i,j  +infinity for k  i to j-1 do N i,j  min{N i,j, N i,k +N k+1,j +d i d k+1 d j+1 }

Dynamic Programming12 answer N … n-1 … j i A Dynamic Programming Algorithm Visualization The bottom-up construction fills in the N array by diagonals N i,j gets values from pervious entries in i-th row and j-th column Filling in each entry in the N table takes O(n) time. Total run time: O(n 3 ) Getting actual parenthesization can be done by remembering “k” for each N entry

Dynamic Programming13 Matrix Chain algorithm Algorithm matrixChain(S): Input:sequence S of n matrices to be multiplied Output:# of multiplications in optimal parenthesization of S for i  0 to n-1 do N i,i  0 for b  1 to n-1 do // b is # of ops in S for i  0 to n-b-1 do j  i+b N i,j  +infinity for k  i to j-1 do sum = N i,k +N k+1,j +d i d k+1 d j+1 if (sum < N i,j ) then N i,j  sum O i,j  k return N 0,n-1 Example: ABCD A is 10 × 5 B is 5 × 10 C is 10 × 5 D is 5 × 10 N A B C D AB BC CD A(BC) (BC)D (A(BC))D

Dynamic Programming14 Recovering operations Example: ABCD A is 10 × 5 B is 5 × 10 C is 10 × 5 D is 5 × 10 N A B C D AB BC CD A(BC) (BC)D (A(BC))D // return expression for multiplying // matrix chain A i through A j exp(i,j) if (i=j) then// base case, 1 matrix return ‘A i ’ else k = O[i,j]// see red values on left S1 = exp(i,k)// 2 recursive calls S2 = exp(k+1,j) return ‘(‘ S1 S2 ‘)’

Dynamic Programming15 The General Dynamic Programming Technique Applies to a problem that at first seems to require a lot of time (possibly exponential), provided we have: Simple subproblems: the subproblems can be defined in terms of a few variables, such as j, k, l, m, and so on. Subproblem optimality: the global optimum value can be defined in terms of optimal subproblems Subproblem overlap: the subproblems are not independent, but instead they overlap (hence, should be constructed bottom-up).

Dynamic Programming16 Knapsack problem Given some items, pack the knapsack to get the maximum total value. Each item has some weight and some benefit. Total weight that we can carry is no more than some fixed number W. So we must consider weights of items as well as their value. Item # Weight benefit Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming17 Knapsack problem There are two versions of the problem: “0-1 knapsack problem” Items are indivisible; you either take an item or not. Solved with dynamic programming. “Fractional knapsack problem” Items are divisible: you can take any fraction of an item. Solved with a greedy algorithm. Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming18 The 0/1 Knapsack Problem Given: A set S of n items, with each item i having b i - a positive benefit w i - a positive weight Goal: Choose items with maximum total benefit but with weight at most W. If we are not allowed to take fractional amounts, then this is the 0/1 knapsack problem. In this case, we let T denote the set of items we take Objective: maximize Constraint:

Dynamic Programming Knapsack problem: a picture W = 20 wiwi bibi WeightBenefit This is a knapsack Max weight: W = 20 Items Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming Knapsack problem: brute-force approach Let’s first solve this problem with a straightforward algorithm Since there are n items, there are 2 n possible combinations of items. We go through all combinations and find the one with the most total value and with total weight less or equal to W Running time will be O(2 n ) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming Knapsack problem: brute-force approach Can we do better? Yes, with an algorithm based on dynamic programming We need to carefully identify the subproblems Let’s try this: If items are labeled 1..n, then a subproblem would be to find an optimal solution for S k = {items labeled 1, 2,.. k} Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming22 Defining a Subproblem If items are labeled 1..n, then a subproblem would be to find an optimal solution for S k = {items labeled 1, 2,.. k} This is a valid subproblem definition. The question is: can we describe the final solution (S n ) in terms of subproblems (S k )? Unfortunately, we can’t do that. Explanation follows…. Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming23 Defining a Subproblem Max weight: W = 20 For S 4 : Total weight: 14; Total benefit: 20 w 1 =2 b 1 =3 w 2 =4 b 2 =5 w 3 =5 b 3 =8 w 4 =3 b 4 =4 wiwi bibi WeightBenefit 9 Item # S4S4 S5S5 w 1 =2 b 1 =3 w 2 =4 b 2 =5 w 3 =5 b 3 =8 w 4 =9 b 4 =10 For S 5 : Total weight: 20 Total benefit: 26 Solution for S 4 is not part of the solution for S 5 !!! Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming24 Defining a Subproblem (continued) As we have seen, the solution for S 4 is not part of the solution for S 5 So our definition of a subproblem is flawed and we need another one! Let’s add another parameter: w, which will represent the exact weight for each subset of items The subproblem then will be to compute B[k,w] Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming25 Recursive Formula for subproblems It means, that the best subset of S k that has total weight w is one of the two: the best subset of S k-1 that has total weight w, or the best subset of S k-1 that has total weight w-w k plus the item k Recursive formula for subproblems: Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming26 Recursive Formula The best subset of S k that has the total weight w, either contains item k or not. First case: w k >w. Item k can’t be part of the solution, since if it was, the total weight would be > w, which is unacceptable Second case: w k <=w. Then the item k can be in the solution, and we choose the case with greater value Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming Knapsack Algorithm for w = 0 to W B[0,w] = 0 for i = 0 to n B[i,0] = 0 for w = 0 to W if w i <= w // item “i” can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w Modified from: David Luebke, University of Virginia Charlottesville What is the running time of this algorithm? O(W) Repeat n times O(n*W) Remember that the brute-force algorithm takes O(2 n ) i w B[i,w]B[i-1,w] B[i-1,w-w i ]

Dynamic Programming28 Example Let’s run our algorithm on the following data: n = 4 (# of elements) W = 5 (max weight) Elements (weight, benefit): (2,3), (3,4), (4,5), (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming29 Example (continue) for w = 0 to W B[0,w] = W i Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming30 Example (continue) for i = 0 to n B[i,0] = W i Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming31 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=1 b i =3 w i =2 w=1 w-w i =-1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) 4 0 Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming32 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=1 b i =3 w i =2 w=2 w-w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming33 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=1 b i =3 w i =2 w=3 w-w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming34 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=1 b i =3 w i =2 w=4 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming35 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=1 b i =3 w i =2 w=5 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming36 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=2 b i =4 w i =3 w=1 w-w i =-2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming37 Example (continue) if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=2 b i =4 w i =3 w=2 w-w i =-1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming38 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=2 b i =4 w i =3 w=3 w-w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming39 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=2 b i =4 w i =3 w=4 w-w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming40 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=2 b i =4 w i =3 w=5 w-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming41 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=3 b i =5 w i =4 w=1..3 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming42 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=3 b i =5 w i =4 w=4 w- w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming43 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=3 b i =5 w i =4 w=5 w- w i =1 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming44 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=3 b i =5 w i =4 w=1..4 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming45 if w i <= w // item i can be part of the solution if b i + B[i-1,w-w i ] > B[i-1,w] B[i,w] = b i + B[i-1,w- w i ] else B[i,w] = B[i-1,w] else B[i,w] = B[i-1,w] // w i > w W i i=3 b i =5 w i =4 w=5 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Example (continue)

Dynamic Programming46 Comments This algorithm only finds the max possible value that can be carried in the knapsack To know the items that make this maximum value, an addition to this algorithm is necessary Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming47 How to find actual knapsack items All of the information we need is in the table. B[n,W] is the maximal value of items that can be placed in the Knapsack. Let i=n and k=W if B[i,k] ≠ B[i -1,k] then mark the i th item as in the knapsack i = i -1, k = k-w i else i = i -1 // Assume the i th item is not in the knapsack // Could it be in the optimally packed knapsack? Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming48 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=4 k=5 b i =5 w i =6 B[i,k]=7 B[i-1,k]=7 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue)

Dynamic Programming49 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=4 k=5 b i =5 w i =6 B[i,k]=7 B[i-1,k]=7 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue)

Dynamic Programming50 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=3 k=5 b i =5 w i =4 B[i,k]=7 B[i-1,k]=7 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue)

Dynamic Programming51 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=2 k=5 b i =4 w i =3 B[i,k]=7 B[i-1,k]=3 k-w i =2 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue)

Dynamic Programming52 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=1 k=2 b i =3 w i =2 B[i,k]=3 B[i-1,k]=0 k-w i =0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue)

Dynamic Programming53 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=0 k=0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue) The optimal knapsack should contain items: { 1, 2 }

Dynamic Programming54 i = n, k = w while i, k > 0 if B[i,k] ≠ B[i-1,k] mark the ith item as the knapsack i = i-1, k = k-w i else i = i W i i=0 k=0 Items: 1: (2,3) 2: (3,4) 3: (4,5) 4: (5,6) Modified from: David Luebke, University of Virginia Charlottesville Finding Items (continue) The optimal knapsack should contain items: { 1, 2 }

Dynamic Programming55 Conclusions Dynamic programming is a useful technique for solving certain kind of problems When the solution can be recursively described in terms of partial solutions, we can store these partial solutions and re-use them as necessary Running time of dynamic programming algorithm vs. naïve algorithm: » 0-1 Knapsack problem: O(W*n) vs. O(2 n ) Modified from: David Luebke, University of Virginia Charlottesville

Dynamic Programming56 Remember Hw #2 is already posted. It is due on the day of the Exam. Midterm Exam is scheduled for Monday, March 28 at 5:30 pm or later. There is a mandatory seminar for CSE student on Monday, March 28, at 4:00, hence we cannot start the exam earlier than 5:30 pm.