1. Dynamic Programming 15-211 Fundamental Data Structures and Algorithms Klaus Sutner March 30, 2004

2. Plan  Quiz tomorrow (available 1:30)  Homework 6...  Today: Dynamic Programming  Reading  Section 7.6

3. From Exponential Time to Linear Time

4. Fibonacci  Leonardo Pisano  aka Leonardo Fibonacci  How many rabbits can be produced from a single pair in a year’s time? (1202)  Assume New pair of offspring each month Each pair becomes fertile after one month Rabbits never die

5. Fibonacci Recurrence  The Fibonacci sequence  f(0) = 0  f(1) = 1  f(n) = f(n-1) + f(n-2)

6. Recursion int fib( int n ) { if( n<=1 ) return n; return fib(n-1) + fib(n-2); } Nearly verbatim translation. Alas …

7. Recursion int fib( int n ) { if (n<=1) return n; return fib(n-1) + fib(n-2); } What is the running time of this implementation?

8. Recomputation Principle: Avoid recomputation. Typically store already known results in a hash table. If the computation has additional structure, a plain table may be better. Example: fib(n) requires fib(i) for all i < n. Space versus time trade-off.

9. Memoizing Often called “memoizing”. Ideally one would like to write memoize int fib( int n ) {.... } Supported in some computer algebra systems, but not in classical programming languages.

10. Memoizing And for a good reason: the function must be pure. Side effects such as I/O, changes in globals, etc. would be fatal. Also, return value has to be determined exclusively by the input argument (no randomization, no globals,...).

11. Memoizing ITRW int fib( int n ) { if( n <= 1 ) return n; if( fibmem[n] == -1 ) fibmem[n] = fib(n-1) + fib(n-2); return fibmem[n]; } Requires initialization of array. Only works since fib(n) never has the value -1. Really a hack.

12. Bottom-Up int fib( int n ) { if( n <= 1 ) return n; F[0] = 0; F[1] = 1; for( i = 2; i <= n; i++ ) F[i] = F[i-1] + F[i-2]; return F[n]; } Observation: Only last two entries are needed.

13. Memoryless int fib( int n ) { if( n <= 1 ) return n; F0 = 0; F1 = 1; for( i = 2; i <= n; i++ ) (F0,F1) = (F1,F0+F1); return F1; } Observation: Don't need the array at all.

14. Top-Down vs. Bottom-Up Top-down places the burden on the storage mechanism (memoization) – it must keep track of known values. More efficient if only a few values are needed. Bottom-up requires the algorithm designer to figure out the right order in which to compute values – no hash table needed, just a k- dimensional array (usually k = 1 or k = 2). More efficient if all (most) values in array are used.

15. Exercise Determine the running time of the plain recursive Fibonacci function with/without memoizing. Determine the running time of the fast recursive Fibonacci function with/without memoizing. Assume that arithmetic is constant time. Then do the same charging quadratic time for multiplication (addition is linear and doesn’t matter much).

16. Recall: Greedy Algorithms

17. The Fractional Knapsack Problem (FKP)  You rob a store: find n kinds of items  Gold dust. Wheat. Beer.  The total inventory for the i th kind of item:  Weight: w i pounds  Value: v i dollars  Knapsack can hold a maximum of W pounds.  Q: how much of each kind of item should you take? (Can take fractional weight)

18. FKP: a greedy solution  Fill knapsack with “most valuable” item until all is taken.  Most valuable = v i /w i (dollars per pound)  Then next “most valuable” item, etc.  Repeat until knapsack is full.

19. The Binary Knapsack Problem  You win the Supermarket Shopping Spree contest.  You are given a shopping cart with capacity C.  You are allowed to fill it with any items you want from Giant Eagle.  Giant Eagle has items 1, 2, … n, which have values v 1, v 2, …, v n, and sizes s 1, s 2, …, s n.  How do you (efficiently) maximize the value of the items in your cart?

20. BKP is not greedy  The obvious greedy strategy of taking the maximum value item that still fits in the cart does not work.  Consider:  Suppose item i has size s i = C and value v i.  It can happen that there are items j and k with combined size s j +s k  C but v j +v k > v i.

21. BKP: Greedy approach fails item 1 item 2 item 3 cart $60, 10 lbs $100, 20 lbs $120, 30 lbs Maximum weight = 50 lbs $4Item 3 $5Item 2 $6Item 1 Dollars/pound BKP has optimal substructure, but not greedy-choice property: optimal solution does not contain greedy choice. $160 $180$220 (optimal)

22. A Solution: Dynamic Programming

23. Brute-Force Solution Try all possible combinations of items that fit into a cart of capacity C. Incidentally: interesting combinatorial problem: how do we enumerate all possible combinations elegantly? At any rate, since there are exponentially many potential solutions in general, any such approach is exponential.

24. Precise formulation  Let V(k, A) be the max value when choosing among items 1, 2, …, k and with a shopping cart of size A.  V(k, A) is a subproblem of the final problem of V(n, C). n is the number of items

25. Subproblem formulation, cont’d  V(k, A) = max of  V(k-1, A)  [that is, don’t take the k th item]  V(k-1, A – s k ) + v k  [or else take the k th item] Note the simple recursive structure.

26. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A

27. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0

28. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10

29. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10 0 2 2 10 … 12

30. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10 0 2 2 10 … 12 0 2 9 11 …

31. Observations  Inefficient, or “brute-force,” solutions to dynamic programming problems often have simple recursive definitions.

32. Dynamic programming  Build up to a solution.  Solve all smaller subproblems first.  Typically start with smallest subproblems and then work up to larger ones.  Hope that there is substantial overlap between subproblems.  Combine solutions to get answers to larger subproblems.

33. Dynamic programming  Underlying idea:  Use memoization so that overlap can be exploited.  As smaller subproblems are solved, solving the larger subproblems might get easier.  Can sometimes reduce seemingly exponential problems to polynomial time.

34. Using dynamic programming  Key ingredients:  Simple subproblems.  Problem can be broken into subproblems, typically with solutions that are easy to store in a table/array.  Subproblem optimization.  Optimal solution is composed of optimal subproblem solutions.  Subproblem overlap.  Optimal solutions to separate subproblems can have subproblems in common.

35. Longest Common Subsequence Problem

36. String subsequences  Consider DNA sequences.  Strings over the alphabet {A,C,G,T}.  Given a string X = x 0 x 1 x 2 …x n-1 :  a subsequence of X is any string that is of the form x i 1 x i 2 …x i k where i j <i j+1.  Example: CGATAATTGAGA AAAG A subsequence

37. LCS problem  The longest common subsequence problem:  Given two strings X and Y, find the longest string S that is a subsequence of both X and Y. CGATAATTGAGA GTTCCTAATA LCS = 6

38. Who Cares? LCS is the key ingredient for crucial tools such as diff and patch. > diff file1 file2 Find a LCS, then record only the edit difference between the remainder of the two files. Revision control systems depend on this.

39. A brute-force approach  A brute-force algorithm:  for each subsequence S of X:  determine whether S is a subsequence of Y.  if yes, then remember if it is the longest encountered so far.  Return the longest subsequence found.  Requires O(m 2 n ) time!  There are 2 n different subsequences of X.  Determining whether S is a subsequence of Y requires m time.

40. Towards Dynamic Programming Informally we have LCS(xa,ya) = LCS(x,y) a LCS(xa,yb) = LCS(x,yb) || LCS(xa,y) Read this with a pound of salt.

41. More precisely …  Simple subproblems?  (that are easy to store in a table?)  Yes!  Let L(i,j) = LCS for the first i+1 letters of X and first j+1 letters of Y.  L(-1,j) = L(i,-1) = 0. L(8,9) = 5 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X =

42. Storing solutions in a table 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 X Y 0000000000000000000000 0 0 0 0 0 0 5 A memo table

43. DP ingredients, cont’d  Subproblem optimization?  Yes! Optimal solution for L(i,j) is a combination of optimal solutions for L(k,l) for some k,l such that ki and lj.  Let X = x 0 x 1 x 2 …x n-1 and Y = y 0 y 1 y 2 …y m-1.

44. DP ingredients, cont’d  Subproblem optimization, Case 1: CGATAATTGAGA GTTCCTAATA L(8,10) = 5 0123456789 1011 0123456789 Y = X = If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Example: Suppose i=9 and j=11

45. Ingredients, cont’d  Subproblem optimization, Case 2: CGATAATTGAG GTTCCTAATA L(9,9) = 6 L(8,10) = 5 0123456789 10 0123456789 Y = X = L(i,j) = max(L(i-1,j),L(i,j-1)). Example: Suppose i=9 and j=10

46. DP ingredients, cont’d  Subproblem optimization?  Yes! Optimal solution for L(i,j) is a combination of optimal solutions for L(k,l) for some k,l such that ki and lj.  Let X = x 0 x 1 x 2 …x n-1 and Y = y 0 y 1 y 2 …y m-1.  Case 1: If x i =y j, then  L(i,j) = L(i-1,j-1) + 1.  Case 2: If x i y j, then  L(i,j) = max(L(i-1,j),L(i,j-1)).

47. DP ingredients, cont’d  Subproblem overlap?  Yes! Solutions for L(i-1,j-1) can be re- used in computing L(i,j), L(i-1,j), and L(i,j-1).

48. We have the ingredients  So, we have all of the ingredients for an application of dynamic programming.  We will start with L(0,0) and fill out a memoization table of optimal solutions to all subproblems.

49. The problem CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X =

50. Algorithm initialization X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X =

51. Computing L(0,j) X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 Case 1: If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Case 2: If x i y j, then L(i,j) = max(L(i-1,j),L(i,j-1))

52. Computing L(1,j) X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 2 2 2 2 2 2 2 2 2 Case 1: If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Case 2: If x i y j, then L(i,j) = max(L(i-1,j),L(i,j-1))

53. Computing L(2,j) X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 2 2 2 2 2 2 2 2 2 0 1 1 2 2 2 3 3 3 3 3 3 Case 1: If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Case 2: If x i y j, then L(i,j) = max(L(i-1,j),L(i,j-1))

54. Computing L(3,j) X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 2 2 2 2 2 2 2 2 2 0 1 1 2 2 2 3 3 3 3 3 3 1 1 1 2 2 2 3 3 3 3 3 3 Case 1: If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Case 2: If x i y j, then L(i,j) = max(L(i-1,j),L(i,j-1))

55. The complete table X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 2 2 2 2 2 2 2 2 2 0 1 1 2 2 2 3 3 3 3 3 3 1 1 1 2 2 2 3 3 3 3 3 3 1 1 1 2 2 2 3 4 4 4 4 4 1 1 2 2 3 3 3 4 4 5 5 5 1 1 2 2 3 4 4 4 4 5 5 6 1 1 2 3 3 4 5 5 5 5 5 6 1 1 2 3 4 4 5 5 5 6 6 6 Case 1: If x i =y j, then L(i,j) = L(i-1,j-1) + 1 Case 2: If x i y j, then L(i,j) = max(L(i-1,j),L(i,j-1))

56. Values versus Witnesses So far, we only have the length of the LCS, not the actual sequence. Have to post-process the matrix to obtain the sequence. This is a standard feature of many dynamic programming algorithms.

57. Reading the subsequence X Y 0000000000000000000000 0 1 2 3 4 5 6 7 8 9 L -1 0 1 2 3 4 5 6 7 8 9 1011 0 0 0 0 0 0 CGATAATTGAGA GTTCCTAATA 0123456789 1011 0123456789 Y = X = 0 1 1 1 1 1 1 1 1 1 1 1 0 1 1 2 2 2 2 2 2 2 2 2 0 1 1 2 2 2 3 3 3 3 3 3 1 1 1 2 2 2 3 3 3 3 3 3 1 1 1 2 2 2 3 4 4 4 4 4 1 1 2 2 3 3 3 4 4 5 5 5 1 1 2 2 3 4 4 4 4 5 5 6 1 1 2 3 3 4 5 5 5 5 5 6 1 1 2 3 4 4 5 5 5 6 6 6