1. 15-211 Fundamental Data Structures and Algorithms Ananda Guna March 18, 2003 Dynamic Programming Part 1

2. Introduction  Many algorithms are based on solving large problems by solving smaller sub- instances and using them to find the solution  Examples – Quicksort, MergeSort  Sub-instances are independent  What if sub-instances are not mutually exclusive?  S ij  S i and S ij  S j, - Same sub-sub- instance is in two other sub-instances?  Is there a way to keep track of sub- instances that are already solved to avoid exponential blow-up?

3. Two main methods  To keep track of sub-instances  Memoizing : Store Partial Results in a hashtable for later reuse Easier to keep track of sub-instances that are solved A top-down approach  Dynamic Programming: Construct a table of Partial results A bottom-up approach More later..

4. Motivation Examples  Suppose S and T are two strings of lengths n and m  S = a 1 a 2 ….. a n  T = b 1 b 2 ….. b m  What is the longest subsequence (not necessarily consecutive) that is common to S and T?  Brute Force Solution  Its complexity?

5. Motivation Examples ctd..  Consider a homework assignment with different parts  Each part will have a point value and a time to do it.  You have 15 hours  Which tasks should you do to maximize the points?  We can look at all possible combinations…  Or alternatively, can a greedy algorithm work? part ABCDEFG pts 7951214612 hrs 3426735

6. Optimizing non-greedy problems  When the greedy-choice property fails, what can we do?  An alternative approach:  Dynamic programming  More later…  Start with Memoizing example

7. Fibonacchi  Start with something familiar  The Fibonacci sequence  f(0) = 0  f(1) = 1  f(n) = f(n-1) + f(n-2)

8. Fibonacci – the recursive program public static long fib(int n) {f (memo[n] != -1) return memo[n]; if (n<=1) return n; else return fib(n-1) + fib(n-2) memo[n] = u + v; }  What is the running time of fib(n)?

9. Improving Fibonacci  Let’s save all of the previous results  When computing the value of fib(n-1) and fib(n-2), return the saved result rather than calculating it again  Note:  It is important that fib is a “pure” function! No side effects Returns the same value each time

10. Memoization  Name coined by Donald Michie, Univ of Edinburgh (1960s)  Fib is the Perfect Example  Useful for  game searches  evaluation functions  web caching

11. memo = new long[n+1]; for(int i=0; i<=n; i++) memo[i] = -1; Fibonacci - the improved version public static long fib(int n) { if (n<=1) return n; if (memo[n] != -1) return memo[n]; else memo[n] = fib(n-1) + fib(n-2); return memo[n]; }

12. Recursive Fibonacci  What is the running time of our new version of Fibonacci?

13. Fibonacci – optimizing the memo table public static long fib(int n) { if (n<=1) return n; long last = 1; long prev = 0; long t = -1; for (int i = 2; i<=n; i++) { t = last + prev; prev = last; last = t; } return t; } Reduce memo table to two entries

14. Memoization  Works best with “pure” functions  No side effects  Returns the same value each time  Lots of ways to save the values  Arrays  Hashtables  …  Trade-offs  Time to retrieve vs. time to compute  Storage space vs. time to compute

15. Dynamic Programming

16. Binary Knapsack Problem, revisited  You win the Supermarket Shopping Spree contest.  You are given a shopping cart with capacity C.  You are allowed to fill it with any items you want from Giant Eagle.  Giant Eagle has items 1, 2, … n, which have values v 1, v 2, …, v n, and sizes s 1, s 2, …, s n.  How do you (efficiently) maximize the value of the items in your cart?

17. Consider the brute-force solution  The brute-force solution:  Try all possible combinations of items and compute the max value that fits in C.

18. But isn’t this impractical?  Although seemingly impractical, the brute-force approach usually has the advantage that it can be formulated simply and precisely.

19. Precise formulation  Let V(k, A) be the max value when choosing among items 1, 2, …, k and with a shopping cart of size A.  V(k, A) is a subproblem of the final problem of V(n, C).  (where n is the number of items in Giant Eagle)

20. Subproblem formulation, cont’d  V(k, A) = max of  V(k-1, A) [that is, don’t take the k th item]  V(k-1, A – s k ) + v k [or else take the k th item] Note the recursive structure (and similarity to Fibonacci).

21. FKP formulation V(k, A) = V(k-1, A) V(k-1, A – s k ) + v k max Max shopping cart value when choosing from items 1…k and using a cart with capacity A Possibility 1: Max value is the same as when choosing from items 1…k-1 Possibility 2: Max value includes selecting item k Max value from items 1…k-1, but leaving room for item k

22. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A

23. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0

24. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10

25. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10 0 2 2 10 … 12

26. The memoization table Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 V 0 1 2 3 … C 0 1 2 3 … n k A 0 0 0 0 … 0 0 0 0 10 … 10 0 2 2 10 … 12 0 2 9 11 …

27. Observations  Inefficient, or “brute-force,” solutions to dynamic programming problems often have simple recursive definitions.

28. Dynamic programming  Build up to a solution.  Solve all smaller subproblems first. Typically start with smallest subproblems and then work up to larger ones. Hope that there is substantial overlap between subproblems.  Combine solutions to get answers to larger subproblems.

29. Dynamic programming  Underlying idea:  Use memoization so that overlap can be exploited.  As smaller subproblems are solved, solving the larger subproblems might get easier.  Can sometimes reduce seemingly exponential problems to polynomial time.

30. Homework Example - Again  Consider a homework assignment with different parts  Each part will have a point value and a time to do it.  You have 15 hours  Which parts should you do to maximize the points? part ABCDEFG pts 7951214612 hrs 3426725

31. Finding a Solution Notation:  Call the parts "items" and number them 1,2,3,...,N. time[i] is the time of the i th item and value[i] is its value. Let T = total time we have. IDEA: 1. We're going to make a big 2-D array "valueArray" where valueArray[i][t] is the maximum value we can get if we only considered items 1,...,i and we only allowed ourselves t time units. 2. We're going to fill this out row by row, starting from small values of i, and using the solutions we've computed so far to fill out the next rows.

32. The Big question?  How to figure out the next row from the previous ones?  valueArray[i][t] = MAX { valueArray[i-1][t] OR // don't use item i { valueArray[i-1][t - time[i]] + value[i] //use item i.

33. Complete the table valueArray: t 0 1 2 3 4 5 6 7 8 9 15 +------------------------------------------------------ i :0| 0........... |------------------------------------------------------ 1| 0 0 0 7 7 |------------------------------------------------------ 2| 0 0 0 7 9 9 |------------------------------------------------------ 3| 0 0 5 7 9 12 14 |------------------------------------------------------ 4| 0 0 5 7 9 12 14 16...|  What is the solution if i = 7 & t = 15?

34. The recursive Program ComputeValue(N,T) // T = time left, N = # items still to choose from { if (T <= 0 || N = 0) return 0; if (time[N] > T) return ComputeValue(N-1,T); // can't use Nth item return max(value[N] + ComputeValue(N-1, T - Time[N]), ComputeValue(N-1, T)); }  What is the running time of this algorithm?

35. Using Memoizing ComputeValue(N,T) // T = time left, N = # items still to choose from { if (T <= 0 || N = 0) return 0; if (arr[N][T] != unknown) return arr[N][T]; // OK, we haven't computed it yet. Compute and store. if (time[N] > T) arr[N][T] = ComputeValue(N-1,T); else arr[N][T] = max(value[N] + ComputeValue(N-1, T - Time[N]), ComputeValue(N-1, T)); return arr[N][T]; }  What is the running time of this memoized version?

36. For Thursday  Read 7.6 (page 267)  HW: Solve 63-cents problem using dynamic programming, if 21-cent coin is allowed  More examples of dynamic programming  Homework 5 will be out today - due 4/14/03  “Mini Google” Assignment Web reader Web Crawler Web reader, web indexer, web spider Web Search