1. 15-211 Fundamental Structures of Computer Science
Dynamic Programming
Fundamental Structures of Computer Science
Ananda Guna
February 10, 2005

2. Basic Idea of Dynamic Programming
Not much to do with “dynamic” or “programming”
idea from control theory
Solve problems by building up solutions to sub problems
Basically reduce exponential time algorithms to polynomial time algorithms

3. Example 1 - Fibonacci The Fibonacci sequence Leonardo Pisano f(0) = 0
f(n+2) = f(n+1) + f(n)
Leonardo Pisano
aka Leonardo Fibonacci
How many rabbits can be produced from a single pair in a year’s time? (1202)
Assume
New pair of offspring each month
Each pair becomes fertile after one month
Rabbits never die

4. Memoization

5. Fibonacci – the recursive program
public static long fib(int n) {
if (memo[n] != -1) return memo[n];
if (n<=1) return n;
long u = fib2(n-1);
long v = fib2(n-2);
memo[n] = u + v;
return u + v; }

6. Memoization Remember previous results
When computing the value of a function, return the saved result rather than calculating it again
The “function” must be a function!
No side effects
Returns the same value each time

7. Memoization Remember previous results
When computing the value of a function, return the saved result rather than calculating it again
The “function” must be a function!
No side effects
Returns the same value each time
Saving the values
Array
Hashtable
Trade-offs
Time to retrieve vs. time to compute
Storage space vs. time to compute

8. Fibonacci – the recursive program
public static long fib(int n) {
if (memo[n] != -1) return memo[n];
if (n<=1) return n;
long u = fib(n-1);
long v = fib(n-2);
memo[n] = u + v;
return u + v; }
Fib:13

9. Fibonacci – the recursive program
public static long fib(int n) {
if (memo[n] != -1) return memo[n];
if (n<=1) return n;
long u = fib(n-1);
long v = fib(n-2);
memo[n] = u + v;
return u + v; }
Fib2s

10. Fibonacci – the recursive program
public static long fib(int n) {
if (memo[n] != -1) return memo[n];
if (n<=1) return n;
long u = fib(n-1);
long v = fib(n-2);
memo[n] = u + v;
return u + v; }
memo = new long[n+1];
for(int i=0; i<=n; i++)
memo[i] = -1;
Fib2s

11. Memoization Name coined by Donald Michie, Univ of Edinburgh (1960s)
Fib is the Perfect Example
Useful for
game searches
evaluation functions
web caching

12. Fibonacci – optimizing the memo table
public static long fib(int n) {
if (n<=1) return n;
long last = 1;
long prev = 0;
long t = -1;
for (int i = 2; i<=n; i++) {
t = last + prev;
prev = last;
last = t; }
return t;
Reduce memo table to two entries
Fib3

13. Fibonacci – static table
static long[] tab;
public static void buildTable(int n) {
tab = new long[n+1];
tab[0] = 0;
tab[1] = 1;
for (int i = 2; i<=n; i++)
tab[i] = tab[i-1] + tab[i-2];
return; }
public static long fib(int n) {
return tab[n];
Amortize table building
cost against multiple
calls to fib.
Fib4

14. Dynamic Programming Build up to a solution
Solve all smaller subproblems first
Combine solutions to get answers to larger subproblems

15. Dynamic Programming Build up to a solution Issues
Solve all smaller subproblems first
Combine solutions to get answers to larger subproblems
Issues
Are there too many subproblems?
Can answers be combined?

16. Example 2 - Knapsack Problem
Imagine a homework problem with few different parts, A thru G
A B C D E F G
value:
time:
You have 15 hours
Which parts should you complete in order to get “maximum” credit?

17. Problem Class Operations Research What about a “greedy algorithm” ?
Prioritize tasks to maximize the outcome
What about a “greedy algorithm” ?

18. A Dynamic Programming Approach
Consider a general problem with N parts, 1, 2, …., N and time[i] and value[i] are the time and value of item i.
Let T be the total time
Idea
Create a table A where A[i][t] denotes max value we get if we use items 1,2…i and allow t time.

19. Knapsack problem ctd.. A B C D E F G value: 7 9 5 12 14 6 12
time:
Table A t |
i :0| | 1| 2| 3| 4|
...|
A has size (N+1) by (T+1).

20. The big question?
How to figure out the next row from the previous ones?
valueArray[i][t] = MAX( valueArray[i-1][t],
// don't use item i
valueArray[i-1][t - time[i]] + value[i]) //use it.

21. Pseudo Code for(t=0; t <= T; ++t) valueArray[0][t] = 0;
for(i=1; i <= N; ++i) {
for(t=0; t < time[i]; ++t)
valueArray[i][t] = valueArray[i-1][t];
for(t=time[i]; t <= T; ++t) {
valueArray[i][t] =
MAX( valueArray[i-1][t],
valueArray[i-1][t - time[i]] + value[i]); }

22. Questions?
Complete the table for i=4, t=11.
What is the running time?

23. Work area

24. Code ComputeValue(N,T)
// T = time left, N = # items still to choose from {
if (T <= 0 || N = 0) return 0;
if (time[N] > T) return ComputeValue(N-1,T);
// can't use Nth item
return max(value[N] + ComputeValue(N-1, T - Time[N]),
ComputeValue(N-1, T)); }
What is the runtime of this code?

25. Memoizing As we calculate values, make a memo of those
ComputeValue(N,T)
// T = time left, N = # items still to choose from {
if (T <= 0 || N = 0) return 0;
if (arr[N][T] != unknown) return arr[N][T];
// OK, we haven't computed it yet. Compute and store.
if (time[N] > T)
arr[N][T] = ComputeValue(N-1,T);
else arr[N][T] =
max(value[N] + ComputeValue(N-1, T - Time[N]),
ComputeValue(N-1, T));
return arr[N][T]; }
What is the runtime of this code?

26. Fractional Knapsack Problem aka Supermarket Shopping Spree

27. FKP formulation V(k, A) = V(k-1, A) V(k-1, A – sk) + vk
Max shopping cart value when choosing from items 1…k and using a cart with capacity A
V(k, A) =
V(k-1, A)
V(k-1, A – sk) + vk
Possibility 1:
Max value is the same as when choosing from items 1…k-1
max
Max value from items 1…k-1, but leaving room for item k
Possibility 2:
Max value includes selecting item k

28. The memoization table V 0 1 2 3 … C 1 2 3 … n
Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 A
V … C 1 2 3 … n k

29. The memoization table V 0 1 2 3 … C 1 0 0 0 0 … 0 2 3 0 0 0 10 … 10 …
Let v1=10, s1=3, v2=2, s2=1, v3=9, s3=2 A
V … C 1 2 3 … n
… 0
… 10
… 12 k …

30. Using dynamic programming
Key ingredients:
Simple subproblems.
Problem can be broken into subproblems, typically with solutions that are easy to store in a table/array.
Subproblem optimization.
Optimal solution is composed of optimal subproblem solutions.
Subproblem overlap.
Optimal solutions to separate subproblems can have subproblems in common.

31. Example 3 - Matrix multiplication
Four matrices. Compute ABCD.
A: 50 x 10 B: 10 x 40
C: 40 x 30 D: 30 x 5
Matrix multiplication is associative
Can compute the product in many ways
(((AB)C)D) (A((BC)D))
((AB)(CD)) (A(B(CD))) etc.

32. Matrix multiplication
Four matrices. Compute ABCD.
A: 50 x 10 B: 10 x 40
C: 40 x 30 D: 30 x 5
Matrix multiplication is associative
Can compute the product in many ways
(((AB)C)D) (A((BC)D))
((AB)(CD)) (A(B(CD))) etc.
Cost of multiplication
M1: p x q M2: q x r
Naïve algorithm requires pqr multiplications

33. Matrix multiplication
Four matrices. Compute ABCD.
A: 50 x 10 B: 10 x 40
C: 40 x 30 D: 30 x 5
Cost of multiplication
M1: p x q M2: q x r
Naïve algorithm requires pqr multiplications

34. Matrix multiplication
Four matrices. Compute ABCD.
A: 50 x 10 B: 10 x 40
C: 40 x 30 D: 30 x 5
Cost of multiplication
M1: p x q M2: q x r
Naïve algorithm requires pqr multiplications
Example costs
(((AB)C)D)
50x10x x40x x30x5 = 87,500
(A(B(CD)))
40x30x5 + 10x40x5 + 50x10x5 = 10,500

35. Matrix multiplication
How to find the best association?
Can this be solved using a greedy approach?

36. Matrix multiplication
How to find the best association?
Can this be solved using a greedy approach?

37. Matrix multiplication
How to find the best association?
Can this be solved using a greedy approach?

38. Dynamic programming How best to associate
A1 x A2 x x An
Determine best costs of all possible segments
Aleft , . . ., Aright

39. Dynamic programming How best to associate
A1 x A2 x x An
Determine best costs of all possible segments
Aleft , . . ., Aright
How many possible orderings?
(A1 x A2 x. . .x Ai) (Ai+1 x Ai+2 x. . .x An)
T(N) = sum1i<N T(i) T(N-i)
Catalan numbers – exponential growth
T(1)=T(2)=1 T(3)=2 T(4)=5

40. Using dynamic programming
4 ingredients needed:
An optimization problem.
Simple subproblems.
Problem can be broken into subproblems
Easy to store in a table or array.
Subproblem optimization.
Optimal solution is composed of optimal subproblem solutions.
Subproblem overlap.
Optimal solutions to separate subproblems can have subproblems in common.

41. Dynamic programming Are there too many subproblems?
Can answers be combined?

42. Next Week
More on
Dynamic Programming