1. Lecture 4 Dynamic Programming

2. Basic Algorithm Design Techniques
Analyzing running time
Divide and conquer
Dynamic Programming
Greedy
Common Theme: To solve a large, complicated problem, break it into many smaller sub-problems.
Design algorithm

3. Dynamic Programming
Idea: Break the problem into many closely related sub-problems, memorize the result of the sub- problems to avoid repeated computation.

4. Warmup Example: Fibonacci Numbers
f(n) = f(n-1) + f(n-2), f(1) = f(2) = 1

5. Naïve solution … … Follow the recursion directly f(7) f(6) f(5) f(5)

6. Naïve solution … … Follow the recursion directly f(7) f(6) f(5) f(5)

7. Dynamic Programming Table
Memoization
f(n)
IF n < 3 THEN RETURN 1.
IF f(n) has been computed before THEN RETURN stored result.
res = f(n-1) + f(n-2)
Mark f(n) as computed, Store f(n) = res
RETURN res n 1 2 3 4 5 6 7 8
f(n) 13 21
Dynamic Programming Table

8. Filling in the table iteratively
Observation: f(n) is always computed in the order of n = 1, 2, 3, …
No need to recurse
f(n)
IF n < 3 THEN RETURN 1.
a[1] = a[2] = 1
FOR i = 3 TO n
a[i] = a[i-1] + a[i-2]
RETURN a[n]

9. Basic Steps in Designing Dynamic Programming Algorithms
Relate the problem recursively to smaller sub- problems. (Transition function, f(n) = f(n-1)+f(n-2))
Organize all sub-problems as a dynamic programming table. (A table for all values of n)
Fill in values in the table in an appropriate order. (n = 1, 2, 3, …)

10. How to figure out the transition function
Often: need to think of the problem as making a sequence of decisions
To find the transition function, enumerate all options for the last decision (rabbit + bunny)
For each option relate it to smaller subproblems (rabbit = f(n-1); bunny = f(n-2))

11. Example 1 Knapsack Problem
There is a knapsack that can hold items of total weight at most W. There are now n items with weights w1,w2,…, wn. Each item also has a value v1,v2,…,vn.
Goal: Select some items to put into knapsack 1. Total weight is at most W. 2. Total value is as large as possible.

12. Designing a DP algorithm for Knapsack
Step 1: think of the problem as making a seq. of decisions
For each item, decide whether we put it into the knapsack
Step 2: Focus on last decision, enumerate the options
For the last item, we either put it in, or leave it out.
Step 3: Try to relate each option to a smaller subproblem
Subproblem: Fill the remaining capacity using the remaining items
leave it out: number of items is now smaller.
put it in: reserve the capacity, both capacity and number of items are smaller.

13. States and Transition function
Example: Capacity W = 4, 3 items with (weight, value) = (1, 2), (2, 3), (3, 4).
States (sub-problems): What is the maximum value for a Knapsack with capacity j (= 0, 1, 2, 3, 4) and the first i (= 0, 1, 2, 3) items?
Use a[i, j] to denote this maximum value, we have
a[i - 1, j - wi] + vi (item i in knapsack)
a[i, j] =
max
a[i - 1, j] (item i not in knapsack)

14. Dynamic Programming Table
Example: Capacity W = 4, 3 items with (weight, value) = (1, 2), (2, 3), (3, 4). 1 2 3 4 5 6 +4

15. Outputting the Solution
Remember the choice (arrows) in the DP table.
Solution = {1, 3}, value = 6 1 2 3 4 5 6

16. Outputting the Solution
Another Example (capacity = 3)
Solution = {1, 2}, value = 5 1 2 3 4 5 6

17. Pseudo-code Knapsack Initialize a[i, 0] = 0, a[0, j] = 0 (Base Case)
FOR i = 1 to n (Enumerate #items)
FOR j = 1 to W (Enumerate capacity)
a[i, j] = a[i-1, j] (Case 1: not using item i)
IF j >= w[i] and a[i-1, j-w[i]] + v[i] > a[i,j] (if Case 2 is better)
a[i, j] = a[i-1, j-w[i]] + v[i] (Case 2: using item i)
Output(n, W)
IF n = 0 or W = 0 THEN RETURN
IF a[n, W] = a[n-1, W] THEN (Case 1)
Output(n-1, W)
ELSE (Case 2)
Output(n-1, W-w[n])
Print(n)