1. Dynamic Programming

2. Dynamic Programming Originally the “Tabular Method” Key idea:
Problem solution has one or more subproblems that can be solved recursively
The subproblems are overlapping
The same subproblem will get solved multiple times
Solve each subproblem only one time
After solving, future attempts just “look up” the solution
Can be a tremendous time savings when it applies
Compared to divide-and-conquer/recursion, can compute much more
Can expand the range of what is feasible to solve dramatically
Greedy approaches better, but don’t usually apply

3. Two approaches to Dynamic Programming
Top-Down:
The “natural” way to create DP solutions from a recursive one
Formulate problem as a recursive problem
Need to identify a set of parameters that define the problem state
This is usually the key challenge in DP problems
Test that it works on simple cases!
Every time a function is called, first check to see if that has been computed before
If so, just return the pre-computed result
If it needs to be computed, compute like usual, but store the result once computed
Referred to as “memorization”
Usually stored in a table of some sort, but could use map or other structure.
Should never have to recompute for those parameters in the future.

4. Two approaches to Dynamic Programming
Bottom-up
Again, need to formulate problem as a state with some parameters
Each parameter becomes a “dimension”
The “base case” of the table should be easily computed/defined
Fill table from base case up to more complex states
Each row/col/etc. should need to look at earlier rows/cols/etc.
Finally, just look up an entry to find the answer

5. Comparisons of Methods
Top-down:
Easier to convert a recursive program to this approach
If the exploration of the state space is sparse, can be more efficient
Only visits the states that are needed
Bottom-up:
Avoids repeated function calls – usually just several for-loops
Can offer savings in terms of memory: e.g. keep only previous row of table to compute next row

6. Example: Fibonacci Sequence
F(i) = F(i-1) + F(i-2)
So, F(5) = F(4) F(3)
= F(3) + F(2) + F(2) + F(1)
= F(2) + F(1) + F(1) + F(0) + F(1) + F(0) + F(1)
= F(1)+F(0)+F(1)+F(1)+F(0)+F(1)+F(0)+F(1)
= 5

7. Example: Fibonacci Sequence
Notice:
Lots of overlapping subproblems (you end up solving for F(1) and F(0) several times)
We can speed this up by not recomputing the same thing over and over
Top-down: memoize
Bottom-up: fill array

8. Top down Dynamic Programming
Memoization: store the solutions when computed (e.g. in an array or a map). Any time you need to compute something, first see if it’s already been stored in that map.
Fibonacci:
F(5) = F(4) + F(3)
= F(4) + (F(2) + F(1))
= F(4) + ((F(1) + F(0)) + 1) // We already know F(1)
= F(4) + ( 1 + 1) // We now computed and store F(2)=1
= F(4) // We now computed and store F(3)=2
= (F(3) + F(2)) + 2
= (2 + 1) // We already know F(2) and F(3) from earlier
= // We now computed and store F(4)
= 5 // We now know F(5)

9. Top down Dynamic Programming
Memoization: store the solutions when computed (e.g. in an array or a map). Any time you need to compute something, first see if it’s already been stored in that map.
Fibonacci:
F(5) = F(4) + F(3)
= F(4) + (F(2) + F(1))
= F(4) + ((F(1) + F(0)) + 1) // We already know F(1)
= F(4) + ( 1 + 1) // We now computed and store F(2)=1
= F(4) // We now computed and store F(3)=2
= (F(3) + F(2)) + 2
= (2 + 1) // We already know F(2) and F(3) from earlier
= // We now computed and store F(4)
= 5 // We now know F(5)

10. Bottom-up Dynamic Programming
Start from “base” case and compute everything from there
Fibonacci:
F(0) = 0, F(1) = 1
F(2) = F(1) + F(2) = 0+1 = 1
F(3) = F(2) + F(1) = 1+1 = 2
F(4) = F(3) + F(2) = 2+1 = 3
F(5) = F(4) + F(3) = 3+2 = 5

11. Pros and Cons Top-Down: Bottom-Up:
You can convert a recursive solution to a top-down memorization very easily
Just add 2 things to the recursive call:
Start the recursive call with a check to see if that parameter combination already was computed, and if so, return that result
At the end of the recursive call, store the results for that parameter combination in a map
But, you still have recursive calls – sometimes the call stack can get too deep.
Can be efficient if the space of parameter values to test is not “dense”
Bottom-Up:
More efficient in many cases – no extra function calls.
Need to write the routine as a bottom-up routine (can’t just convert a recursive solution)
More efficient only if basically all lower values are needed to compute upper values.

12. Example: Range Sum
When you need to compute sums over various ranges of an array
Store a running sum in an array:
Sum will keep the sum of all values from 1 to x (notice starting index at 1, here)
To calculate sum over [a,b], Take Sum[b]-Sum[a-1]
e.g. sum from [3,9] = Sum[3] – Sum[2] = 3 – (-1) = 4
This works well if lots of queries, and values don’t change
If the values do change, use Range/Fenwick trees…
Index 1 2 3 4 5 6 7 8 9 10
Value - -4 -1 -6
Sum

13. Max 1D Range Sum
What if we want to find maximum sum over any subrange?
Naïve approach: For i from 1 to n, for j from i to n
Calculate sum[i,j] (including an array of length 0)
Generally too slow: quadratic time!
Alternative: linear time Dynamic Programming
Will keep track of a sum, work from index 0 forward.
Keeps positive sum – maximum of sum to that point
If sum ever drops below 0, set it to 0
Indicates that the sum of stuff up to that point can’t be positive

14. Max 1D Range Sum
What if we want to find maximum sum over any subrange?
Will keep track of a sum, work from index 0 forward.
Keeps positive sum
If sum ever drops below 0, set it to 0
Indicates that the sum of stuff up to that point can’t be positive
Index 1 2 3 4 5 6 7 8 9 10
Value - -4 -1 -6
Sum

15. Other examples 2D Range Sum Longest Increasing Subsequence
Coin Change (value, n-numcoins)
NP complete problems with fast actual implementations using DP:
0-1 knapsack
Traveling Salesman