1. CSCE 411 Design and Analysis of Algorithms
Set 5: Dynamic Programming
Dr. J. Michael Moore
Fall 2014
Based primarily on slides by Prof. Jennifer Welch
CSCE 411, Fall 2014: Set 5

2. Drawback of Divide & Conquer
Sometimes can be inefficient
Fibonacci numbers:
F0 = 0
F1 = 1
Fn = Fn-1 + Fn-2 for n > 1
Sequence is 0, 1, 1, 2, 3, 5, 8, 13, …
CSCE 411, Fall 2014: Set 5

3. Computing Fibonacci Numbers
Obvious recursive algorithm:
Fib(n):
if n = 0 or 1 then return n
else return (Fib(n-1) + Fib(n-2))
CSCE 411, Fall 2014: Set 5

4. Recursion Tree for Fib(5)
CSCE 411, Fall 2014: Set 5

5. How Many Recursive Calls?
If all leaves had the same depth, then there would be about 2n recursive calls.
But this is over-counting.
However with more careful counting it can be shown that it is Ω((1.6)n)
Still exponential!
CSCE 411, Fall 2014: Set 5

6. Save Work Wasteful approach - repeat work unnecessarily
Fib(2) is computed three times
Instead, compute Fib(2) once, store result in a table, and access it when needed
CSCE 411, Fall 2014: Set 5

7. More Efficient Recursive Alg
F[0] := 0; F[1] := 1; F[n] := Fib(n);
Fib(n):
if n = 0 or 1 then return F[n]
if F[n-1] = NIL then F[n-1] := Fib(n-1)
if F[n-2] = NIL then F[n-2] := Fib(n-2)
return (F[n-1] + F[n-2])
computes each F[i] only once
called memoization
CSCE 411, Fall 2014: Set 5

8. Example of Memoized Fib
fills in F[4] with 3,
returns 3+2 = 5 1 2 3 4 5
Fib(5) 1
fills in F[3] with 2,
returns 2+1 = 3
NIL 1
Fib(4)
NIL 2
NIL 3
fills in F[2] with 1,
returns 1+1 = 2
Fib(3)
NIL 5
returns 0+1 = 1
Fib(2)
CSCE 411, Fall 2014: Set 5

9. Get Rid of the Recursion
Recursion adds overhead
extra time for function calls
extra space to store information on the runtime stack about each currently active function call
Avoid the recursion overhead by filling in the table entries bottom up, instead of top down.
CSCE 411, Fall 2014: Set 5

10. Subproblem Dependencies
Figure out which subproblems rely on which other subproblems
Example:
F0 F F F … Fn Fn Fn
CSCE 411, Fall 2014: Set 5

11. Order for Computing Subproblems
Then figure out an order for computing the subproblems that respects the dependencies:
when you are solving a subproblem, you have already solved all the subproblems on which it depends
Example: Just solve them in the order
F0 , F1 , F2 , F3 , …
called Dynamic Programming
CSCE 411, Fall 2014: Set 5

12. DP Solution for Fibonacci
Fib(n):
F[0] := 0; F[1] := 1;
for i := 2 to n do
F[i] := F[i-1] + F[i-2]
return F[n]
Can perform application-specific optimizations
e.g., save space by only keeping last two numbers computed
time reduced from
exponential to linear!
CSCE 411, Fall 2014: Set 5

13. Matrix Chain Order Problem
Multiplying non-square matrices:
A is n x m, B is m x p
AB is n x p whose (i,j) entry is ∑aik bkj
Computing AB takes nmp scalar multiplications and n(m-1)p scalar additions (using basic algorithm).
Suppose we have a sequence of matrices to multiply. What is the best order?
must be equal
CSCE 411, Fall 2014: Set 5

14. Why Order Matters Suppose we have 4 matrices:
A, 30 x 1
B, 1 x 40
C, 40 x 10
D, 10 x 25
((AB)(CD)) : requires 41,200 scalar mults.
(A((BC)D)) : requires 1400 scalar mults.
CSCE 411, Fall 2014: Set 5

15. Matrix Chain Order Problem
Given matrices A1, A2, …, An, where Ai is
di-1 x di:
[1] What is minimum number of scalar multiplications required to compute A1· A2 ·… · An?
[2] What order of matrix multiplications achieves this minimum?
Focus on [1]; see textbook for how to do [2]
CSCE 411, Fall 2014: Set 5

16. A Possible Solution Try all possibilities and choose the best one.
Drawback is there are too many of them (exponential in the number of matrices to be multiplied)
Need to be more clever - try dynamic programming!
CSCE 411, Fall 2014: Set 5

17. Step 1: Develop a Recursive Solution
Define M(i,j) to be the minimum number of scalar mults. needed to compute Ai· Ai+1 ·… · Aj
Goal: Find M(1,n).
Basis: M(i,i) = 0.
Recursion: How to define M(i,j) recursively?
CSCE 411, Fall 2014: Set 5

18. Defining M(i,j) Recursively
Consider all possible ways to split Ai through Aj into two pieces.
Compare the costs of all these splits:
best case cost for computing the product of the two pieces
plus the cost of multiplying the two products
Take the best one
M(i,j) = mink(M(i,k) + M(k+1,j) + di-1dkdj)
CSCE 411, Fall 2014: Set 5

19. Defining M(i,j) Recursively
(Ai ·…· Ak)·(Ak+1 ·… · Aj) P1 P2
minimum cost to compute P1 is M(i,k)
minimum cost to compute P2 is M(k+1,j)
cost to compute P1· P2 is di-1dkdj
CSCE 411, Fall 2014: Set 5

20. Step 2: Find Dependencies Among Subproblems1 2 3 4 5
n/a
GOAL!
computing the red
square requires the
blue ones: to the
left and below.
CSCE 411, Fall 2014: Set 5

21. Defining the Dependencies
Computing M(i,j) uses
everything in same row to the left:
M(i,i), M(i,i+1), …, M(i,j-1)
and everything in same column below:
M(i,j), M(i+1,j),…,M(j,j)
CSCE 411, Fall 2014: Set 5

22. Step 3: Identify Order for Solving Subproblems
Recall the dependencies between subproblems just found
Solve the subproblems (i.e., fill in the table entries) this way:
go along the diagonal
start just above the main diagonal
end in the upper right corner (goal)
CSCE 411, Fall 2014: Set 5

23. Order for Solving Subproblems1 2 3 4 5
n/a
CSCE 411, Fall 2014: Set 5

24. Pseudocode pay attention here to remember actual sequence of mults.
for i := 1 to n do M[i,i] := 0
for d := 2 to n-1 do // diagonals
for i := 1 to n-d+1 to // rows w/ an entry on d-th diagonal
j := i + d // column corresponding to row i on d-th diagonal
M[i,j] := infinity
for k := i to j-1 to
M[i,j] := min(M[i,j], M[i,k]+M[k+1,j]+di-1dkdj)
endfor
pay attention here
to remember actual
sequence of mults.
running time O(n3)
CSCE 411, Fall 2014: Set 5

25. Example M: 1 2 3 4
1200
700
1400
n/a
400
650
10,000
1: A is 30x1
2: B is 1x40
3: C is 40x10
4: D is 10x25
CSCE 411, Fall 2014: Set 5

26. Keeping Track of the Order
It's fine to know the cost of the cheapest order, but what is that cheapest order?
Keep another array S and update it when computing the minimum cost in the inner loop
After M and S have been filled in, then call a recursive algorithm on S to print out the actual order
CSCE 411, Fall 2014: Set 5

27. Modified Pseudocode keep track of cheapest split point
for i := 1 to n do M[i,i] := 0
for d := 1 to n-1 do // diagonals
for i := 1 to n-d to // rows w/ an entry on d-th diagonal
j := i + d // column corresponding to row i on d-th diagonal
M[i,j] := infinity
for k := 1 to j-1 to
M[i,j] := min(M[i,j], M[i,k]+M[k+1,j]+di-1dkdj)
if previous line changed value of M[i,j] then S[i,j] := k
endfor
keep track of cheapest split point
found so far: between Ak and Ak+1 
CSCE 411, Fall 2014: Set 5

28. Example M: 1 2 3 4
1200
700
1400
n/a
400
650
10,000
1: A is 30x1
2: B is 1x40
3: C is 40x10
4: D is 10x25 S: 1 1 1 2 3 3
CSCE 411, Fall 2014: Set 5

29. Using S to Print Best Ordering
Call Print(S,1,n) to get the entire ordering.
Print(S,i,j):
if i = j then output "A" + i //+ is string concat
else
k := S[i,j]
output "(" + Print(S,i,k) + Print(S,k+1,j) + ")"
CSCE 411, Fall 2014: Set 5

30. Example S: <<draw recursion tree on board>> 1 2 4 n/a 3
CSCE 411, Fall 2014: Set 5

31. Longest Common Subsequence
Another example of dynamic programming
Given two strings
X = [x1, x2 ,…, xm] and
Y = [y1, y2, …, yn]
find the longest common subsequence (LCS) of X and Y
not necessarily contiguous!
Example:
X = [A, B, C, B, D, A, B]
Y = [B, D, C, A, B, A]
one LCS is [B, C, B, A]
another LCS is [B, D, A, B]
CSCE 411, Fall 2014: Set 5

32. Brute Force Solution Take one of the strings, say X
Consider each possible subsequence of X
Check if that subsequence appears in Y
Would take exponential time since there are 2m possible subsequences of X
for each of the m characters in X, there are two choices (it is in the subsequence or is not)
CSCE 411, Fall 2014: Set 5

33. Recursive Expression for Solution
Let C[i,j] be length of LCS of
Xi = [x1, …, xi] and
Yj = [y1, …, yj]
Goal: C[m,n]
Basis:
C[0,j] = 0 (Xi is empty) for all j
C[I,0] = 0 (Yj is empty) for all I
i ranges from 0 to m, j ranges from 0 to n
CSCE 411, Fall 2014: Set 5

34. Recursive Expression for Solution
Formula for C[i,j]:
Case 1: xi = yj.
Recursively find LCS of Xi-1 and Yj-1 and append xi
So C[i,j] = C[i-1,j-1] + 1 if i > 0, j > 0, and xi = yj xi yj
equal
Xi-1
Yj-1
CSCE 411, Fall 2014: Set 5

35. Recursive Expression for Solution
Case 2: xi ≠ yj.
Recursively find LCS of Xi-1 and Yj
Recursively find LCS of Xi and Yj-1
Take the longer one
So C[i,j] = max{C[i,j-1], C[i-1,j]} if i > 0, j > 0, and xi ≠ yj xi yj
not equal
Xi-1
Yj-1
CSCE 411, Fall 2014: Set 5

36. Determine Dependencies Among Subproblems
C[i,j] depends on C[i-1,j-1], C[i-1,j], and C[i,j-1]
j-1 j n 1 1
i-1 i m
Goal
CSCE 411, Fall 2014: Set 5

37. Determine Order for Solving Subproblems
An order for solving the subproblems (i.e., filling in the array) that respects the dependencies is row major order:
do the rows from top to bottom
inside each row, go from left to right
CSCE 411, Fall 2014: Set 5

38. Pseudocode
fill in row 0 with all 0’s fill in column 0 with all 0’s for each row 1 through m do for each column 1 through n do fill in C[i,j] according to formula
CSCE 411, Fall 2014: Set 5

39. Time Complexity There are O(mn) subproblems (entries in the array).
Each subproblem takes O(1) time to compute.
Total time is O(mn).
CSCE 411, Fall 2014: Set 5

40. Computing the Actual Subsequence
Keep track of which neighboring table entry gives the optimal solution to a subproblem:
above, to the left, or diagonal
break ties arbitrarily
Use another 2-D array, b, for this information.
store  in b[i,j] if xi = yj (this character is part of a common subsequence)
store  in b[i,j] if LCS of Xi and Yj-1 was longer
store  in b[i,j] if LCS of Xi-1 and Yj was longer
Trace backwards from b[m,n] until reaching an edge
every time  is encountered, the corresponding character is added to the longest common subsequence being constructed
CSCE 411, Fall 2014: Set 5

41. More Dynamic Programming Examples
Assembly Line Scheduling
Floyd-Warshall All-Pairs Shortest Path Algorithm
<board work>
CSCE 411, Fall 2014: Set 5

42. Bellman-Ford Single-Source Shortest Paths Algorithm
If we only want to find the shortest (min. wt.) paths starting from a particular vertex, maybe we can do it faster than for the all-pairs problem
In Floyd-Warshall, the recursive calls considered subproblems that used a smaller set of vertices
Now let’s see what we can do if the recursive calls consider paths that have fewer edges in them (don’t confuse with “shortest path”)
CSCE 411, Fall 2014: Set 5

43. Ideas for Bellman-Ford
As usual for shortest path problems, we assume the graph has no negative-weight cycles (otherwise, no solution)
Note that the maximum number of edges in any shortest path is |V|−1
Use as building blocks shortest paths with at most k edges to find shortest paths with at most k+1 edges
CSCE 411, Fall 2014: Set 5

44. Bellman-Ford: Recursive Solution
Let D(i,k) be the minimum weight of all paths from source vertex s to vertex i that use at most k edges
If there is no path from source to i with at most k edges, then D(i,k) is ∞
Goals: D(i,|V|−1) for all vertices i
Basis: D(i,1) = w(s,i) (weight of edge (s,i))
Recursion: D(i,k) is equal to the minimum, over all (in) neighbors j of i, of D(j,k−1) + w(j,i)
If this minimum is larger than D(i,k−1), then set D(i,k) to D(i,k−1)
CSCE 411, Fall 2014: Set 5

45. Bellman-Ford Idea Picture
min wt. path from s to j
with at most k−1 edges j
edge from j to i s
edge from
j’ to i j’ i
min wt. path from s to j’
with at most k−1 edges
edge from j’’ to i
min wt. path from s to j’’
with at most k−1 edges
j’’
Use this info
to find min wt. path
from s to i with at
most k edges
CSCE 411, Fall 2014: Set 5

46. Implementing Bellman-Ford Idea: First Attempt
We have a 2-D array D of size |V|*(|V|−1) = Θ(V2)
Determine dependencies: To fill in D(i,k), we need D(j,k−1), for all (in) neighbors j of i.
If G is a DAG, we can fill in D in topological sort order, with one pass
But what if we have cycles?
Also, do we really need to keep all the values for smaller values of k?
CSCE 411, Fall 2014: Set 5

47. Implementing Bellman-Ford Idea: Second Attempt
Keep one variable d[v] for each vertex v, which is estimate of minimum weight from source s to v; initially ∞
Consider each edge (u,v) and see if u offers v a cheaper path from s
compare d[v] to d[u] + w(u,v)
Repeat this process |V| − 1 times to ensure that accurate information propagates from s, no matter what order the edges are considered in
CSCE 411, Spring 2013: Set 5a

48. Bellman-Ford SSSP Algorithm, Part 1
input: directed or undirected graph G = (V,E,w)
//initialization
initialize d[v] to infinity and parent[v] to nil for all v in V other than the source
initialize d[s] to 0 and parent[s] to s
// main body
for i := 1 to |V| − 1 do
for each (u,v) in E do // consider in arbitrary order
if d[u] + w(u,v) < d[v] then
d[v] := d[u] + w(u,v)
parent[v] := u
CSCE 411, Spring 2013: Set 5a

49. Bellman-Ford SSSP Algorithm, Part 2
// check for negative weight cycles
for each (u,v) in E do
if d[u] + w(u,v) < d[v] then
output "negative weight cycle exists"
CSCE 411, Spring 2013: Set 5a

50. Running Time of Bellman-Ford
O(V) iterations of outer for loop
O(E) iterations of inner for loop
O(VE) time total
CSCE 411, Spring 2013: Set 5a

51. Bellman-Ford Example a 3 2 s b —4 4 1 c <board work>
process edges in order
(c,b)
(a,b)
(c,a)
(s,a)
(s,c) 3 a 2 s b —4 4 1 c
<board work>
CSCE 411, Spring 2013: Set 5a

52. Correctness of Bellman-Ford
Assume no negative-weight cycles.
Lemma: d[v] is never an underestimate of the actual shortest path distance from s to v.
Lemma: If there is a shortest s-to-v path containing at most i edges, then after iteration i of the outer for loop, d[v] is at most the actual shortest path distance from s to v.
Theorem: Bellman-Ford is correct.
Proof: Follows from these 2 lemmas and fact that every shortest path has at most |V| − 1 edges.
CSCE 411, Spring 2013: Set 5a

53. Correctness of Bellman-Ford
Suppose there is a negative weight cycle.
Then the distance will decrease even after iteration |V| − 1
shortest path distance is negative infinity
This is what part 2 of the code checks for.
CSCE 411, Spring 2013: Set 5a

54. Speeding Up Bellman-Ford
The previous example would have converged faster if we had considered the edges in a different order in the for loop
move outward from s
If the graph is a DAG (no cycles), we can fully exploit this idea to speed up the running time
CSCE 411, Spring 2013: Set 5a

55. DAG Shortest Path Algorithm
input: directed graph G = (V,E,w) and source vertex s in V
topologically sort G
d[v] := infinity for all v in V
d[s] := 0
for each u in V in topological sort order do
for each neighbor v of u do
d[v] := min{d[v],d[u] + w(u,v)}
CSCE 411, Spring 2013: Set 5a

56. DAG Shortest Path Algorithm
Running Time is O(V + E).
Example: <board work>
CSCE 411, Spring 2013: Set 5a