2. Dynamic Programming 2 Mani Chandy mani@cs.caltech.edu

3. The Pattern Given a problem P, obtain a sequence of problems Q 0, Q 1, …., Q m, where: –You have a solution to Q 0 –The solution to P can be obtained from the solution to Q m, –The solution to a problem Q j, j > 0, can be obtained from solutions to problems Q k, k < j, that appear earlier in the sequence.

4. Dynamic Progamming Pattern P Given problem P Propose a partial ordering of problems Q0Q0 QmQm You know how to compute solution to Q 0 You can compute the solution to P from the solution to Q m

5. Creative Step Finding problems Q i from problem P More mechanical step: Determining the function that computes the solution S k for problem Q k from the solutions S j of problems Q j for j < k.

6. Example: Matrix Multiplication 1 X N N X N NX1NX1 What is the cost of multiplying matrices of these sizes?

7. Cost of multiplying 2 matrices p x q q x r p rows and q columns. Cost is 2pqr because resultant matrix has pr elements, and The cost of computing each element is 2q operations.

8. Parenthesization 1 X N N X N NX1NX1 If we multiply these matrices first the cost is 2N 3. N X N Resulting matrix

9. Parenthesization 1 X N NX1NX1 N X N Cost of multiplication is N 2. Thus, total cost is proportional to N 3 + N 2 + N if we parenthesize the expression in this way.

10. Different Ordering 1 X N N X N NX1NX1 Cost is proportional to N 2

11. The Ordering Matters! One ordering costs O(N 3 ) The other ordering costs O(N 2 ) 1 X N N X N NX1NX1 1 X N N X N NX1NX1

12. Generalization: Parenthesization A1A1 op AnAn A3A3 A2A2 op …. op Associative operation Cost of operation depends on parameters of the operands. Parenthesize to minimize total cost. () () ()

13. Creative Step Come up with partial-ordering of problems Q i given problem P. Propose a partial ordering of problems Q0Q0 QmQm

14. Creative Step: Solution Q i,j is: optimally parenthesize the expression A i op ….. op A j Relatively “mechanical” steps: 1.Find partial ordering of problems Q i,j 2.Find function f that computes solution S i,j from solutions of problems earlier in the ordering.

15. Partial Ordering Structure Q 1,1 Q 2,2 Q 3,3 Q 4,4 Q 1,2 Q 2,3 Q 3,4 Q 1,3 Q 2,4 Q 1,4 Depends on Solutions known Solution to given problem obtained from solution to this problem Q 1,n.

16. The Recurrence Relation Let C[j,k] be the minimum cost of executing A j op … op A k. Base Case: ???? C[j,j] = 0 Induction Step: ???? C[j,k] for k > j is: min over all v of C[j,v]+C[v+1,k] + cost of the operation combining [j…v] and [v+1 … k] Proof: ???

17. For matrix multiplication Let j-th matrix have size: p (j-1) X p j Then the size of matrix obtained by combining [ j … v] is: ? p (j-1) X p v Then the size of matrix obtained by combining [ v+1 … k] is: ? p v X p k Cost of multiplying [j … v] and [v+1 … k] is p (j-1) X p v X p k

18. Computational Complexity Recurrence Relation for matrix multiplication: C[j,k] for k > j is: min over all v of (C[j,v]+C[v+1,k] + p (j-1) p v p k ) Time to compute C[j,k] is proportional to (k-j) Total time to compute C, for all j,k is: 0 x N {for bottom level, i.e., for C[j,j] } + 1 x (N-1) { for first level, i.e., for C[j,j+1]} + …. + k x (N-k) {for k-th level, i.e., for C[j, j+ k]} + ….. Show that total time is proportional to O(N 3 ).

19. Proof Structure What is the theorem that we are proving? We make an assertion about the meaning of a term. For instance, C[j,k] is the minimum cost of executing A j op …. op A k We are proving that this assertion is correct.

20. Proof Structure Almost always, we use induction. Base case: establish that the value of C[j,j] is correct. Induction step: Assume that the value of C[j, j+u] is correct for all u where u is less than V, and prove that the value of C[j, j+V] is correct. Remember what we are proving: C[j,k] is the minimum cost of executing A j op …. op A k

21. The Central Idea Bellman’s optimality principle Q i,j Q u,v Pick optimal Discard others Q a,z The discarded solutions for the smaller problem remain discarded because the optimal solution dominates them.

22. All-Points Shortest Path Given a weighted directed graph. The edge-weight W[j,k] represents the distance from vertex j to vertex k. There are no cycles of negative weight. For all j, k, compute D[j,k] where D[j,k] is the length of the shortest path from vertex j to vertex k.

23. The Creative Step Come up with partial-ordering of problems Q i given problem P. There are different problem sets Q i some better than others.

24. Creative Step Let F[j,k,m] be the length of the shortest path that has 1.at most m hops, and 2.is from vertex j to vertex k What is the partial-ordering of problems Q[j,k,m]?

25. A recurrence relation F[j,k,m] = min over all r of F[j,r,m-1] + W[r,k] Base case: F[j,k,1] = ????? W[j,k] (assume W[j,j] = 0 for all j) Obtaining solution for given problem P D[j,k] = F[j,k,n-1]

26. Proof of Correctness What are we proving? We are proving that the meaning we gave to F[j,k,m] is correct Base Case We show that F[j,k,1] is indeed the length of the shortest path from vertex j to vertex k that traverses at most one edge. Induction Step Assume that F[j,k,m] is the length of the shortest path from j to k that traverses at most m edges, for all m less than p, and prove that F[j,k,p] is the min length from j to k that traverses at most p edges

27. Proof of Induction Step Consider any path Z from a vertex j to a different vertex k where the path traverses at most p edges. We must prove: length(Z) is at least F[j,k,p] Let r be the prefinal vertex in path Z, i.e., r is the last vertex before k in path Z. Partition the path Z into the path Z’ from vertex j to vertex r followed by the edge (r,k). Then: length(Z) = length(Z’) + W[r,k] By the induction assumption: length(Z’) ?????

28. Proof of induction step length(Z’) >= F[j,r,p-1] From the induction hypothesis j r k at most p-1 hops 1hop Path Z Path Z’

29. Proof of Induction Step length(Z) = length(Z’) + W[r,k] length(Z’) >= F[j,r,p-1] Hence, length(Z) >= F[j,r,p-1] + W[r,k] F[j,k,p] = min over all r of F[j,r,p-1] + W[r,k] Hence, length(Z) >= F[j,k,p]

30. Proof of Induction Step We have shown that any path with at most p hops from vertex j to vertex k has length at least F[j,k,p]. Next, we show there exists a path with at most p hops from vertex j to vertex k that has length F[j,k,p]. From these two we conclude that the minimum length path with at most p hops from vertex j to vertex k has length F[j,k,p]. Left to you

31. Complexity? n4n4 Can you do better? Come up with partial-ordering of problems Q i given problem P. Let F[j,k,m] be the length of the shortest path from vertex j to vertex k that has at most m hops. 2m2m

32. Recurrence Relation Let F[j,k,m] be the length of the shortest path from vertex j to vertex k that has at most 2 m hops. Derive a recurrence relation from this meaning of F[j,k,m]. F[j,k,0] = ??? Length of shortest path from vertex j to vertex k that has at most 1 edge. W[j,k] F[j,k,m] = ??? Length of shortest path from vertex j to vertex k that has at most 2 m edges.

33. Derive Recurrence Relation F[j,k,m] = ??? Length of shortest path from vertex j to vertex k that has at most 2 m edges. F[j,k,m+1] = min over all v of (F[j,v,m] + F[v,k,m])

34. Proof Exactly the same proof structure as the previous case. Consider any path Z from j to k with at most 2 (m+1) hops. Let the number of hops in the path be t. Partition path Z into path Y followed by path Y’ where the number of hops in path Y is floor(t/2). Let Y end at vertex r. YY’ 1 2 3 4 5 6 7 8 Path Z r

35. Proof length(Z) = length(Y) + length(Y’) From the induction hypothesis: length(Y) >= F[j,r,m] and length(Y’) >= F[r,k,m] Since Z has at most 2 (m+1) hops, Y and Y’ each have at most 2 m hops. Hence length(Z) >= F[j,r,m] + F[r,k,m] From recurrence relation, F[j,r,m]+F[r,k,m] >= F[j,k,m+1] Hence, length(Z) >= F[j,k,m+1]

36. Proof We have shown that every path with at most 2 m hops from vertex j to vertex k has length at least F[j,k,m]. Now show that there exists a path with at most 2 m hops from vertex j to vertex k that has length F[j,k,m]. Proof has the same structure and is left to you.

37. Complexity? n 3 log(n) Can you do better? Come up with partial-ordering of problems Q i given problem P. Let F[j,k,m] be the length of the shortest path from vertex j to vertex k that traverses intermediate vertices only in set {1, … m-1}

38. Derive and Prove Recurrence