1. 1 Chapter 9 Dynamic Programming General Problem-Solving strategy Was born in the 1950s (Bellman) Based on successive decomposition Leads to functional equations More “Art” than “Science” (?) Difficult to teach/learn No general purpose software Usually taught/learned by example

2. 2 programme Shortest Path problem Knapsack problem Tower of Hanoi Puzzle Travelling Salesman Problem (??) Critical Path Problem (Chapter 10) And Dijkstra’s Algorithm (BYO) (details in due course)

3. 3 9.1 Shortest Path Problem Graphs: –Nodes (vertices) –Arcs (edges) Networks –Nodes –Arcs –Additional information (eg. length)

4. 4 The significance of the “drawing” is the precedence relationships it represents, not the visual picture. On the other hand, the picture can tell us a lot about the nature of the relationships... Make sure that you distinguish between the drawing and the relationships!!!!!

5. 5 5 1 43 2 5 1432

6. 6 Terminology Path Predecessor Successor Immediate Predecessor immediate Successor Origin (no predecessor) Destination (no successors) Cycle Acyclic graph

7. 7 Notation S(n) := Set of immediate predecessors of node n P(n) := Set of Immediate successors of node n.  := empty set

8. 8 Lemma 9.1.1 If a directed graph with a finite number of nodes N is acyclic, then the nodes can be numbered in such a way that the following condition is satisfied: m < n for all m in P(n), n=1,2,...,N In words, an ordering of nodes exists such that all arcs are directed from a lower numbered node to a higher numbered node.

9. 9 Labeling Procedure Find a node without a predecessor Label this node “1” Delete all arcs incident from “1” Set n=1 Repeat the process for n=n+1 until all nodes are labeled.

10. 10 Example 9.1.2

11. 11 11 2

12. 12 1 2 1 2 3 4 5 6 7

13. 13 Problem Statement Given a directed graph where each arc is assigned a numerical length, we want to find the shortest path between a specified origin s and a specified destination t, where the length of a path is equal to the sum of the arc lengths on that path. There are “generalised” shortest path problems where the length of a path is not “additive” (Eg. ??????)

14. 14 DP Strategy We embed the problem of interest (shortest path from origin to destination) in a family of other related problems. We establish some relationaships between the optimal solutions to these problems. We solve this relationship (functional equation) From the solution of the functional equation we obtain the solution to the problem of interest.

15. 15 Convention Origin = node 1 destination = node N

16. 16 Let f(j) := length of the shortest path from the origin to node j, j=1,2,...,N Note that we are interested “only” in f(N).

17. 17 Observation #1 f(j) = f(k) + a k,j for some k in P(j) (why?) j k 1 N (NILN)

18. 18 But,.... ? What is the value of k ????????? f(j) = f(k) + a k,j (k := “Best” immediate predecessor) We identify k by “doing the best we can”, ie. we find the value of k for which f(j) = f(k) + a k,j is the smallest possible among all immediate predecessor of node j.

19. 19 result DP functional equation

20. 20 Remark Since a ij = infinity if k is not in P(j), the functional equation can be rewritten as follows:

21. 21 Notation Let (Constructed as we solve the functional equation)

22. 22 Procedure We start by setting f(1)=0 Then, we determine the values of f(2),f(3),f(4),....... in this order. This is possible because our networks are acyclic. For small problems we can do it on the picture itself.

23. 23 Example 9.1.4 P(2)={1}, P(3){1}; P(4)={2,3}; P(5)={2,4}; P(6)={3,4}; P(7)={5,6}. 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6

24. 24 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0)

25. 25 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) x x

26. 26 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x x x

27. 27 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x

28. 28 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x

29. 29 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) x x

30. 30 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) x x Summary: f(1)=0; f(2)=3; f(3)=2;f(4)=6;f(5)=10;f(6)=9; f(7)=15. Question: How do we recover optimal paths?

31. 31 We go backward along marked arcs. 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) x x

32. 32 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) x x 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) O O

33. 33 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) x (9) x (15) O O 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) O (9) O (15) O O

34. 34 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x O (10) O (9) O (15) O O 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x x (10) O (9) O (15) O O

35. 35 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x x O (10) O (9) O (15) O O 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x O O (10) O (9) O (15) O O

36. 36 There are two optimal paths: p 1 =(1,3,4,5,7) ; p 2 =(1,3,4,6,7) 1 2 3 4 5 6 7 3 2 5 4 10 3 45 6 (0) (3) (2) (6) x O O (10) O (9) O (15) O O

37. 37 Remark It is OK (in the exam, assignment, etc) to do the calculation directly on the picture, but........ write down the details for one nontrivial node, eg:

38. 38 Comment The are many generalisations..........