Dynamic Programming Key characteristic of a Dynamic Program: Breaking up a large, unwieldy problem into a series of smaller, more tractable problems. Shortest Path Example:

Dynamic Programming General Idea: work backward from day 4 to day 1. For day 4, what is the shortest path to reach LA given you began the day in either Denver or San Antonio? f 4 (8) = 1030 f 4 (9) = 1390

Dynamic Programming Now consider day 3. Shortest path from Kansas City to LA is the minimum of the distance traveled in day 3 to a day 4 city, plus the shortest path from a day 4 city to LA : f 3 (5) = min{ f 4 (8), f 4 (9)} = min{1640*,2180} = 1640 Repeat for Omaha and Dallas: f 3 (6) = min{ f 4 (8)*, f 4 (9)} = 1570 f 3 (7) = min{ f 4 (8), f 4 (9)*} = 2330

Dynamic Programming Now consider day 2. f 2 (2) = min{ f 3 (5), f 3 (6), f 3 (7) } = min{2320*,2360,2710} = 2320 f 2 (3) = min{ f 3 (5)*, f 3 (6), f 3 (7) } = 2220 f 2 (4) = min{ f 3 (5)*, f 3 (6), f 3 (7) }= 2150 Now consider day 1. f 1 (1) = min{ f 2 (2)*, f 2 (3), f 2 (4) } = 2870

Dynamic Programming The shortest distance is The route that obtained the shortest distance can be found by backtracking from the optimal(*) solution found for day 1 through day 4. 1 – 2 – 5 – 8 – 10

Dynamic Programming Considering a single equation from the shortest route problem, a recursion formula can be developed: f 2 (2) = min{ f 3 (5), f 3 (6), f 3 (7) } f n-1 (i) = min{ d ij + f n (j)} where i is the city at the start of the day j and j is the set of possible cities at the end of the day.

Dynamic Programming Key characteristics of a dynamic program: 1)The problem can be subdivided into stages, with a policy decision at each stage. 2)Each stage has a number of states associated with the beginning of that stage. 3)The effect of the policy decision at each stage is to transform the current state to a state associated with the beginning of the next stage. 4)The solution procedure is designed to find an optimal policy for the overall problem. 5)Given the current state, an optimal policy for the remaining stages is independent of the policy decisions adopted in previous stages. 6)The solution procedure begins by finding the optimal policy for the last stage. 7)A recursive relationship that identifies the optimal policy for stage n, given the optimal policy for stage n + 1, is available.