Dynamic Programming

Dynamic Programming Dividing a problem into subproblems Dynamic programming vs divide and conquer - Dynamic programming : subproblems are overlapped - Divide and conquer : subproblems are independent Used for finding optimized solutions

Assembly-Line Scheduling station 1 station 2 station 3 station 4 station 5 station 6 n = 6 Assembly Line 1 7 9 3 4 8 4 enters exits 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling 7 9 3 4 8 4 3 2 enters exits 4 2 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling 7 9 3 4 8 4 3 2 2 3 1 3 4 enters exits 2 1 2 2 1 4 2 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling 7 9 3 4 8 4 40 3 2 2 3 1 3 4 enters exits 2 1 2 2 1 4 2 41 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling 7 9 3 4 8 4 39 3 2 2 3 1 3 4 enters exits 2 1 2 2 1 4 2 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling station 1 station 2 station 3 station 4 station 5 station 6 number of cases = 2n Assembly Line 1 7 9 3 4 8 4 enters exits 8 5 6 4 5 7 Assembly Line 2

Assembly-Line Scheduling T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 Assembly Line 1 T enters exits Assembly Line 2 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6

Assembly-Line Scheduling T1,5 T1,6 T = min( T1,6 + 3, T2,6 + 2) 8 4 Divide and Conquer? 3 T 4 exits T1,6 = min( T1,5 + 4, T2,5 + 1 + 4) 1 2 T2,6 = min( T1,5 + 4 + 7, T2,5 + 7) 5 7 T2,5 T2,6

Assembly-Line Scheduling T1,5 T1,6 T = min( T1,6 + 3, T2,6 + 2) 8 4 Divide and Conquer? 3 T 4 exits T1,6 = min( T1,5 + 4, T2,5 + 1 + 4) 1 2 T2,6 = min( T1,5 + 4 + 7, T2,5 + 7) 5 7 T2,5 T2,6

Assembly-Line Scheduling 9 12 T1,1 T1,2 T1,3 Assembly Line 1 T1,1 = 9 7 9 3 T2,1 = 12 2 2 3 enters 2 1 4 8 5 6 Assembly Line 2 T2,1 T2,2 T2,3

Assembly-Line Scheduling 9 18 T1,1 T1,2 T1,3 Assembly Line 1 T1,1 = 9 7 9 3 T2,1 = 12 2 2 3 T1,2 = min(T1,1 + 9, T2,1 + 2 + 9) enters 2 1 4 8 5 6 Assembly Line 2 T2,1 T2,2 T2,3 12

Assembly-Line Scheduling 9 18 T1,1 T1,2 T1,3 Assembly Line 1 T1,1 = 9 7 9 3 T2,1 = 12 2 T1,2 = 18 2 3 enters T2,2 = min(T1,1 + 2 + 5, T2,1 + 5) 2 1 4 8 5 6 Assembly Line 2 T2,1 T2,2 T2,3 12 16

Assembly-Line Scheduling 9 18 20 22 24 25 32 30 35 37 T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 Assembly Line 1 7 9 3 4 8 4 38 3 2 T 2 3 1 3 4 enters exits 2 1 2 2 1 4 2 8 5 6 4 5 7 Assembly Line 2 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16

Assembly-Line Scheduling 32 30 35 37 T1,5 T1,6 T = T1,6 + 3 ? or T2,6 + 2 ? 8 4 38 3 T1,6 + 3 = T T 4 exits T2,6 + 2 > T 1 2 5 7 T2,5 T2,6

Assembly-Line Scheduling 32 30 35 37 T1,5 T1,6 T = T1,6 + 3 ? or T2,6 + 2 ? 8 4 38 3 T1,6 + 3 = T T 4 exits T2,6 + 2 > T 1 2 5 7 T2,5 T2,6

Assembly-Line Scheduling 32 30 35 37 T1,5 T1,6 T1,6 = T1,5 + 4 ? or T2,5 + 1 + 4 ? 8 4 38 3 T 4 T1,5 + 4 > T1,6 exits T2,5 + 1 + 4 = T1,6 1 2 5 7 T2,5 T2,6

Assembly-Line Scheduling 32 30 35 37 T1,5 T1,6 T1,6 = T1,5 + 4 ? or T2,5 + 1 + 4 ? 8 4 38 3 T 4 T1,5 + 4 > T1,6 exits T2,5 + 1 + 4 = T1,6 1 2 5 7 T2,5 T2,6

Assembly-Line Scheduling 9 18 20 22 24 25 32 30 35 37 T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 Assembly Line 1 7 9 3 4 8 4 38 3 2 T 2 3 1 3 4 enters exits 2 1 2 2 1 4 2 8 5 6 4 5 7 Assembly Line 2 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16

Assembly-Line Scheduling T1,5 T1,6 T2,5 T2,6 1. Analyze the problem T exits T = min( T1,6 + 3, T2,6 + 2)

Assembly-Line Scheduling T1,5 T1,6 T2,5 T2,6 1. Analyze the problem 2. Find a recursive solution. T exits T = min( T1,6 + 3, T2,6 + 2) T1,6 = min( T1,5 + 4, T2,5 + 1 + 4) T2,6 = min( T1,5 + 4 + 7, T2,5 + 7)

Assembly-Line Scheduling 9 18 20 24 32 35 1. Analyze the problem. T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 2. Find a recursive solution. T 3. Compute the fastest time. 38 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 25 30 37

Assembly-Line Scheduling 9 18 20 24 32 35 1. Analyze the problem. T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 2. Find a recursive solution. T 3. Compute the fastest time. 38 4. Construct the fastest way. T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 25 30 37

Assembly-Line Scheduling 9 18 20 24 32 35 1. Analyze the problem. T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 2. Find a recursive solution. T 3. Compute the fastest time. 38 4. Construct the fastest way. T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 This algorithm takes time. 12 16 22 25 30 37 Brute force :

When can we use Dynamic Programming? Optimal substructures Overlapping subproblems

Overlapping Subprograms T1,5 T1,6 T 8 4 3 T1,6 T2,6 T 4 exits T1,5 T2,5 T1,5 T2,5 1 2 5 7 T1,4 T2,4 T1,4 T2,4 T1,4 T2,4 T1,4 T2,4 T2,5 T2,6

Overlapping Subprograms T1,5 T1,6 T 8 4 3 T1,6 T2,6 T 4 exits T1,5 T2,5 T1,5 T2,5 1 2 5 7 T1,4 T2,4 T1,4 T2,4 T1,4 T2,4 T1,4 T2,4 T2,5 T2,6

Optimal Substructures Problems should be divided into subproblems. Optimal solution : from optimal subproblem solutions One part of the optimal solution to the problem should be an optimal solution to the subproblem solutions to the subproblems should be independent

Optimal Substructures 1. Shortest Path Problem 2. Longest Simple Path Problem d d s s

Optimal Substructures 1. Shortest Path Problem 2. Longest Simple Path Problem d d s s w w

Memoization Ordinary - Bottom-up strategy Memoization - Top-down strategy

Memoization Memoization Bottom Up T1,1 T1,2 T1,3 T1,4 T1,5 T1,6 - T2,1

Memoization Memoization T T1,6 T2,6 T1,5 T2,5 T1,4 T2,4 … … T1,1 T1,2 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 T1,4 T2,4 … …

Memoization Memoization T1,3 T1,2 T2,2 T1,1 T2,1 T1,1 T1,2 T1,3 T1,4 9 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 T1,1 T2,1

Memoization Memoization T1,3 T1,2 T2,2 T1,1 T2,1 T1,1 T1,2 T1,3 T1,4 9 18 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 T1,1 T2,1

Memoization Memoization T1,3 T1,2 T2,2 T1,1 T2,1 T1,1 T2,1 T1,1 T1,2 9 18 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 T1,1 T2,1 T1,1 T2,1

Memoization Memoization T1,4 T1,3 T2,3 T1,2 T2,2 T1,2 T2,2 T1,1 T2,1 9 18 20 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 T1,2 T2,2 T1,2 T2,2 T1,1 T2,1 T1,1 T2,1

Memoization Memoization T T1,6 T2,6 T1,5 T2,5 T2,4 T1,4 T1,4 T2,4 … … 9 18 20 24 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 T1,4 T2,4 … … T1,3 T2,3

Memoization Memoization T T1,6 T2,6 T1,5 T2,5 T2,4 T1,4 T1,4 T2,4 … … 9 18 20 24 32 - T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 25 T1,4 T2,4 … … … …

Memoization Memoization T T1,6 T2,6 T2,5 T1,5 T1,5 T2,5 T2,4 T1,4 T1,4 9 18 20 24 32 35 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 25 30 - T1,4 T2,4 … … … …

Memoization Memoization T = 38 T1,6 T2,6 T2,5 T1,5 T1,5 T2,5 T2,4 T1,4 9 18 20 24 32 35 T2,1 T2,2 T2,3 T2,4 T2,5 T2,6 12 16 22 25 30 37 T1,4 T2,4 … … … …

Why Memoization? Ordinary Dynamic Programming (Bottom-Up strategy) - when all subproblems need to be solved Memoization - when some subproblems do not need to be solved

Longest Common Subsequence DNA sequences : composed of four components – {A, C, G, T} How similar are they? S1 = ACCGGTCGTGCGCGGAAGCCGGCCGAA S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA

Longest Common Subsequence S1 = ACCGGTCGTGCGCGGAAGCCGGCCGAA S2 = GTCGTTCGGAATGCCGTTGCTCTGTAAA Longest Common Sequence : GTCGTCGGAAGCCGGCCGAA

Longest Common Subsequence X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> for i = 1 to m Yj = <y1, y2, …, yj> for j = 1 to n Zi,j = <z1, z2, …, zk> for i = 1 to m, j = 1 to n Longest common subsequence of Xi and Yj

Longest Common Subsequence 1. Analyze the problem. X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> Yj = <y1, y2, …, yj> Zi,j = <z1, z2, …, zk> (Longest common subsequence) Zi-1,j = <z1, z2, …, zk-1> or <z1, z2, …, zk> when xi used as zk otherwise Zi,j-1 = <z1, z2, …, zk> or <z1, z2, …, zk-1>

Longest Common Subsequence 1. Analyze the problem. X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> Yj = <y1, y2, …, yj> Zi,j = <z1, z2, …, zk> (Longest common subsequence) If xi = yj then zk = xi or zk = yj / / / Zi-1,j = <z1, z2, …, zk-1> or <z1, z2, …, zk> when xi used as zk otherwise

Longest Common Subsequence 1. Analyze the problem. X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> Yj = <y1, y2, …, yj> Zi,j = <z1, z2, …, zk> (Longest common subsequence) If xi = yj then zk = xi or zk = yj / / / Zi,j = Zi-1,j or Zi,j-1

Longest Common Subsequence 1. Analyze the problem. X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> Yj = <y1, y2, …, yj> Zi,j = <z1, z2, …, zk> (Longest common subsequence) If xi = yj then / Zi,j = Zi-1,j or Zi,j-1 Optimal If xi = yj then If xi = yj is not used as zk then Zi,j = Zi-1,j or Zi,j-1 If xi = yj is used as zk then Zi,j = Zi-1,j-1 + zk

Longest Common Subsequence 1. Analyze the problem. X = <x1, x2, …, xm>, Y = <y1, y2, …, yn> Xi = <x1, x2, …, xi> Yj = <y1, y2, …, yj> Zi,j = <z1, z2, …, zk> (Longest common subsequence) If xi = yj then / Zi,j = Zi-1,j or Zi,j-1 ci,j = max(ci-1,j, ci,j-1) ci,j = ci-1,j-1 + 1 If xi = yj then Zi,j = Zi-1,j-1 + zk

Longest Common Subsequence 2. Find a recursive solution. If i = 0 or j = 0 then ci,j = 0 Else If xi = yj then / ci,j = max(ci-1,j, ci,j-1) If xi = yj then ci,j = ci-1,j-1 + 1

Longest Common Subsequence 3. Compute the fastest time. C1,1 C1,2 . . . C1,n C2,1 C2,2 C2,n . Cm,1 Cm,2 Cm,n If xi = yj then / ci,j = max(ci-1,j, ci,j-1) If xi = yj then ci,j = ci-1,j-1 + 1

Longest Common Subsequence 4. Construct the common subsequence. C1,1 C1,2 . . . C1,n C2,1 C2,2 C2,n . Cm,1 Cm,2 Cm,n

Longest Common Subsequence 4. Construct the common subsequence. Ci-2,j-2 Ci-2,j-1 Ci-2,j Ci-1,j-2 Ci-1,j-1 Ci-1,j Ci,j-2 Ci,j-1 Ci,j If xi = yj then one element of The longest common subsequence move to ci-1,j-1 Else If ci,j = ci-1,j then move to ci-1,j If ci,j = ci,j-1 then If xi = yj then / ci,j = max(ci-1,j, ci,j-1) move to ci,j-1 If xi = yj then ci,j = ci-1,j-1 + 1

Dynamic Programming 1. Analyze the problem. Dividing a problem into subproblems. For optimization problems. Optimal substructures / overlapping subproblems Process of dynamic programmings 1. Analyze the problem. 2. Find a recursive solution. 3. Compute the fastest time/cost. 4. Construct the fastest path.