1. DYNAMIC PROGRAMMING

2. Optimization Problems
If a problem has only one correct solution, then optimization is not required
For example, there is only one sorted sequence containing a given set of numbers.
Optimization problems have many solutions.
We want to compute an optimal solution e. g. with minimal cost and maximal gain.
There could be many solutions having optimal value.
Dynamic programming is very effective technique.
Development of dynamic programming algorithms can be broken into a sequence steps as in the next.

3. Why Dynamic Programming?
Dynamic programming, like divide and conquer method, solves problems by combining the solutions to sub-problems.
Divide and conquer algorithms:
Partition the problem into independent sub-problem
Solve the sub-problem recursively
Combine their solutions to solve the original problem
In contrast, dynamic programming is applicable when the sub- problems are not independent.
Dynamic programming is typically applied to optimization problems.

4. Dynamic programming
Dynamic programming is a way of improving on inefficient divide-and-conquer algorithms.
By “inefficient”, we mean that the same recursive call is made over and over.
If same subproblem is solved several times, we can use table to store result of a subproblem the first time it is computed and thus never have to recompute it again.
Dynamic programming is applicable when the subproblems are dependent, that is, when subproblems share subsubproblems.
“Programming” refers to a tabular method

5. Difference between DP and Divide-and-Conquer
Using Divide-and-Conquer to solve these problems is inefficient because the same common subproblems have to be solved many times.
DP will solve each of them once and their answers are stored in a table for future use.

6. Elements of Dynamic Programming (DP)
DP is used to solve problems with the following characteristics:
Simple subproblems
We should be able to break the original problem to smaller subproblems that have the same structure
Optimal substructure of the problems
The optimal solution to the problem contains within optimal solutions to its subproblems.
Overlapping sub-problems
there exist some places where we solve the same subproblem more than once.

7. Steps to Designing a Dynamic Programming Algorithm
1. Characterize optimal substructure
2. Recursively define the value of an optimal solution
3. Compute the value bottom up
4. (if needed) Construct an optimal solution

8. Fibonacci Numbers Fn= Fn-1+ Fn-2 n ≥ 2 F0 =0, F1 =1
0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, …
Straightforward recursive procedure is slow!
Let’s draw the recursion tree

9. Fibonacci Numbers

10. Fibonacci Numbers
We can calculate Fn in linear time by remembering solutions to the solved subproblems – dynamic programming
Compute solution in a bottom-up fashion
In this case, only two values need to be remembered at any time

11. Assembly-line scheduling

12. Assembly-line scheduling
Automobiles factory with two assembly lines.
Each line has the same number “n” of stations. Numbered j = 1, 2, ..., n.
We denote the jth station on line i (where i is 1 or 2) by Si,j .
The jth station on line 1 (S1,j) performs the same function as the jth station on line 2 (S2,j ).
The time required at each station varies, even between stations at the same position on the two different lines, as each assembly line has different technology.
Time required at station Si,j is (ai,j) .
There is also an entry time (ei) for the auto to enter assembly line i and an exit time (xi) for the completed auto to exit assembly line i.

13. Assembly-line scheduling
After going through station Si,j, can either
q stay on same line
next station is Si,j+1
no transfer cost , or
q transfer to other line
next station is S3-i,j+1 or Si+i,j+1
transfer cost from Si,j to S3-i,j+1 is ti,j ( j = 1, … , n–1)
No ti,n, because the assembly line is complete after Si,n

14. Assembly-line Scheduling
Station S1,1
Station S1,2
Station S1,3
Station S1,4 …
Station S1,n-1
Station S1,n
Assembly line 1
a1,1
a1,2
a1,3
a1,4
a1,n-1
a1,n e1
t2,1
t1,1
t1,2
t2,2
t1,3
t2,3
t1,n-1
t2,n-1 … x1
Stations Si,j;
2 assembly lines, i = 1,2;
n stations, j = 1,...,n.
ai,j = assembly time at Si,j;
ti,j = transfer time to other line after station Si,j
ei = entry time from line i;
xi = exit time from line i .
auto
enters
Completed auto exits
a2,1
a2,2
a2,3
a2,n-1
a2,n
a2,4 x2 e2
Assembly line 2
Station S2,1
Station S2,2
Station S2,3
Station S2,4 …
Station S2,n-1
Station S2,n

15. To minimize the total processing time for one auto.
Problem Definition
Which stations should be chosen from line 1 and which from line 2 in order to minimize the total time through the factory for one car?
To minimize the total processing time for one auto.

16. Requires to examine Ω(2n) possibilities
Brute Force
Enumerate all possibilities of selecting stations
Compute how long it takes in each case and choose the best one
Solution:
There are 2n possible ways to choose stations
Infeasible when n is large!! 1
1 if choosing line 1
at step j (= n) 2 3 4 n
0 if choosing line 2
at step j (= 3)
Requires to examine Ω(2n) possibilities

17. Step 1: Find Optimal Structure
An optimal solution to a problem contains within it an optimal solution to subproblems.
The fastest way through station Si,j contains within it the fastest way through station S1,j-1 or S2,j-1 .
Thus can construct an optimal solution to a problem from the optimal solutions to subproblems.

18. Step 1: Optimal Solution Structure
Dynamic Programming
Step 1: Optimal Solution Structure
optimal substructure : choosing the best path to Sij.
The structure of the fastest way through the factory (from the starting point)
The fastest possible way to get through Si,1 (i = 1, 2)
Only one way: from entry starting point to Si,1
Take time is entry time (ei)

19. Step 1: Optimal Solution Structure
a1,j-1
a1,j
The fastest way to reach S1,j-1
The fastest possible way to get through Si,j (i = 1, 2)
(j = 2, 3, ..., n). Two choices:
Stay in the same line: Si,j-1  Si,j
Time is fi[j-i]+ ai,j
Transfer to other line: S1,j-1  S2,j or S2,j-1  S1,j
Time is f2[j-1] + t2,j-1 + a1,j or f1[j-1] + t1,j-1 + a2,j or
a2,j-1
a1,j
The fastest way to reach S2,j-1
t2,n-1

20. Step 2: Recursive Solution
Define the value of an optimal solution recursively in terms of the optimal solution to sub-problems.
Sub-problem here
Finding the fastest way through station j on both lines (i=1,2)
Let fi [j] be the fastest possible time to go from starting point through Si,j
The fastest time to go all the way through the factory: f*
x1 and x2 are the exit times from lines 1 and 2, respectively

21. Step 2: Recursive Solution
The fastest time to go through Si,j
e1 and e2 are the entry times for lines 1 and 2 3 2 In
Out
f1[1] = e1 + a1,1;
f2[1] = e2 + a2,1.
f1[j] = min (f1[j-1] + a1,j, f2[j-1] + t2,j-1 + a1,j) for j ≥ 2;
f2[j] = min (f2[j-1] + a2,j, f1[j-1] + t1,j-1 + a2,j) for j ≥ 2; 4 2

22. Step 2 : Recursive Solution
․ Overlapping subproblem: The fastest way through station S1,j is
either through S1,j-1 and then S1,j , or through S2,j-1 and then transfer to
line 1 and through S1,j.
․ fi[j]: fastest time from the starting point through Si,j
․ The fastest time all the way through the factory.
fi[j] (i=1,2; j=1,2,…,n) records optimal values to the subproblems.
f* = min(f1[n] + x1, f2[n] + x2)

23. Step 2: Recursive Solution
To keep the track of the fastest way, introduce li[j] to record the line number (1 or 2), whose station j-1 is used in a fastest way through Si,j.
Introduce l* to be the line whose station n is used in a fastest way through the factory.
We avoid defining li[1] because no station precedes station 1 on either lines.

24. Step 2: Recursive Solution
Base Cases
f1[1] = e1 + a1,1
f2[1] = e2 + a2,1

25. Step 2: Recursive Solution
Two possible ways of computing f1[j],For j = 2, 3, . . ., n
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
Symmetrically, f2[j], For j = 2, 3, . . ., n
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
Objective function = f* = min(f1[n] + x1, f2[n] + x2)

26. Example: Computation of f1[1] & f2[1]]7 9 3 4 8 5 6 In
Out 2 1
f1[1] = e1 + a1,1 = = 9 7 9 3 4 8 5 6 In
Out 2 1
f2[1] = e2 + a2,1 = = 12

27. Example: Computation of f1[2]7 9 3 4 8 5 6 In
Out 2 1
f1[1] = 9
f2[1] = 12
j = 2
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
f1[2] = min (f1[1] + a1, 2, f2[1] + t2, 1 + a1, 2)
= min (9 + 9, ) = min (18, 23) = 18
l1[2] =
l1[2] = 1

28. Example: Computation of f2[2]7 9 3 4 8 5 6 In
Out 2 1
f1[1] = 9
f2[1] = 12
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j = 2
f2[2] = min (f2[1] + a2, 2, f1[1] + t1, 1 + a2, 2)
= min (12 + 5, ) = min (17, 16) = 16
l2[2] =
l2[2] = 1

29. Example: Computation of f1[3]7 9 3 4 8 5 6 In
Out 2 1
f1[2] = 18
f2[2] = 16
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j = 3
f1[3] = min (f1[2] + a1, 3, f2[2] + t2, 2 + a1, 3)
= min (18 + 3, )
= min (21, 20) = 20
l1[3] =
l1[3] = 2

30. Example: Computation of f2[3]7 9 3 4 8 5 6 In
Out 2 1
f1[2] = 18
f2[2] = 16
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j = 3
f2[3] = min (f2[2] + a2, 3, f1[2] + t1, 2 + a2, 3)
= min (16 + 6, )
= min (22, 27) = 22
l2[3] =
l2[3] = 2

31. Example: Computation of f1[4]7 9 3 4 8 5 6 In
Out 2 1
f1[3] = 20
f2[3] = 22
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j = 4
f1[4] = min (f1[3] + a1, 4, f2[3] + t2, 3 + a1, 4)
= min (20 + 4, )
= min (24, 27) = 24
l1[4] =
l1[4] = 1

32. Example: Computation of f2[4]7 9 3 4 8 5 6 In
Out 2 1
f1[3] = 20
f2[3] = 22
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j = 4
f2[4] = min (f2[3] + a2, 4, f1[3] + t1, 3 + a2, 4)
= min (22 + 4, )
= min (26, 25) = 25
l2[4] =
l2[4] = 1

33. Example: Computation of f1[5]7 9 3 4 8 5 6 In
Out 2 1
f1[4] = 24
f2[4] = 25
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j = 5
f1[5] = min (f1[4] + a1, 5, f2[4] + t2, 4 + a1, 5)
= min (24 + 8, )
= min (32, 35) = 32
l1[5] =
l1[5] = 1

34. Example: Computation of f2[5]7 9 3 4 8 5 6 In
Out 2 1
f1[4] = 24
f2[4] = 25
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j = 5
f2[5] = min (f2[4] + a2, 5, f1[4] + t1, 4 + a2, 5)
= min (25 + 5, )
= min (30, 32) = 30
l2[5] =
l2[5] = 2

35. Example: Computation of f1[6]7 9 3 4 8 5 6 In
Out 2 1
f1[5] = 32
f2[5] = 30
f1[j] = min (f1[j-1] + a1, j, f2[j-1] + t2, j-1 + a1, j)
j = 6
f1[6] = min (f1[5] + a1, 6, f2[5] + t2, 5 + a1, 6)
= min (32 + 4, )
= min (36, 35) = 35
l1[6] =
l1[6] = 2

36. Example: Computation of f2[6]7 9 3 4 8 5 6 In
Out 2 1
f1[5] = 32
f2[5] = 30
f2[j] = min (f2[j-1] + a2, j, f1[j-1] + t1, j-1 + a2, j)
j = 6
f2[6] = min (f2[5] + a2, 6, f1[5] + t1, 5 + a2, 6)
= min (30 + 7, )
= min (37, 43) = 37
l2[6] =
l2[6] = 2

37. Example: Computation of f*
f* = min (f1[6] + x1, f2[6] + x2)
= min (35 + 3, )
= min (38, 39) = 38
f1[6] = 35
f2[6] = 37
L* = 1

38. Entire Solution Set: Assembly-Line Scheduling7 9 3 4 8 5 6 In
Out 2 1 j j 1 2 3 4 5 6 9 18 20 24 32 35 12 16 22 25 30 37 2 3 4 5 6 1
Fi[j]
Li[j]
f* = 38
l* = 1

39. Fastest Way: Assembly-Line Scheduling
Let li[j] be the line number, 1 or 2, whose station j-1 is used in the solution.
eg. l1[5] = 2 means that the fastest way to reach station 5 of line 1 should pass through station 4 of line 2.
Let l* be the line whose station n is used in the solution.
eg. l*=2 means that the fastest way involves station n of line 2. 7 9 3 4 8 4 3 2 1 3 4 2 3 In
Out 2 1 2 1 2 4 2 8 5 6 4 5 7
l* = 1 =>
l1[6] = 2 =>
l2[5] = 2 =>
l2[4] = 1 =>
l1[3] = 2 =>
l2[2] = 1 =>
Station S1, 6 j 2 3 4 5 6 1
Station S2,5
Li[j]
Station S2,4
Station S1,3
Station S2,2
Station S1,1
l* = 1

40. Step 3: Optimal Solution Value
What we are doing, is:
Starting at the bottom level, continuously filling in the tables (f1[], f2[] and l1[], l2[]), and
Looking up the tables to compute new results for next higher level.
Most of the sub-problems are visited more than once.
Any clever method to handle them?
Exercise: what is the complexity of this algorithm?

41. Step 3: Optimal Solution Value
Compute initial values of f1 and f2
Compute the values of
f1[j] and l1[j]
Compute the values of
f1[j] and l1[j]
O(N)
Compute the values of
the fastest time through the
entire factory

42. Step 3: Optimal Solution Value

43. Step 4: Construct an optimal solution
Step 4. Construct an optimal solution from computed information.
Sample
PRINT-STATIONS()
1 i  l*
2 print “line ” i “, station ” n
3 for j  n down to 2
4 do i  li[j]
5 print “line ” i “, station ” j-1
line 1, station 6
line 2, station 5
line 2, station 4
line 1, station 3
line 2, station 2
line 1, station 1

44. Assembly-line Scheduling
What we have done is indeed Dynamic Programming
Now it is time to test your memory:
Dynamic Programming
Step 1 Characterize the structure of an optimal solution
Eg. Study the structure of the fastest way through the factory.
Step 2 Recursively define the value of an optimal solution
Step 3 Compute the value of an optimal solution in a bottom-up fashion
Step 4 Construct an optimal solution from computed information.