1. CS 312: Algorithm Design & Analysis Lecture #26: 0/1 Knapsack This work is licensed under a Creative Commons Attribution-Share Alike 3.0 Unported License.Creative Commons Attribution-Share Alike 3.0 Unported License Slides by: Eric Ringger, with contributions from Mike Jones, Eric Mercer, Sean Warnick

2. Announcements  Homework #17  Due now  Project #5 – Gene Sequence Alignment  WB Experience: due Wednesday  Early: next Monday  Due: next Wednesday  Mid-term exam:  Review: Wednesday  Exam days: Thu., Fri., Sat.  In Testing Center

3. Objectives  Contrast 0/1 Knapsack with Divisible Knapsack  Apply the DP methodology to 0/1 Knapsack  Analyze efficiency

4. Divisible Knapsack Problem How would you formulate this problem mathematically?

5. Divisible Knapsack Problem Fix Order of Objects means and means {, } and means

6. Divisible Knapsack Problem Fix Order of Objects Corresponding Object Weights Corresponding Object Values

7. Divisible Knapsack Problem The problem is formulated mathematically as

8. Divisible Knapsack Problem The problem is formulated mathematically as

9. Divisible Knapsack Problem The problem is formulated mathematically as

10. Divisible Knapsack Problem Problems of this form are called Linear Programs. How to solve?

11. 0/1 Knapsack  Still looking for optimal loads  Still constrained by weight capacity  BUT cannot divide objects into fractions  Will greedy approach find an optimal solution every time?  No. There is no universally optimal selection function.  Other solution ideas?  Try all combinations  DP

12. Naïve Solution  Try each combination and see which one gives the best answer.  O(2 n )  Can we do better? n objects

13. Dynamic Programming 1.Ask: Am I solving an optimization problem? 2.Devise a minimal description (address) for any problem instance and sub-problem 3.Divide problems into sub-problems: define the recurrence to specify the relationship of problems to sub-problems 4.Check that the optimality property holds: An optimal solution to a problem is built from optimal solutions to sub-problems. 5.Store results – typically in a table – and re-use the solutions to sub-problems in the table as you build up to the overall solution. 6.Back-trace / analyze the table to extract the composition of the final solution.

14. Optimization? Step #1

15. 0/1 Knapsack Sub-Problems  How should we define the sub-problems?  What if we use the same idea as the DP coins algorithm to inspire our thinking here?  Enforce weight limits  Idea: sub-problems indexed by:  i: consider objects up through type i  j: consider knapsack with capacity j (up to W)  Sub-problem:  V[i,j] stores the max value for a load of capacity j using objects up through type i  Final answer: V[n,W] Step #2

16. 0/1 Knapsack Sub-Problems  Relationships among sub-problems  To compute V[i,j], we can either include an object of type i or not.  If we do not include one, then  V[i,j] = V[i-1,j]  If we do include one (only if w[i]  j), then  V[i,j] = V[i-1,j-w[i]]+v[i]  Initial conditions / boundary cases:  For j  0, V[i,j] = 0  For i  0, V[i,j] = 0  Summary: Step #3 Step #4

17. 0/1 Knapsack Sub-Problems

20. 0/1 Knapsack: Larger Example Remember: Entry V[i, j ] represents the max value you can get from objects 0 through i at weight j.

21. Extracting Solution Components Extracting the solution: Start with V[5,11] (which means “use all objects 1-5 to get a solution with weight = 11”). V[4,11] = V[5,11] and V[4,11-7]+28 != 40, so don’t include object 5. Step #6

22. Next, try V[4,11] which means “use all objects 1-4 to get a solution with weight = 11). V[4,11] != V[3,11] and V[3,11-6]+22 = 40, so do include object 4. Extracting Solution Components

23. Next, try V[3,5] which means “use all objects 1-3 to get a solution with weight = 5). V[3,5] != V[2,5] and V[2,5-5]+18 = 18, so include object 3. Extracting Solution Components

24. Summary: Extracting Solution Components Interpret the blue arrows:

25. Summary: Extracting Solution Components Interpret the blue arrows:

26. How long does it take?

27. Efficiency

29. 0/1 Knapsack on-the-fly, Visualized with the Table

30. 0/1 Knapsack on-the-fly V(5,11) V(4,11) V(4,11-w[5])+v[5]= V(4,7)+28 V(3,11)V(3,5)+22 V(3,7) V(3,1)+22 V(2,11) V(1,11) ………………… It’s a DAG; constructed in constrained fashion.

31. Assignment  Homework #16.5  0/1 Knapsack Example  New DP problem!