1. Brute Force Approaches
Techniques for finding optimal solutions to hard problems “relatively” quickly
Backtracking
Branch and Bound
Combining efficient solutions with brute force approaches
Other issues

2. The Knapsack Problem Input Goal Capacity K
n items with weights wi and values vi
Goal
Output a set of items S such that
the sum of weights of items in S is at most K
and the sum of values of items in S is maximized

3. Greedy Does Not Work What are possible greedy strategies?
Highest Density First
Highest Value First
Lowest Weight First
Prove these are not optimal for the 0-1 knapsack problem.

4. Multi-constraint Knapsack
Input
Capacity K1 and K2
n items with integer weights wi, size si, and values vi
Goal
Output a set of items S such that
the sum of weights of items in S is at most K1
the sum of sizes of items in S is at most K2
and the sum of values of items in S is maximized
Can we use dynamic programming?

5. Situation where dynamic programming does not work
What if our problem can’t be described with integers?
wA = 2 vA = $40
wB =  vB = $50
wC = 1.98 vC = $100
wD = 5 vD = $95
wE = 3 vE = $30
We have to resort to brute force….

6. Brute Force Generate all possible solutions
With n items, there are 2n solutions to be generated
Check each to see if they satisfy the constraint
Save maximum solution that satisfies constraint
Can be represented as a tree

7. Brute Force: Branching
Out B C D E B C D E
There are 2n solutions for us to consider. We scan through them for the most valuable legal one.
As a matter of fact, the first one was already out before we even considered E. So we could have stopped searching right there!
Weight = 15.12
Value = $315
Weight = 8.98
Value = $235
Weight = 9.98
Value = $225

8. Backtracking
In the tree representation, we can think of the previous algorithm doing a DFS in the tree
If we reach a point where a solution no longer is feasible, there is no need to continue exploring
We can “backtrack” from this point and potentially cut off much of the tree and many of the solutions
In the given example, backtracking would be much more effective if we had even more items or a smaller knapsack capacity

9. Backtracking In Out A B B C C C C D D D E D E D D E D E D E E E E E E
2, $40
, $50
1.98, $100
5, $95
3, $30
Backtracking A In
Out B B C C C C D D D E D E D D E D E D E
As soon as we choose the D in the first branch, we’re already over-weight, so we can stop there.
If there were a F, G, H, I, etc… you can imagine that this would be very useful -- it would cut off an entire sub-tree from needing to be checked.
Can we do better? E E E E E E
Weight = 8.98
Value = $235
Weight = 9.98
Value = $225

10. Branch and Bound
We can backtrack if we know the best possible solution in current subtree is worse than current best solution obtained so far.
Estimates for improvement given current ordering
A down can give $315
B down -> $275
C down -> $225
D down -> $125
E down -> $30

11. Branch and Bound In Out A B B C C C C D D D D D D E E E E E E E E
2, $40
, $50
1.98, $100
5, $95
3, $30
Branch and Bound A In
Out B B C C C C D D D D D D
If choosing ALL of the remaining values won’t give us as good an answer as we have, we should stop here!
A down can give $315
B down -> $275
C down -> $225
D down -> $125
E down -> $30 E E E E E E E E
Weight = 7.12
Value = $190
Weight = 8.98
Value = $235

12. Generating a good bound
The key to branch and bound is having a good initial bound.
How might we generate a good initial bound?

13. 2, $40
, $50
1.98, $100
5, $95
3, $30
Order of items
Since there is no fixed ordering of items, is there a better way to order the input items?
Highest weight first
Generate infeasible solutions quickly
Highest density first
Generate good solution quickly
Which is better depends on input

14. Heaviest on Top In Out D B B E E A A A C A C C C C D: 5, $95 B: , $50
(Max remaining = $220) E E A A A C A
Weight comes into play first!
This would be more dramatic if total weight was much larger than allowed weight.
This worked well in this case because the heaviest was worth a lot. But what if it isn’t? C C C C
Weight = 8.14
Value = $145
Weight = 10.0
Value = $165
Weight = 9.98
Value = $225
Weight = 8.98
Value = $235

15. Best Density on Top In Out C A A D D B B B B E E E E
Here we put the most likely to be included on the top.
Next Problem: Polygon Triangulation E E E E
Weight = 8.98
Value = $235

16. Greedy and Brute Force
Suppose half the inputs to our knapsack problem all have the same weight.
If all inputs had the same weight, we could implement a greedy solution
However, since half do not, we cannot use greedy alone to find optimal solution
Combine brute-force approach with greedy to find optimal solution more quickly.

17. The Breakdown... In Out Greedy on half! F1 F2 F2 F3 F3 F3 F3 F4 F4 F4

18. Algorithm Backtrack on half the inputs
At leafs, apply greedy strategy on the other half of the inputs
Comparison
Pure brute force: O(2n)
Combination: O(n2n/2)

19. How much better? Assumptions 250 would take over 35 years
Suppose n=50
We can test 1,000,000 solutions/second.
250 would take over 35 years
225 can be generated in half an hour
Plus marginal time to generate greedy solution for each of the 225 solutions

20. Another combination
Suppose half the inputs to our knapsack problem have small integral weights/values while the other half have real weights/values.
How can we combine approaches to solve this efficiently?

21. The n-Queens Problem Input Task
Positive integer n
Task
Place n queens on an n by n chessboard so that no two queens attack each other (on same row, column, diagonal), or report that this is impossible
Solve the n-queens problem for n = 1, 2, 3

22. n=4 Pure brute force search Improvements
16 squares on a 4 by 4 chessboard
Leads to 16 * 15 * 14 * 13 = 43,680 possible solutions
Improvements
At most one queen per row: 44 = 256 possible solutions
Backtracking: If two queens already attack before final queen placed, backtrack

23. Larger values of n n=8 n=12 At most one queen per row: 88 = 16,777,216
Early backtracking: 2057 nodes total
Time to find first solution: 114 nodes
n=12
At most one queen per row: 1212
Early backtracking: 856,189 nodes total
Time to find first solution: 262 nodes

24. The Problem
In the U.S. navy, the SEALS are each specially trained in a wide variety of skills so that small teams can handle a multitude of missions.
If there are k different skills needed for a mission, and n SEAL members that can be assigned to the team, find the smallest team that will cover all of the required skills.
Andersen knows hand-to-hand, first aid, and camouflage
Butler knows hand-to-hand and snares
Cunningham knows hand-to-hand
Douglas knows hand-to-hand, sniping, diplomacy, and snares
Eckers knows first-aid, sniping, and diplomacy
This is also known as the “Set Cover” problem. Its described in detail in your textbook.

25. Greedy Algorithm
What is the obvious greedy algorithm for this problem?
Find a counter-example to the optimality of greedy for this problem.

26. Brute Force Approach What is the brute-force approach?
How can we simplify the problem as far as possible in polynomial time?
How can we set bounds on the solutions?
When will we need to do backtracking?
What order should we test the potential members in when branching?
* This first one should be easy
* Some simplifications are obvious if we look at our example…
* Bounds are easy… if we’ve already included more people than we did in our best solution, we can stop.
* Will we ever need to? Not with good simplification!
* Can we make both sub-problems much easier?
Another question: What data structures to use?