1. Analysis & Design of Algorithms (CSCE 321)
Prof. Amr Goneid
Department of Computer Science, AUC
Part 13. Branch & Bound
Prof. Amr Goneid, AUC

2. Branch & Bound Algorithms
Introduction
General Idea
Example:
The 0/1 Knapsack
Problem
Prof. Amr Goneid, AUC

3. Introduction
Backtracking uses depth-first search with pruning to traverse the (virtual) state space.
We can achieve better performance for many problems using a breadth-first search with pruning. This approach is known as Branch-and-Bound.
Branch and bound is a systematic method for solving optimization problems
Prof. Amr Goneid, AUC

4. Introduction
B&B is a rather general optimization technique that applies where the greedy method and dynamic programming fail.
However, it is generally slower.
On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on average.
Prof. Amr Goneid, AUC

5. The General Idea
The general idea of B&B is a BFS-like search for the optimal solution
Not all nodes get expanded (i.e., their children generated). Rather, a carefully selected criterion determines which node to expand and when, and another criterion tells the algorithm when an optimal solution has been found.
Prof. Amr Goneid, AUC

6. The General Idea
Compared to backtracking, B&B requires two additional items:
A bound on any further additions,
The value of best solution obtained so far.
Prof. Amr Goneid, AUC

7. Example: The 0/1 Knapsack
Consider a Knapsack of capacity W. Find the most valuable subset of (n) items that can fit into the knapsack.
Item (i) has a weight wi and a value vi, the value density is di = vi / wi
We arrange the items according to the value density, d1 ≥ d2 ≥ … ≥ dn
Prof. Amr Goneid, AUC

8. The 0/1 Knapsack
The state space is like backtracking. Each item is represented in a layer in a binary tree, with levels 0 … n.
The root of the tree at level 0 indicates that no item is yet selected.
A left branch indicates selecting an item, a right branch indicates no selection.
Prof. Amr Goneid, AUC

9. The 0/1 Knapsack Consider w = total weight of items selected so far,
v = total value obtained so far.
Consider dbest = best value density among remaining items.
The upper bound on the value so far is computed as:
ub = v + (W – w) dbest
We branch to the next level from the more promising node (higher ub).
Prof. Amr Goneid, AUC

10. The 0/1 Knapsack
We terminate at the current node for one of the following 3 reasons:
The ub of node is not more promising than what is obtained so far.
The node violates the constraint w ≤ W
No further choices.
Prof. Amr Goneid, AUC

11. Example W = 10, n = 4 Item Weight Value Value Density 1 4 40 10 2 7 426 3 5 25 12
Prof. Amr Goneid, AUC

12. B & B State Space
Numbers in each node are for W, V, UB
0,0,100
1 No
1 yes 1 2
4,40,76
0,0,60
2 No
Inferior to node 8
2 yes 3 4
W = 11
4,40,70
3 yes
3 No 6 5
9,65,69
4,40,64
4 yes
4 No
Inferior to node 8 7 8
W = 12
9,65,65
Optimal Solution
Prof. Amr Goneid, AUC

13. The 0/1 Knapsack Level (0): Node (0) w = 0 v = 0 dbest = 10
ub = 0 + (W - 0)*10 = 100
Level (1):
Node (1) left: with item (1) w = 4 v = 40 ub = 40 + (W - 4)*6 = 76
Node (2) right: without (1) w = 0 v = 0 ub = 0 + (W – 0)*6 = 60
Node (1) is more promising, Branch from that node.
Level (2):
Node (3) left: with item (2) w = = 11 Not feasible
Node (4) right: without (2) w = 4 v = 40 ub = 40 + (W – 4)*5 = 70
Prof. Amr Goneid, AUC

14. The 0/1 Knapsack Level (3): Branching from node (4)
Node (5) left: with item (3) w = 9 v = 65 ub = 65 + (W – 9)* 4 = 69
Node (6) right: without (3) w = 4 v = 40 ub = 40 + (W – 4) *4 = 64
Node (5) is more promising, Branch from that node
Level (4): Branching from node (5)
Node (7) left: with item (4) w = = 12 Not feasible
Node (8) right: without (4) w = 9 v = ub = 65
Node (8) is the optimal solution
The final solution is:
(item 1 + item 3): total weight is 9
total value is 65
Prof. Amr Goneid, AUC

15. Exercise
Solve the following instance of the knapsack problem by the branch-and-bound algorithm:
W = 16, n = 4
Item
Weight
Value
Value Density 1 10
100 2 7 63 9 3 8 56 4 12
Prof. Amr Goneid, AUC