1. Chapter 6
Branch & Bound (B & B)

2. Key Points Understand the pruning policy of B & B
Master the framework of B & B
FIFO
Priority queue
Learn from examples
Container Loading Problem (CLP)
0-1 Knapsack
Maximum Clique Problem (MCP)
Traveling Salesman Problem (TSP)

3. B & B vs. BT
Similar to BT, B & B searches solution in solution space tree
Different targets
BT usually searches all solutions subject to constraints
B & B usually searches a solution subject to constraints or maximal or minimal solution
B & B can be used to solve optimization problems without an exhaustive search in the average case
Different search strategies
BT: DFS
B & B: BFS or best first search

4. General Thoughts For a specific problem, define its solution space
X=(x1,…, xn)， xi∊Si={ai 1,…, ai ri} and the solution space is S1×S2×…×Sn
Determine a solution-space tree easy to search
Traverse the solution-space tree in BFS or best first search and avoid invalid search by pruning function
Constraint function
Bound function

5. General Thought (Cont.)
BFS or best first search
Each active node has only one chance to be an expanding node
Search procedure:
Once being an expanding one, the node produces all its children as candidate active nodes
The child judged as infeasible or sub-optimal solution is discarded
The remaining children are inserted into active node list
Chose next active node as an expanding node and repeat the above step
The process continues until the solution is found or the active node list is empty

6. Two Kinds of B & B Priority Queue FIFO
Use FIFO rule to choose an active node to be the next expanding node
Priority Queue
Use priority queue to choose an active node to be the next expanding node
To accelerate search speed, each active node maintains a bound value and the active node with the optimal bound value is chosen as the next expanding node towards the branch with the (an) optimal solution

7. 0-1 Knapsack Problem -- Review30 16 15 15
¥ ¥ ¥25
(a)3 items and the knapsack

8. (a)3 items and the knapsack
B & B -- FIFO 30 A
wt, cp =0 16 1 15 15 B C
0, 0
16, 45 1 1
¥ ¥ ¥25 D E
16, 45 F
15, 25 G
0, 0
(a)3 items and the knapsack 1 1 1 1 H I J K L M N O
0, 0
16, 45
30, 50
15, 25
15, 25

9. (a)3 items and the knapsack
B & B – Priority Queue 30 A
wt, cp =0 16 1 15 15 B
16，45 C
0，0 1 1
¥ ¥ ¥25 D E
16，45 F
15，25 G
0，0
(a)3 items and the knapsack 1 1 1 1 H I J K L M N O
16，45
30，50
15，25
15，25
0， 0

10. Traveling Salesman Problem -- Review
TSP has many practical applications
plane routing
telephone routing
networks
traveling salespeople

11. A weighted graph of 4 vertices
TSP
A Salesman wishes to travel around a given set of cities, and return to the beginning, covering the smallest total cost
Basic idea
Let G=(V, E) be a weighted graph and its weights are positive
A traveling route is a cycle that includes all vertices in G
The cost of a traveling route is the sum of its edge weights
Find the traveling route with minimal cost 1 4 3 2 6 5 20 10 30
A weighted graph of 4 vertices
Permutation Tree A B E D C K J I H G F Q P O N M L

12. A weighted graph of 4 vertices
TSP – B & B with FIFO A 1 B 1 4 3 2 6 5 20 10 30
A weighted graph of 4 vertices 2 3 4 30 C D 6 E 4 3 4 2 4 2 3 35 F 40 G 11 H 26 I 14 24 J K 4 3 4 2 3 2 L M N O P Q 59 66 25 25 59

13. TSP – B & B with Priority QueueA 1 B 2 3 4 1 4 3 2 6 5 20 10 30
A weighted graph of 4 vertices 30 C D 6 E 4 3 4 2 4 2 3 24 F G 11 H 26 I 14 J K 4 3 4 2 3 2 L M N O P Q 25 25 59

14. TSB – B & B
Conclusion
From the search process of TSP using B & B, we can see that, as compared to BT, B & B continually adjusts its search direction to prioritize the search of the subtree that is most likely to yield an optimal value

15. B & B
This strategy can be used to solve optimization problems without an exhaustive search in the average case
2 mechanisms:
A mechanism to generate branches when searching the solution space
A mechanism to generate a bound so that many braches can be terminated

16. Dynamic Search of Solution Space Tree
Construct a rational bound function to determine the bound of objective function [down, up]
For a maximization problem, compute up
For a minimization problem, compute down
BFS solution space tree:
An expanding node produces all its children and compute their bound values
If the bound value of a child indicates that its subtree cannot yield a solution better than the currently optimal, the child is discarded
Otherwise the child is inserted into the active node list
Pick the next expanding node from the active node list

17. B & B
Efficient in the average case because many branches can be terminated very early
Although it is usually very efficient, a very large tree may be generated in the worst case
Many NP-hard problem can be solved by B & B efficiently in the average case
The worst case time complexity is still exponential

18. Container Loading Problem – Review
Given n containers, each with weight wi, and 2 ships of volumes c1 and c2, and
Determine if there is a rational loading plan such that these n containers can be loaded onto the 2 ships
If yes, offer the loading plan

19. CLP -- Review
If there is a loading plan, the optimal one can be found in this way
Load as much as the 1st ship can load；
The rest containers are loaded onto the 2nd ship
Equivalent to choose a subset of containers such that their total weight is closest to c1, a special 0-1 knapsack problem

20. CLP -- FIFO
When checking an expanding node, first examine if its left child is feasible
If yes, insert the left child into the active node list
Then insert its right child into the active node list (as the right child must be feasible)
When its two children are produced, the expanding node is discard from active node list

21. CLP -- FIFO
The head of the queue of active nodes is deleted from the queue to be the next expanding node
There is a tail flag -1 in the queue after checking all the nodes at each level i
The queue is not empty when deleting the queue head
When the deleted element is -1, check if the queue is empty
If it is empty, the solution is found
Else add a tail flag -1 into the queue and check active nodes at the next level

22. CLP -- FIFO while (true) { // check left child
if (Ew + w[i] <= c) // x[i] = 1
EnQueue(Q, Ew + w[i], bestw, i, n);
// right child is always feasible
EnQueue(Q, Ew, bestw, i, n); // x[i] = 0
Q.Delete(Ew); // pick next expanding node
if (Ew == -1) { // arrive at the tail of nodes at the same level
if (Q.IsEmpty())
return bestw;
Q.Add(-1); // flag of the tail of nodes at the same level
Q.Delete(Ew); // pick next expanding node
i++;} // explore the next level of solution space tree }

23. prune subtree rooted at right child
Improvement
prune subtree rooted at right child
Update bestw earlier
// check left child
Type wt = Ew + w[i];// the weight of left child
if (wt <= c) {// feasible node
if (wt > bestw) bestw = wt;
// insert into active node list
if (i < n) Q.Add(wt); }
// check right child
if (Ew + r > bestw && i < n)
Q.Add(Ew); //optimal solution may be //in this subtree
Q.Delete(Ew);// pick next expanding node

24. Construct Optimal Solution
The path from the active node to the root should be stored
Each node should have the pointer to its parent and flag of left or right child
class QNode
{QNode *parent; // pointer to parent
bool LChild; // flag of left child
Type weight; // corresponding load at this node

25. Construct Optimal Solution
When the optimal value is found, the optimal solution can be obtained by backtracking to the root according to the parent pointers stored in each node on the path
for (int j = n - 1; j > 0; j--) {
bestx[j] = bestE->LChild;
bestE = bestE->parent; }

26. CLP – Priority Queue MaxHeap is used to store active node list
Bound/priority function:
Ew: the current load of the ship
r: the weight of remaining containers
bestw: the currently optimal value
uweight: the upper bound of potential load in the ship
uweight = Ew + r > bestw
The node with maximal priority in the queue is picked as the next expanding node
The uweight of any node in the subtree rooted at node x is not larger than that of x
The load of a leaf is the same as its priority and thus when a leaf is the expanding node, the optimal solution is found

27. 0-1 Knapsack Problem -- Review
Given a set of n items, each with a weight wi and a value vi, and a knapsack with a capacity of C. Determine the number of each item to include in the knapsack so that the total weight is less than or equal to the capacity and the total value is as large as possible
A special integer programming problem
(a) 4 items and the knapsack v:
v/w: 4 7 5 10 3

28. 0-1 Knapsack
Sort items in non-increasing order in terms of their values per weight
Solution X=(x1, x2, …, xn), xi∊{0, 1} denotes item i is chosen or not in the knapsack
Solution space: subset tree
Constraint: the inclusion of item i does not exceed the knapsack capacity
Bound:
wt: the current weight of items in the knapsack
cp: the current value of items in the knapsack
bestp: the currently optimal value of feasible solution
up: the upper bound of the knapsack value for a subtree
gp: value produced by greedy ALG for remaining items
Bound function/priority, up = cp + gp > bestp
MaxHeap for the priority queue
If an expanding node is a leaf, the optimal solution is found

29. 0-1 Knapsack Bound Function // n: the # of items，cleft: left capacity
while (i <= n && w[i] <= cleft) {
cleft -= w[i]; //w[i]: the weight of item i
b += p[i]; //p[i]: the value of item i
i++; }
if (i <= n) b+=p[i]/w[i] * cleft; // fill up the knapsack
return b; //b: the bound value

30. 0-1 Knapsack while (i != n+1) { // nonleaf node
Search process of B & B
while (i != n+1) { // nonleaf node
// check the left child of the expanding node
Typew wt = cw + w[i];
if (wt <= c) { // the left child is a feasible node
if (cp+p[i] > bestp) bestp = cp+p[i];
AddLiveNode(up, cp+p[i], cw+w[i], true, i+1);}
up = Bound(i+1);
// check the right child of the expanding node
if (up >= bestp) // the optimal solution may be in the right subtree
AddLiveNode(up, cp, cw, false, i+1);
// pick the next expanding node (omitting)}

31. 0-1 Knapsack – B & B Priority Queue
Sort items in non-increasing order in terms of their values per weight
Solution X=(x1, x2, …, xn), xi∊{0, 1} denotes item i is chosen or not in the knapsack
Solution space: subset tree
Constraint: the inclusion of item i does not exceed the knapsack capacity
Bound/priority:
wt: the current weight of items in the knapsack
cp: the current value of items in the knapsack
bestp: the currently optimal value of feasible solution
up: the upper bound of the knapsack value for a subtree
gp: value produced by greedy ALG for remaining capacity & items
Bound function/priority, up = cp + gp > bestp
MaxHeap for the priority queue
If an expanding node is a leaf, the optimal solution is found

32. (a) 4 items and the knapsackv:
v/w: 4 7 5 10 3
B & B A
wt=0,cp=0, P=76, O=0 1 B C
0, 0, 57, 40
4, 40, 76, 40 1 1 D E
4, 40, 69, 40 F G 1 1 1 1
4, 40,
52, 65
9, 65,
69, 65 H I J K L M N O 1 1 1 1 1 1 1 1 P Q R S T U V W X Y Z A’ D’ B’ C’ E’
9, 65,
69, 65
Leaf node, optimal solution is found

33. Maximum Clique Problem -- Review
Given a undirected graph G=(V, E)
Complete subgraph
Clique
Maximum Clique (MC) 1 2 4 5 3
Graph G

34. MCP
Solution X=(x1, x2, …, xn), xi∊{0, 1} denotes vertex i is chosen or not in the clique
Solution space: subset tree
Constraint: vertex i is connected to any chosen vertex
Bound/priority:
bestn: the # of vertices in currently found maximal clique
cn: the # of chosen vertices in current clique
n-i: the # of unchecked vertices
Bound function/priority, the upper bound of the # of vertices in current clique un: cn + (n - i) < bestn
MaxHeap for the priority queue

35. A weighted graph of 4 vertices
TSP -- Review
A Salesman wishes to travel around a given set of cities, and return to the beginning, covering the smallest total cost
Basic idea
Let G=(V, E) be a weighted graph and its weights are positive
A traveling route is a cycle that includes all vertices in G
The cost of a traveling route is the sum of its edge weights
Find the traveling route with minimal cost 1 4 3 2 6 5 20 10 30
A weighted graph of 4 vertices
Permutation Tree A B E D C K J I H G F Q P O N M L

36. TSP – B & B Priority Queue
Solution X=(x1, x2, …, xn), xi ∊{1, 2, …, n}
Solution space tree: permutation tree
Constraint: when an expanding node is producing a child of choosing the ith vertex, the vertex should be connected to the (i-1)th vertex on the chosen path
If the expanding node is a parent of a leaf, the chosen vertex should also connected to vertex x1
Bound/Priority function:
cc: current cost
rcost: the sum of minimal outgoing edges of remaining vertices and the current vertex
lcost: lower bound of the potential route in the subtree rooted at a node lcost = cc + rcost
bestc: the cost of currently minimal path
MinHeap for the priority queue

37. TSP – B & B Priority Queue1 4 3 2 6 5 20 10 30
A weighted graph of 4 vertices A 1 B
cc=0, rcost=18, lcost=18 2 3 4
30, 14, 44 C
6, 14, 20 D
4, 14, 18 E 3 4 2 4 2 3
14, 10, 24
cost 1 2 3 4 30 6 5 10 20
24, 10, 34 F G H I J K
11, 9, 20
26, 9, 35 4 3 4 2 3 2 L M N O P Q
25, 0, 25
25, 0, 25
bestc：25
bestc：NoEdge