Analysis and design of algorithm Unit-4 Branch and Bound Analysis and design of algorithm

Outline Least Cost (LC) searching bounding-Concept First In First Out (FIFO) branch and bound Least Cost (LC) branch and bound application - 0/1 Knapsack problem.

Overview Branch and bound is a systematic method for solving optimization problems B&B is a rather general optimization technique that applies where the greedy method and dynamic programming fail. However, it is much slower. Indeed, it often leads to exponential time complexities in the worst case. On the other hand, if applied carefully, it can lead to algorithms that run reasonably fast on average.

Branch and Bound The general idea of B&B is a BFS-like search for the optimal solution, but not all nodes get expanded (i.e., their children generated). Rather, a carefully selected criterion determines which node to expand and when, and another criterion tells the algorithm when an optimal solution has been found. A branch-and-bound algorithm searches the entire space of candidate solutions, with one extra trick: it throws out large parts of the search space by using previous estimates on the quantity being optimized.

Steps in B&B The first one is a splitting procedure that, given a set of candidates, returns two or more smaller sets whose union covers . Note that the minimum of over is , where each is the minimum of within .This step is called branching, since its recursive application defines a search tree whose nodes are the subsets. The second tool is a procedure that computes upper and lower bounds for the minimum value of within a given subset of . This step is called bounding.

Branch and Bound is the generalisation of both graph search strategies, BFS and DFS search.  A BFS like state space search is called as FIFO (First in first out) search as the list of live nodes in a first in first out list (or queue).  A D search like state space search is called as LIFO (Last in first out) search as the list of live nodes in a last in first out (or stack).

Terminology Live node is a node that has been generated but whose children have not yet been generated. E -node (Expanded Node) is a live node whose children are currently being explored. In other words, an E-node is a node currently being expanded. Dead node is a generated node that is not to be expanded or explored any further. All children of a dead node have already been expanded. Branch-and-bound refers to all state space search methods in which all children of an E-node are generated before any other live node can become the E-node.

Search Strategies We can use 3-types of search strategies in branch and bound 1) FIFO (First In First Out) search 2) LIFO (Last In First Out) search 3) LC (Least Count) search

FIFO B&B FIFO Branch & Bound is a BFS. In this, children of E-Node (or Live nodes) are inserted in a queue. Implementation of list of live nodes as a queue. Least() Removes the head of the Queue Add() Adds the node to the end of the Queue

Assume that node ‘12’ is an answer node in FIFO search, 1st we take E- node has ‘1’

LIFO B&B LIFO Brach & Bound is a D-search (or DFS). In this children of E-node (live nodes) are inserted in a stack. Implementation of List of live nodes as a stack Least() Removes the top of the stack ADD() Adds the node to the top of the stack

Assume that node ‘12’ is an answer node in FIFO search, 1st we take E- node has ‘1’

Least Cost (LC) Searching In both LIFO and FIFO Branch and Bound the selection rule for the next E-node in rigid and blind. The selection rule for the next E-node does not give any preference to a node that has a very good chance of getting the search to an answer node quickly. So we have another search strategy named “Least Cost Search”

General Idea The search for an answer node can be speeded by using an “intelligent” ranking function c() for live nodes. The next E-node is selected on the basis of this ranking function. The node x is assigned a rank using: c( x ) = f(h(x)) + g( x ) where, c( x ) is the cost of x. h(x) is the cost of reaching x from the root and f(.) is any non-decreasing function. g ( x ) is an estimate of the additional effort needed to reach an answer node from x.

A search strategy that uses a cost function c( x ) = f(h(x) + g( x ) to select the next E-node would always choose for its next E-node a live node with least LC–search (Least Cost search). BFS and D-search are special cases of LC-search. If g( x ) = 0 and f(h(x)) = level of node x, then an LC search generates nodes by levels. This is eventually the same as a BFS. If f(h(x)) = 0 and essentially a D-search. g( x ) > g( y ) whenever y is a child of x, then the search is essentially a D-search. An LC-search coupled with bounding functions is called an LC-branch and bound search. We associate a cost c(x) with each node x in the state space tree. It is not possible to easily compute the function c(x). So we compute a estimate c( x ) of c(x).

Application 0/1 Knapsack Problem Given two integer arrays val[0..n-1] and wt[0..n-1] that represent values and weights associated with n items respectively. Find out the maximum value subset of val[] such that sum of the weights of this subset is smaller than or equal to Knapsack capacity W.

Algorithm Sort all items in decreasing order of ratio of value per unit weight so that an upper bound can be computed using Greedy Approach. Initialize maximum profit, maxProfit = 0 Create an empty queue, Q. Create a dummy node of decision tree and enqueue it to Q. Profit and weight of dummy node are 0. Do following while Q is not empty. Extract an item from Q. Let the extracted item be u. Compute profit of next level node. If the profit is more than maxProfit, then update maxProfit. Compute bound of next level node. If bound is more than maxProfit, then add next level node to Q. Consider the case when next level node is not considered as part of solution and add a node to queue with level as next, but weight and profit without considering next level nodes.

Example: Input: // First thing in every pair is weight of item // and second thing is value of item Item arr[] = {{2, 40}, {3.14, 50}, {1.98, 100}, {5, 95}, {3, 30}}; Knapsack Capacity W = 10   Output: The maximum possible profit = 235