1. Backtracking Algorithm
Level II, Term ii
CSE – 243
Md. Monjur-ul-hasan
Lecturer
Dept of cse, cuet

2. An Example
Find out all 3-bit binary numbers for which the sum of the 1's is greater than or equal to 2.
The only way to solve this problem is to check all the possibilities: (000, 001, 010, ....,111)
The 8 possibilities are called the search space of the problem. They can be organized into a tree.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

3. An Example (cont…) 0 0 _ 0 0 0 0 0 1 0 1 _ 0 1 0 0 1 1 1 _ _ 0 _ _
1 0 _
1 0 0
1 0 1
1 1 _
1 1 0
1 1 1
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

4. Backtracking: Definition
Start
Success!
Success!
Failure
For some problems, the only way to solve is to check all possibilities.
Backtracking is a systematic way to go through all the possible configurations of a search space.
Problem space consists of states (nodes) and actions (paths that lead to new states). When in a node can only see paths to connected nodes
If a node only leads to failure go back to its "parent“ node. Try other alternatives. If these all lead to failure then more backtracking may be necessary.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

5. Goals of Backtracking Possible goals
Find a path to success
Find all paths to success
Find the best path to success
Not all problems are exactly alike, and finding one success node may not be the end of the search
Start
Success!
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

6. Backtracking and Recursive
Backtracking is easily implemented with recursion because:
The run-time stack takes care of keeping track of the choices that got us to a given point.
Upon failure we can get to the previous choice simply by returning a failure code from the recursive call.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

7. Improving Backtracking: Search Pruning
Search pruning will help us to reduce the search space and hence get a solution faster.
The idea is to a void those paths that may not lead to a solutions as early as possible by finding contradictions so that we can backtrack immediately without the need to build a hopeless solution vector.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

8. A More Concrete Example: Suduku
9 by 9 matrix with some numbers filled in
all numbers must be between 1 and 9
Goal: Each row, each column, and each mini matrix must contain the numbers between 1 and 9 once each
no duplicates in rows, columns, or mini matrices 8
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

9. Suduku with Brut force
A brute force algorithm is a simple but general approach
Try all combinations until you find one that works
This approach isn’t clever, but computers are fast
Then try and improve on the brute force resuts
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

10. Sodoku (Cont…) Brute force Sudoku Soluton
After placing a number in a cell is the remaining problem very similar to the original problem? 1
Brute force Sudoku Soluton
if not open cells, solved
scan cells from left to right, top to bottom for first open cell
When an open cell is found start cycling through digits 1 to 9.
When a digit is placed check that the set up is legal
now solve the board
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

11. Solving Sudoku – Later Steps1 2 4 8 9
uh oh!
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

12. Solving Sudoku – Dead End
We have reached a dead end in our search
With the current set up none of the nine digits work in the top right corner 1 2 4 8 9
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

13. Solving Suduku - Backing Up1 2 4 8 9
When the search reaches a dead end in backs up to the previous cell it was trying to fill and goes onto to the next digit
We would back up to the cell with a 9 and that turns out to be a dead end as well so we back up again
so the algorithm needs to remember what digit to try next
Now in the cell with the 8. We try and 9 and move forward again. 1 2 4 9
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

14. Problem with Suduku Brute force algorithms are slow
The don't employ a lot of logic
For example we know a 6 can't go in the last 3 columns of the first row, but the brute force algorithm will plow ahead any way
But, brute force algorithms are fairly easy to implement as a first pass solution
backtracking is a form of a brute force algorithm
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

15. Solving Suduku-Search pruning
After trying placing a digit in a cell we want to solve the new sudoku board
Isn't that a smaller (or simpler version) of the same problem we started with?!?!?!?
After placing a number in a cell the we need to remember the next number to try in case things don't work out.
We need to know if things worked out (found a solution) or they didn't, and if they didn't try the next number
If we try all numbers and none of them work in our cell we need to report back that things didn't work
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

16. Solving Suduku-Recursive Backtracking
Problems such as Suduko can be solved using recursive backtracking
recursive because later versions of the problem are just slightly simpler versions of the original
backtracking because we may have to try different alternatives
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

17. General Back Tracking Algorithm
void checknode (node v)
{ if (promising ( v )) if (aSolutionAt( v )) write the solution else //expand the node for ( each child u of v ) checknode ( u ) }
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

18. General Back Tracking Algorithm (Cont…)
Checknode uses the functions:
promising(v) which checks that the partial solution represented by v can lead to the required solution
aSolutionAt(v) which checks whether the partial solution represented by node v solves the problem.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

19. Eight Queen Problem: A Classic Puzzle
Attempts to place 8 queens on a chessboard in such a way that no queen can attack any other.
A queen can attack another queen if it exists in the same row, columm or diagonal as the queen.
This problem can be solved by trying to place the first queen, then the second queen so that it cannot attack the first, and then the third so that it is not conflicting with previously placed queens.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

20. N Queen Problem (N=8)
Place N Queens on an N by N chessboard so that none of them can attack each other
Number of possible placements?
In 8 x 8
64 * 63 * 62 * 61 * 60 * 58 * 59 * 57 = 4,426,165,368
roughly four and a half billion possible set ups
n choose k
How many ways can you choose k things from a
set of n items?
In this case there are 64 squares and we want to choose 8 of them to put queens on
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

21. Solution: N Queen Puzzle
We know that for queens:
each row will have exactly one queen
each column will have exactly one queen
each diagonal will have at most one queen
The previous calculation includes set ups like following:
Includes lots of set ups with multiple queens in the same column
Number of set ups 8 * 8 * 8 * 8 * 8 * 8 * 8 * 8 = 16,777,216
We have reduced search space by two orders of magnitude by applying some logic
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

22. N Queen Solution: Chess board Store
The previous calculation help us to model the chessboard not as a 2-D array, but as a set of rows, columns and diagonals.
To simplify the presentation, we will study for smaller chessboard, 4 by 4
First: we need to define an array to store the location of so far placed queens
PositionInRow 1 3 2
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

23. N Queen Solution: Chess board Store (Cont..)
We need an array to keep track of the availability status of the column when we assign queens.
Suppose that we
have placed two
queens F T
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

24. N Queen Solution: Chess board Store (Cont..)
We have 7 left diagonals, we want to keep track of available diagonals after the so far allocated queens T F
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

25. N Queen Solution: Chess board Store (Cont..)
We have 7 left diagonals, we want to keep track of available diagonals after the so far allocated queens T F
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

26. N Queen Solution: Chess board Store (Cont..)
Now Run a loop from 1 to 4 with counter i and place a Queen to safe position of the ith column.
Now try to make a solution for N Queen.
Try to find all possible solution.
[4,3]
[3,1]
[2,4]
[1,2]
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

27. N Queen Puzzle: Algorithm
putQueen(int row) {
for col=1 to N
if (column[col]==available && leftDiagonal[row+col]==available &&
rightDiagonal[row-col+norm]== available)
positionInRow[row]=col;
column[col]=!available;
leftDiagonal[row+col]=!available;
rightDiagonal[row-col+norm]=!available;
if (row< squares-1)
putQueen(row+1);
else
PRINT solution found
column[col]=available;
leftDiagonal[row+col]=available;
rightDiagonal[row-col+norm]= available; }
Base
Update
Forward Recursi.
Back Track
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

28. Backtracking Algorithm for optimization problems
Procedure checknode (node v ) {
node u ; if ( value(v) is better than best ) best = value(v); if (promising (v) ) for (each child u of v) checknode (u ); }
best is the best value so far and is initialized to a value that is equal or worse than any possible solution.
value(v) is the value of the solution at the node.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

29. Backtracking for optimization problems (Cont…)
To deal with optimization we compute:
best - value of best solution achieved so far
value(v) - the value of the solution at node v
Modify promising(v)
Best is initialized to a value that is equal to a candidate solution or worse than any possible solution.
Best is updated to value(v) if the solution at v is “better”
By “better” we mean:
larger in the case of maximization and
smaller in the case of minimization
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

30. Modifying promising A node is promising when
it is feasible and can lead to a feasible solution and
“there is a chance that a better solution than best can be achieved by expanding it”
Otherwise it is nonpromising
A bound on the best solution that can be achieved by expanding the node is computed and compared to best
If the bound > best for maximization, (< best for minimization) the node is promising
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

31. Modifying promising for Maximization Problems
For a maximization problem the bound is an upper bound,
the largest possible solution that can be achieved by expanding the node is less or equal to the upper bound
If upper bound > best so far, a better solution may be found by expanding the node and the feasible node is promising
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

32. Modifying promising for Minimization Problems
For minimization the bound is a lower bound,
the smallest possible solution that can be achieved by expanding the node is less or equal to the lower bound
If lower bound < best a better solution may be found and the feasible node is promising
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

33. Optimization Example:0-1 Knapsack Problem
The 0-1 knapsack problem:
N items which cost p1,p2,p3….pn and have weight w1, w2,w3….wn. (where the i-th item is worth pi dollars and weight wi pounds.)
vi and wi are integers.
A thief can carry at most C (integer) pounds.
How to take as valuable a load as possible.
An item cannot be divided into pieces.
The fractional knapsack problem:
The same setting, but the thief can take fractions of items.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

34. Knapsack Problem: In Mathematical Term
Input description:
A set of items 1n, where item i has size wi and value pi, and a knapsack capacity C
Problem description:
Find a subset S that maximizes given that
i.e. all the items fit in a knapsack of size C
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

35. 0-1 Knapsack - Example E.g.1: E.g.2: 50 50 50 Item: 1 2 3 4 5 6 720
$100
$120 +
$220 30
Item 3 30 20
Item 2
$100 + 20
Item 1 10 10
$60
$60
$100
$120 W
$160
$6/pound
$5/pound
$4/pound
E.g.2:
Item:
Benefit:
Weight:
Knapsack holds a maximum of 22 pounds
Fill it to get the maximum benefit
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

36. Brute force tree A B C D E B C D E Weight = 15.12 Value = $315In
Out B C D E B C D E
Weight = 15.12
Value = $315
Weight = 8.98
Value = $235
Weight = 9.98
Value = $225
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

37. 0-1 Knapsack Problem: Brute force Algorithm
Let’s first solve this problem with a straightforward algorithm
Since there are n items, there are 2n possible combinations of items.
We go through all combinations and find the one with the most total value and with total weight less or equal to W
Running time will be O(2n)
In the tree representation, we can think of the previous algorithm doing a DFS in the tree
If we reach a point where a solution no longer is feasible, there is no need to continue exploring
We can “backtrack” from this point and potentially cut off much of the tree and many of the solutions
We can backtrack if we know the best possible solution in current subtree is worse than current best solution obtained so far.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

38. 0-1 Knapsack: Algorithm
Inputs: Positive integers n and W; arrays w and p, each indexed from 1 to n, and each containing positive integers sorted in nonincreasing order according to the values of p[i]/w[i].
Outputs: an array bestset indexed from 1 to n, where the values of bestset[i] is "yes" if the ith item is included in the optimal set and is "no" otherwise; an integer maxprofit that is the maximum profit.
void knapsack (index i, int profit, int weight)
{ if (weight <= W&& profit > maxprofit){ // This set is best
maxprofit = profit; // so far.
numbest = i; // Set numbest to
bestset = include; // number of items
} // considered. Set
// bestset to this
// solution.
if (promising(i)){
include [i + 1] = "yes"; // Include w[i + 1].
knapsack(i + 1, profit + p[i + 1], weight + w[i + 1]);
include [i + 1] = "no"; // Do not include
knapsack (i + 1, profit, weight); // w[i + 1]. }
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

39. 0-1 Knapsack: Algorithm (Cont…)
bool promising (index i) {
index j, k;
int totweight;
float bound;
  if (weight >= W) // Node is promising only
return false; // if we should expand to
else { // its children. There must
j = i + 1; // be some capacity left for
bound = profit; // the children.
totweight = weight;
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

40. 0-1 Knapsack: Algorithm (Cont…)
while (j <= n && totweight + w[j] < = W){
// Grab as many items as
totweight = totweight + w[j]; // possible.
bound = bound + p[j];
j++; }
k = j; // Use k for consistency
if (k <=n) // with formula in text.
bound = bound + (W - totweight) * p[k]/w[k];
// Grab fraction of kth
return bound > maxprofit; // item.
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

41. 0-1 Knapsack: Implementation (Cont…)
numbest = 0;
maxprofit = 0;
knapsack(0, 0, 0);
print maxprofit; // Write the maximum profit.
for (j = 1; j <= numbest; j++) // Show an optimal set of items.
print bestset[i];
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

42. Subset finding problem
We say that
running time of an alg. is O(f(n))
If there exists c such that for large enough n:
RunningTime(n) < c·f(n)
Translation: From some point c·f(n) is an upper bound on the running time.
Why only for “large enough n”?
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET

43. Other Problems
Do not assume that because we analyze running time asymptotically we no longer care about the constants.
If you have two algorithms with different asymptotic limits – easy to choose.
(may be misleading if constants are really large).
Usually we are happy when something runs in polynomial time and not exponential time. O(n2) is much much better than O(2n).
Md. Monjur-ul-hasan. Lecturer, Dept. of CSE CUET