1. Analysis & Design of Algorithms (CSCE 321)
Prof. Amr Goneid
Department of Computer Science, AUC
Part 11. Backtracking Algorithms
Prof. Amr Goneid, AUC

2. Backtracking Algorithms
Prof. Amr Goneid, AUC

3. Backtracking Algorithms
Problem Statement
Brute Force & Backtracking
Permutation Tree
The 4-Queens Problem
Sum of Subsets
Unbound DFG: Permutation Tree
Bound DFG: Backtracking
General Backtracking Algorithm
Prof. Amr Goneid, AUC

4. Backtracking Algorithms
Example: n-Binary Variables Problem
The n-Queens Problem
Another Backtracking Schema
The Hamiltonian Circuit Problem by Backtracking
Prof. Amr Goneid, AUC

5. 1. Problem Statement
Given sets S1, S2, …, Sn of values x with m1, m2, …, mn values in the sets.
We seek a solution vector (n-tuple)
X = (x1, x2, …, xn)
chosen from the sets out of
m = m1 m2 ….mn possible candidates
satisfying a criterion function P(X)
Prof. Amr Goneid, AUC

6. 2. Brute Force & Backtracking
Sometimes the best algorithm for a problem is to try all possibilities.
This is always slow, but straightforward.
Brute Force (Exhaustive Search) Methods:
Will try all possible m trials and save those satisfying P(X)
Prof. Amr Goneid, AUC

7. Brute Force & Backtracking
Yields same answer with fewer than m trials.
Builds solution vector one component at a time.
Examines if a partial vector Xi = (x1,x2,..,xi) has a chance of success.
If it does not satisfy constraints, we drop out the k remaining trials, k = mi+1 mi+2 …mn
Prof. Amr Goneid, AUC

8. The 8-Queens Problem
Place 8 Queens on an 8 x 8 chessboard such that no two can attack each other. There are about 4.4 billion trials.
Obvious: Each queen must be on a different row. All solutions are 8-tuples
X = (x1,x2,..,x8) , where xi is the column number of the ith queen.
Brute Force Trials: m = 8*8*….*8 = 88 = 224 = 16 M
Constraint(1) : Each queen must be on a different column. This reduces number of trials to 8! = 40,320
Constraint(2): No two queens can be on the same diagonal.
Prof. Amr Goneid, AUC

9. The 8-Queens Problem (continued)
Backtracking:
will use only 1625 trials
to achieve solution.
Possible Solution:
X = (4 , 6 , 8 , 2 , 7 , 1 , 3 , 5) Q
Prof. Amr Goneid, AUC

10. 3. Permutation Tree Represents the problem to be solved.
Problem State: A node in the tree.
Layer: between two consecutive levels and represents one variable in the tuple. Edges from level (i) to level (i+1) are labeled with values of variable xi.
State Space: All paths from root to other nodes.
Solution States: Nodes from the root defining a tuple.
Answer States: Solution states satisfying implicit conditions.
Node Generation: Depth First
Prof. Amr Goneid, AUC

11. Example
Problem:
Find all sequences of three binary variables such that no two consecutive values are the same.
S1 = S2 = S3 = {0,1} , m1 = m2 = m3 = 2
n = 3 , xi = 0 or 1 , i = 1 , 2 , 3
Total number of brute force trials = m1m2m3 =
2 * 2 * 2 = 8
Implicit Conditions:
x1  x2 , x2  x3
Prof. Amr Goneid, AUC

12. Brute Force Permutation Tree1 x1 1 2 9 1 x2 1 3 6 10 13 1 x3 1 1 1 4 7 8 11 12 5 14 15
{0,1,0)
{1,0,1)
Prof. Amr Goneid, AUC

13. Backtracking Method Generate nodes in Depth First order.
Kill nodes (and their children) not satisfying constraints.
Backtrack to higher level to seek a different path to a leaf node.
If all leaves are killed, the problem has no solution.
Prof. Amr Goneid, AUC

14. Bactracking Permutation Tree1 x1 1 2 9 1 x2 1 3 6 10 13 x3 1 1 1 1 4 7 8 11 12 5 14 15
{0,1,0)
{1,0,1)
Prof. Amr Goneid, AUC

15. The Solution # of solution states = # of leaves = 8
Total # of nodes in tree = measure of brute force cost = 15
# of nodes generated = measure of Backtracking cost = 11
# of surviving nodes = 7
# of killed parent nodes = 4
# of answer states (3-tuples) = 2
X1 = {0,1,0} , X2 = {1,0,1}
Prof. Amr Goneid, AUC

16. Conclusion
Backtracking is a Divide & Conquer Brute Force (exhaustive) search with pruning.
Saves time by killing internal nodes that have no useful leaves.
Let Ne and Nb be the number of nodes generated by the exhaustive search and backtracking methods, respectively.
A measure of the Gain obtained by backtracking is
G = (1 – Nb / Ne)*100 %
For the previous example, G = (1 – 11/15)*100 = 26.7%
Prof. Amr Goneid, AUC

17. 4. The 4-Queens Problem (a) The Problem
Place 4 Queens on an 4 x 4 chessboard such that no two can attack each other.
With no constraints, there are
16 * 15 * 14 * 13 = 43,680 possible placements.
Constraint (1): Each Queen must be on a different row. Now we seek the 4-tuples
X = {x1 , x2 , x3 , x4} representing column numbers. There are 4*4*4*4 = 256 such tuples.
Prof. Amr Goneid, AUC

18. The Problem (cont.)
Constraint(2): Each Queen must be on a different column.
The number of possible tuples reduces to 4*3*2*1 = 4! = 24
Backtracking will be used to impose the final No Attack constraint so that no two queens can be placed on the same diagonal
Prof. Amr Goneid, AUC

19. (b) The Permutation Tree
The root will have 4 children
In general, each node in level L will have
(4-L+1) children. The total # of nodes in the tree will be
*3 + 4*3*2 + 4*3*2*1 = 65 nodes
The # of leaves will be 4! = 24 tuples. Which of these will be answers ?
Prof. Amr Goneid, AUC

20. (c) Portion of the Permutation Tree1 2 1 2 18 4 2 4 1 3 3 3 8 13 19 24 29 4 2 3 2 1 9 11 14 16 30 3 3 15 31
X = {2,4,1,3}
Prof. Amr Goneid, AUC

21. (d) Solutions { 2 , 4 , 1 , 3 } { 3 , 1 , 4 , 2 } Q Q Leaf 31 Leaf 39
Prof. Amr Goneid, AUC

22. (e) Performance Brute Force cost = 65 Backtracking:
# nodes generated = 32
# nodes surviving = 18
# parent nodes killed = 14
Gain G = (1-32/65)*100 = 50.8%
Prof. Amr Goneid, AUC

23. 5. Sum of Subsets (a) The Problem
Given a set W of positive integers wi ,
i = 1,2,…,n and m, find all subsets of W whose sums are equal to m.
Example: n = 4 , W = {11,13,24,7},
m = 31
There are 2 possible subsets:
S1 = {11,13,7} , S2 = {24,7}
Prof. Amr Goneid, AUC

24. The Problem (cont.) Constraints:
1. A member appears only once in the subset.
2. The sum of a subset is exactly m.
3. No multiple instances of the same subset. For example:
{1,4,2} and {1,2,4}
This is satisfied by requiring wi < w i+1
Prof. Amr Goneid, AUC

25. (b) Backtracking Solution
Consider the previous example with
n = 4 ,
W = {11,13,24,7}, m = 31
Let X = {x1,x2,..,xn} be a solution such that xi  {0,1} ,
xi = 1 if wi is selected and xi = 0 otherwise. Hence, we obtain fixed-size tuples.
Prof. Amr Goneid, AUC

26. Brute Force Permutation Tree
The permutation tree will be a complete binary tree with a height of n+1. The # of leaves (possible subsets) will be
2n = 16 and the total # of nodes will be 31.
Prof. Amr Goneid, AUC

27. (c) Portion of Permutation Tree1 1 2 17 1 3 10 1 1 4 7 11 14 1 1 5 6 8 9 12 13 15 16
Prof. Amr Goneid, AUC

28. Performance # nodes generated = 25 # nodes survived = 14 # killed = 11
Two possible solutions X1 = {1,1,0,1} and X2 = {0,0,1,1}
Prof. Amr Goneid, AUC

29. 6. Unbound DFG: Permutation Tree
The full (Brute Force) permutation tree is generated by an unbound Depth-First Generation (DFG) algorithm
Example:
Binary strings of length (n) bits.
The DFG algorithm generates a full permutation tree of n+1 levels
Prof. Amr Goneid, AUC

30. Unbound DFG: Algorithm
// Assume there is a root node
void Unbound_DFG(int k, int n) {
for (i = 0; i <= 1; i++) {
Generate edge (i) from current parent;
Generate child node;
if (k == n) then leaf node has been reached;
else Unbound_DFG(k+1, n); }
Prof. Amr Goneid, AUC

31. Example: All 2-bit strings1 x1 1 2 5 x2 1 1 3 4 6 7
{00}
{01}
{10}
{11}
n = 2; Unbound_DFG(1,n)
Prof. Amr Goneid, AUC

32. 7. Bound DFG: Backtracking
A partial permutation tree is generated by a bound Depth-First Generation (DFG) algorithm
Example:
n-bit binary strings, with a bounding function B(k,i) == true
The DFG algorithm generates a partial permutation tree of n+1 levels
Prof. Amr Goneid, AUC

33. Bound DFG: Algorithm // Assume there is a root node
void Bound_DFG(int k, int n) {
for (i = 0; i <= 1; i++) {
if ( B( k , i ) ){
Generate edge (i) from current parent;
Generate child node;
if (k == n) then leaf node has been reached;
else Bound_DFG(k+1, n); }
Prof. Amr Goneid, AUC

34. Example: n-bit binary strings1 x1 1 2 5 x2 3 6
{00}
{10}
n = 2; Bound: 2nd bit should not be 1
B’(k,i) = (k==2) && (i==1)
Hence B(k,i) = (k == 1) || (i == 0) Invoke as Bound_DFG(1,n)
Prof. Amr Goneid, AUC

35. 8. General Backtracking Algorithm
Consider:
n = no. of variables
k = index of a variable
x1 , x2, .. ,xk, .. xn is a path to a solution state.
m = no. of possible values for x
vi = a value for a variable, i = 1, 2, .. m
B(k,i) = Bounding Function
= true if xk can take the value vi
Prof. Amr Goneid, AUC

36. Brute Force Algorithm void Brute_Force (int k, int n) {
for all possible values vi of a variable (i = 1..m)
Let variable xk take the value vi ;
if (xk is the last variable )
output vector x[1:n];
else Brute_Force (k+1, n); }
Prof. Amr Goneid, AUC

37. General Brute Force Code
void Brute_Force (int k, int n) {
for (i = 1; i <= m; i++)
x[k] = v[i] ;
if (k == n) output vector x[1:n];
else Brute_Force (k+1, n); }
Prof. Amr Goneid, AUC

38. Backtracking Algorithm
void Backtrack(int k, int n) {
for (i = 1; i <= m; i++) {
if ( B( k , i )) { // Pruning
x[k] = v[i] ;
if (k == n) output vector x[1:n];
else Backtrack(k+1, n); }
Prof. Amr Goneid, AUC

39. 9. Example: n-Binary bits Problem
Find all strings of n-binary bits such that no two consecutive bits are the same.
n given , m = 2 , xk = 0 or 1 , k = 1 , 2 , .. , n
Bounding Function:
Assume x0 = -1 then x1  x2 , x2  x3 , ……
or if (xk-1  i) then xk can take the value (i)
Prof. Amr Goneid, AUC

40. Backtracking Algorithm
void Backtrack(int k, int n) {
for (i = 0; i <= 1; i++) {
if ( x[k-1] != i) { // assume x[0] = -1
x[k] = i ;
if (k == n) output vector x[1:n];
else Backtrack(k+1, n); }
Prof. Amr Goneid, AUC

41. Permutation Tree for n = 31 x1 1 2 9 1 x2 1 3 6 10 13 1 1 x3 7 8 11 12
{0,1,0)
{1,0,1)
Prof. Amr Goneid, AUC

42. 10. The n-Queens Problem (a) The Problem
Find all possible arrangements of n Queens on an n x n chessboard such that no two can attack each other.
Example:
8 queens on an 8x8 board
The problem has 92 solutions.
Only 12 are unique, others are
reflections or rotations.
Prof. Amr Goneid, AUC

43. The n-Queens Problem Pre-Condition: Each Queen is on a different row.
We seek the n-tuples
X = {x1 , x2 , … , xn} representing column numbers satisfying the problem.
Hence, we seek all permutations of X
Prof. Amr Goneid, AUC

44. (b) The Bounding Function
Assume that Queens 1, 2, .. K-1 have already been properly placed.
The bounding function for Queen (k) is that it can be placed in column (i) iff:
It does not share the same column (i) with any of the previous queens (j), j = 1,2,.., k-1, i.e Xj  i
It is not on the same diagonal with any of the
previous queens:
│k-j│  │Xj - i│ , j = 1,2,.., k-1 Qj
Row j
Row k Qk
Col X[ j]
Col (i)
Prof. Amr Goneid, AUC

45. The Bounding Function bool can_place (int k, int i) {
for (int j = 1; j < k; j++)
if ((x[ j ] == i) || abs(x[ j ] - i) == abs(k-j))
return false;
return true; }
Prof. Amr Goneid, AUC

46. (c)Backtracking Algorithm
void NQueens(int k, int n) {
for (i = 1; i <= n; i++) {
if ( can_place( k , i ) ){
x[k] = i ;
if (k == n) output vector x[1:n];
else NQueens(k+1, n); }
Prof. Amr Goneid, AUC

47. The 8-Queens Problem Animation
Prof. Amr Goneid, AUC

48. 11. Another Backtracking Schema
Another schema exists when we replace the bounding function by a function that assigns to x[k] a legal value after x[1: k-1] have already been assigned legal values out of m possible values. If no such value is available, the algorithm backtracks.
Assume the function to be
NextValue (k , n) and the vector x[1:n] is initially set to zero.
Prof. Amr Goneid, AUC

49. The Algorithm void Backtrack2(int k, int n) { do {
NextValue(k,n); // Assign to x[k]
// a legal value
if( !x[k] ) break ; // No possible value
if (k == n) output vector x[1:n];
else Backtrack2(k+1, n);
} while(1); }
Prof. Amr Goneid, AUC

50. 12. Hamiltonian Circuit Problem by Backtracking
The Knight’s Tour problem is
a Hamiltonian circuit problem
Prof. Amr Goneid, AUC