Analysis & Design of Algorithms (CSCE 321) Prof. Amr Goneid Department of Computer Science, AUC Part 11. Backtracking Algorithms Prof. Amr Goneid, AUC

Backtracking Algorithms Prof. Amr Goneid, AUC

Backtracking Algorithms Problem Statement Brute Force & Backtracking Permutation Tree The 4-Queens Problem Sum of Subsets Unbound DFG: Permutation Tree Bound DFG: Backtracking General Backtracking Algorithm Prof. Amr Goneid, AUC

Backtracking Algorithms Example: n-Binary Variables Problem The n-Queens Problem Another Backtracking Schema The Hamiltonian Circuit Problem by Backtracking Prof. Amr Goneid, AUC

1. Problem Statement Given sets S1, S2, …, Sn of values x with m1, m2, …, mn values in the sets. We seek a solution vector (n-tuple) X = (x1, x2, …, xn) chosen from the sets out of m = m1 m2 ….mn possible candidates satisfying a criterion function P(X) Prof. Amr Goneid, AUC

2. Brute Force & Backtracking Sometimes the best algorithm for a problem is to try all possibilities. This is always slow, but straightforward. Brute Force (Exhaustive Search) Methods: Will try all possible m trials and save those satisfying P(X) Prof. Amr Goneid, AUC

Brute Force & Backtracking Yields same answer with fewer than m trials. Builds solution vector one component at a time. Examines if a partial vector Xi = (x1,x2,..,xi) has a chance of success. If it does not satisfy constraints, we drop out the k remaining trials, k = mi+1 mi+2 …mn Prof. Amr Goneid, AUC

The 8-Queens Problem Place 8 Queens on an 8 x 8 chessboard such that no two can attack each other. There are about 4.4 billion trials. Obvious: Each queen must be on a different row. All solutions are 8-tuples X = (x1,x2,..,x8) , where xi is the column number of the ith queen. Brute Force Trials: m = 8*8*….*8 = 88 = 224 = 16 M Constraint(1) : Each queen must be on a different column. This reduces number of trials to 8! = 40,320 Constraint(2): No two queens can be on the same diagonal. Prof. Amr Goneid, AUC

The 8-Queens Problem (continued) Backtracking: will use only 1625 trials to achieve solution. Possible Solution: X = (4 , 6 , 8 , 2 , 7 , 1 , 3 , 5) Q Prof. Amr Goneid, AUC

3. Permutation Tree Represents the problem to be solved. Problem State: A node in the tree. Layer: between two consecutive levels and represents one variable in the tuple. Edges from level (i) to level (i+1) are labeled with values of variable xi. State Space: All paths from root to other nodes. Solution States: Nodes from the root defining a tuple. Answer States: Solution states satisfying implicit conditions. Node Generation: Depth First Prof. Amr Goneid, AUC

Example Problem: Find all sequences of three binary variables such that no two consecutive values are the same. S1 = S2 = S3 = {0,1} , m1 = m2 = m3 = 2 n = 3 , xi = 0 or 1 , i = 1 , 2 , 3 Total number of brute force trials = m1m2m3 = 2 * 2 * 2 = 8 Implicit Conditions: x1  x2 , x2  x3 Prof. Amr Goneid, AUC

Brute Force Permutation Tree 1 x1 1 2 9 1 x2 1 3 6 10 13 1 x3 1 1 1 4 7 8 11 12 5 14 15 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

Backtracking Method Generate nodes in Depth First order. Kill nodes (and their children) not satisfying constraints. Backtrack to higher level to seek a different path to a leaf node. If all leaves are killed, the problem has no solution. Prof. Amr Goneid, AUC

Bactracking Permutation Tree 1 x1 1 2 9 1 x2 1 3 6 10 13 x3 1 1 1 1 4 7 8 11 12 5 14 15 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

The Solution # of solution states = # of leaves = 8 Total # of nodes in tree = measure of brute force cost = 15 # of nodes generated = measure of Backtracking cost = 11 # of surviving nodes = 7 # of killed parent nodes = 4 # of answer states (3-tuples) = 2 X1 = {0,1,0} , X2 = {1,0,1} Prof. Amr Goneid, AUC

Conclusion Backtracking is a Divide & Conquer Brute Force (exhaustive) search with pruning. Saves time by killing internal nodes that have no useful leaves. Let Ne and Nb be the number of nodes generated by the exhaustive search and backtracking methods, respectively. A measure of the Gain obtained by backtracking is G = (1 – Nb / Ne)*100 % For the previous example, G = (1 – 11/15)*100 = 26.7% Prof. Amr Goneid, AUC

4. The 4-Queens Problem (a) The Problem Place 4 Queens on an 4 x 4 chessboard such that no two can attack each other. With no constraints, there are 16 * 15 * 14 * 13 = 43,680 possible placements. Constraint (1): Each Queen must be on a different row. Now we seek the 4-tuples X = {x1 , x2 , x3 , x4} representing column numbers. There are 4*4*4*4 = 256 such tuples. Prof. Amr Goneid, AUC

The Problem (cont.) Constraint(2): Each Queen must be on a different column. The number of possible tuples reduces to 4*3*2*1 = 4! = 24 Backtracking will be used to impose the final No Attack constraint so that no two queens can be placed on the same diagonal Prof. Amr Goneid, AUC

(b) The Permutation Tree The root will have 4 children In general, each node in level L will have (4-L+1) children. The total # of nodes in the tree will be 1 + 4 + 4*3 + 4*3*2 + 4*3*2*1 = 65 nodes The # of leaves will be 4! = 24 tuples. Which of these will be answers ? Prof. Amr Goneid, AUC

(c) Portion of the Permutation Tree 1 2 1 2 18 4 2 4 1 3 3 3 8 13 19 24 29 4 2 3 2 1 9 11 14 16 30 3 3 15 31 X = {2,4,1,3} Prof. Amr Goneid, AUC

(d) Solutions { 2 , 4 , 1 , 3 } { 3 , 1 , 4 , 2 } Q Q Leaf 31 Leaf 39 Prof. Amr Goneid, AUC

(e) Performance Brute Force cost = 65 Backtracking: # nodes generated = 32 # nodes surviving = 18 # parent nodes killed = 14 Gain G = (1-32/65)*100 = 50.8% Prof. Amr Goneid, AUC

5. Sum of Subsets (a) The Problem Given a set W of positive integers wi , i = 1,2,…,n and m, find all subsets of W whose sums are equal to m. Example: n = 4 , W = {11,13,24,7}, m = 31 There are 2 possible subsets: S1 = {11,13,7} , S2 = {24,7} Prof. Amr Goneid, AUC

The Problem (cont.) Constraints: 1. A member appears only once in the subset. 2. The sum of a subset is exactly m. 3. No multiple instances of the same subset. For example: {1,4,2} and {1,2,4} This is satisfied by requiring wi < w i+1 Prof. Amr Goneid, AUC

(b) Backtracking Solution Consider the previous example with n = 4 , W = {11,13,24,7}, m = 31 Let X = {x1,x2,..,xn} be a solution such that xi  {0,1} , xi = 1 if wi is selected and xi = 0 otherwise. Hence, we obtain fixed-size tuples. Prof. Amr Goneid, AUC

Brute Force Permutation Tree The permutation tree will be a complete binary tree with a height of n+1. The # of leaves (possible subsets) will be 2n = 16 and the total # of nodes will be 31. Prof. Amr Goneid, AUC

(c) Portion of Permutation Tree 1 1 2 17 1 3 10 1 1 4 7 11 14 1 1 5 6 8 9 12 13 15 16 Prof. Amr Goneid, AUC

Performance # nodes generated = 25 # nodes survived = 14 # killed = 11 Two possible solutions X1 = {1,1,0,1} and X2 = {0,0,1,1} Prof. Amr Goneid, AUC

6. Unbound DFG: Permutation Tree The full (Brute Force) permutation tree is generated by an unbound Depth-First Generation (DFG) algorithm Example: Binary strings of length (n) bits. The DFG algorithm generates a full permutation tree of n+1 levels Prof. Amr Goneid, AUC

Unbound DFG: Algorithm // Assume there is a root node void Unbound_DFG(int k, int n) { for (i = 0; i <= 1; i++) { Generate edge (i) from current parent; Generate child node; if (k == n) then leaf node has been reached; else Unbound_DFG(k+1, n); } Prof. Amr Goneid, AUC

Example: All 2-bit strings 1 x1 1 2 5 x2 1 1 3 4 6 7 {00} {01} {10} {11} n = 2; Unbound_DFG(1,n) Prof. Amr Goneid, AUC

7. Bound DFG: Backtracking A partial permutation tree is generated by a bound Depth-First Generation (DFG) algorithm Example: n-bit binary strings, with a bounding function B(k,i) == true The DFG algorithm generates a partial permutation tree of n+1 levels Prof. Amr Goneid, AUC

Bound DFG: Algorithm // Assume there is a root node void Bound_DFG(int k, int n) { for (i = 0; i <= 1; i++) { if ( B( k , i ) ){ Generate edge (i) from current parent; Generate child node; if (k == n) then leaf node has been reached; else Bound_DFG(k+1, n); } Prof. Amr Goneid, AUC

Example: n-bit binary strings 1 x1 1 2 5 x2 3 6 {00} {10} n = 2; Bound: 2nd bit should not be 1 B’(k,i) = (k==2) && (i==1) Hence B(k,i) = (k == 1) || (i == 0) Invoke as Bound_DFG(1,n) Prof. Amr Goneid, AUC

8. General Backtracking Algorithm Consider: n = no. of variables k = index of a variable x1 , x2, .. ,xk, .. xn is a path to a solution state. m = no. of possible values for x vi = a value for a variable, i = 1, 2, .. m B(k,i) = Bounding Function = true if xk can take the value vi Prof. Amr Goneid, AUC

Brute Force Algorithm void Brute_Force (int k, int n) { for all possible values vi of a variable (i = 1..m) Let variable xk take the value vi ; if (xk is the last variable ) output vector x[1:n]; else Brute_Force (k+1, n); } Prof. Amr Goneid, AUC

General Brute Force Code void Brute_Force (int k, int n) { for (i = 1; i <= m; i++) x[k] = v[i] ; if (k == n) output vector x[1:n]; else Brute_Force (k+1, n); } Prof. Amr Goneid, AUC

Backtracking Algorithm void Backtrack(int k, int n) { for (i = 1; i <= m; i++) { if ( B( k , i )) { // Pruning x[k] = v[i] ; if (k == n) output vector x[1:n]; else Backtrack(k+1, n); } Prof. Amr Goneid, AUC

9. Example: n-Binary bits Problem Find all strings of n-binary bits such that no two consecutive bits are the same. n given , m = 2 , xk = 0 or 1 , k = 1 , 2 , .. , n Bounding Function: Assume x0 = -1 then x1  x2 , x2  x3 , …… or if (xk-1  i) then xk can take the value (i) Prof. Amr Goneid, AUC

Backtracking Algorithm void Backtrack(int k, int n) { for (i = 0; i <= 1; i++) { if ( x[k-1] != i) { // assume x[0] = -1 x[k] = i ; if (k == n) output vector x[1:n]; else Backtrack(k+1, n); } Prof. Amr Goneid, AUC

Permutation Tree for n = 3 1 x1 1 2 9 1 x2 1 3 6 10 13 1 1 x3 7 8 11 12 {0,1,0) {1,0,1) Prof. Amr Goneid, AUC

10. The n-Queens Problem (a) The Problem Find all possible arrangements of n Queens on an n x n chessboard such that no two can attack each other. Example: 8 queens on an 8x8 board The problem has 92 solutions. Only 12 are unique, others are reflections or rotations. Prof. Amr Goneid, AUC

The n-Queens Problem Pre-Condition: Each Queen is on a different row. We seek the n-tuples X = {x1 , x2 , … , xn} representing column numbers satisfying the problem. Hence, we seek all permutations of X Prof. Amr Goneid, AUC

(b) The Bounding Function Assume that Queens 1, 2, .. K-1 have already been properly placed. The bounding function for Queen (k) is that it can be placed in column (i) iff: It does not share the same column (i) with any of the previous queens (j), j = 1,2,.., k-1, i.e Xj  i It is not on the same diagonal with any of the previous queens: │k-j│  │Xj - i│ , j = 1,2,.., k-1 Qj Row j Row k Qk Col X[ j] Col (i) Prof. Amr Goneid, AUC

The Bounding Function bool can_place (int k, int i) { for (int j = 1; j < k; j++) if ((x[ j ] == i) || abs(x[ j ] - i) == abs(k-j)) return false; return true; } Prof. Amr Goneid, AUC

(c)Backtracking Algorithm void NQueens(int k, int n) { for (i = 1; i <= n; i++) { if ( can_place( k , i ) ){ x[k] = i ; if (k == n) output vector x[1:n]; else NQueens(k+1, n); } Prof. Amr Goneid, AUC

The 8-Queens Problem Animation Prof. Amr Goneid, AUC

11. Another Backtracking Schema Another schema exists when we replace the bounding function by a function that assigns to x[k] a legal value after x[1: k-1] have already been assigned legal values out of m possible values. If no such value is available, the algorithm backtracks. Assume the function to be NextValue (k , n) and the vector x[1:n] is initially set to zero. Prof. Amr Goneid, AUC

The Algorithm void Backtrack2(int k, int n) { do { NextValue(k,n); // Assign to x[k] // a legal value if( !x[k] ) break ; // No possible value if (k == n) output vector x[1:n]; else Backtrack2(k+1, n); } while(1); } Prof. Amr Goneid, AUC

12. Hamiltonian Circuit Problem by Backtracking The Knight’s Tour problem is a Hamiltonian circuit problem Prof. Amr Goneid, AUC