1. CS420 lecture ten BACKTRACK

2. Solution vectors In optimization problems, or more general in search problems, a set of choices are to be made to arrive at an optimum or solution. A solution can be written in the form of a vector X = (x 1,x 2,..,x n ) and the search is directed by criterion P, defining feasible choices of X. Often P can also decide whether a partial solution X i =(x 1,..,x i ), i  n, is feasible, ie, there are extensions of X i to a feasible solution. We can either search for one solution, or we can search for all solutions.

3. Constraints Explicit constraints define the domain of the choice variables, eg x i >0, or x i =0 or 1, or l i <x i <u i. Implicit constraints specify that x 1.. x i satisfy P, ie relate the x i -s to each other. The explicit constraints define a possible solution space for the problem The implicit constraints narrow the solution space down to an answer space.

4. Search Spaces We concern us here with tree shaped spaces A prefix x 1.. x i is a path from the root (the empty vector) with edges labeled x 1 to x i Children of x i form solution vectors of the form x 1.. x i+1.

5. n Queens Given an nxn chessboard and n queens, find all placements of the queens such that none of them attack each other – none of the queens occupy the same row, column, or diagonal. As each queen occupies her own row, a (partial) choice vector X is a vector of integers from 1 to n, where x j is the column for the queen in row j. One solution to the 4-queens problem is (2,4,1,3)

6. Q 4 queens 2 4 1 3 solution

7. Solution space for n-Queens root has n children, 1 for each column each of these has n-1 children, excluding its own column next has n-2 children etc. so the tree has n! nodes / states, many of these are infeasible – diagonal constraint n! is a gross overestimate of the number of states we need to check

8. Subset sum (SSS) Given a set of integers m 1 to m n and another integer M, find all subsets of m i -s such that their sum is M Eg for m ={1,2,3}, M=3 we have two solutions {1,2} and {3}.

9. Solution Vectors for SSS One option: the indices of the chosen elements then solution vectors have different length

10. Solution Vectors for SSS One option: the indices of the chosen elements then solution vectors have different length X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3

11. Alternative Another, more regular, solution space has nodes with up to n choice values c i = 0 if m i is not in the subset and c i = 1 if it is. In this case the leaves represent the subsets.

12. Alternative Another, more regular, solution space has nodes with up to n choice values c i = 0 if m i is not in the subset and c i = 1 if it is. In this case the leaves represent the subsets. C 1 = 0 C 1 = 1 C 2 = 1 C 2 = 0 C 3 = 0 C 3 = 1

13. Backtrack Backtrack walks depth first through the solution space and uses the criterion P (sometimes called bounding function B) to prune the solution space to create the answer space.

14. Iterative Backtrack We are searching for all possible answers. X will hold the (partial) solution vector (x 1,x 2,..,x i ). B is the bounding function. Next(X) is the set of valid next states x i+1 from state x i, and therefore B i+1 (X[1..(i+1)) is true for all states in Next(X) and false for all other states. We also need a function that decides whether a state is an answer state.

15. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set }

16. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=1 x=[] cap = 3

17. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=2 x=[1] feasible cap=2

18. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=3 x=[1,2] answer path cap=0 print X=[1,2]

19. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=3 x=[1,2,3] infeasible backtrack

20. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=2 X=[1,3] infeasible backtrack

21. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=1 X=[2] feasible

22. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=2 X=[2,3] infeasible backtrack

23. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=1 X=[3] feasible

24. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 k=2 X=[3] cap=0 path to answer node print X=[3]

25. IBackTrack(n){ k = 1 while (k>0) { if (there remains an x in Next(X[1..(k-1)]) and B k (X[1..k]) ) { if (X[1..k] is path to answer node) print X[1..k] // end if k++ // consider next set } else k-- // backtrack to previous set } X 1 =1 X 1 =3 X 2 =2 X 1 =2 X 2 =3 X 3 =3 paths to answer node X=[1,2] X=[3]

26. Recursive back track global variable X[1:n], initial call RBackTrack(1) RBackTrack(k) { for each X[k] with B k (X[1..k])==true if (X[1..k] is a path to an answer state) print(X[1..k] RBackTrack(k+1) }

27. n-queens X[1:k] is a partial solution with X[i] the column number (1..n) of a queen in row i 1:k. When we place queen k+1 we need to check that she does not occupy any of the already occupied columns (that's easy) and any of the already occupied diagonals: – two queens in (i,j) (row i, column j) and (p,q) occupy the same diagonal if (i-j) == (p-q) or (i+j)==(p+q), which is equivalent with |i-p|==|j- q|.

28. iterative n-queens with global partial solution X Place(k) for (i = 1 to k-1) if (X[i]==X[k] or // same columns | X[i]-X[k] | == |i-k| ) // same diagonal return(false); // end for return(true)

29. nQueens(n) { X[1] = 0 //place first queen "in front of" row 1 k=1 while (k>0) { X[k]++ // move queen k to next column while (X[k]<=n and not(Place(k)) { X[k]++ } if (X[k] <= n) // found valid spot for queen k if (k==n) print(X) else {k++; X[k]=0} // start next row else k-- // backtrack }

30. Sum of Subsets Two ways to represent the search space for subset sum. Here we consider the regular case with choices x i =0 (do not take object i) and x i =1 (take object i) at level i. A choice for the bounding function B k =

31. Simplifying / Strengthening assumption for SSS From hereon we assume that the objects are sorted in non-decreasing order, because this allows us to strengthen our observations.

32. Weight gathered so far In some state X[1..k] at level k the total weight gathered by objects 1 to k is either M, and we have found a solution and thus we don't need to go deeper in that state... or the weight gathered is less than M, and then X[1..k] cannot lead to an answer node if

33. Weight gathered so far In some state X[1..k] at level k the total weight gathered by objects 1 to k is either M, and we have found a solution and thus we don't need to go deeper in that state... or the weight gathered is less than M, and then X[1..k] cannot lead to an answer node if WHY?

34. A better bound Original bound plus previous observation leads to a better bound: not too little and not too much

35. Recursive SSS s is the weight gathered by the objects chosen so far, and thus needs to be initialized at 0 r is the total weight of the remaining objects, and thus needs to be initialized at the sum SumW of all the weights. k is the current 1-based level. If we take object k we can simplify the bound test because the sum of the weights of the chosen objects (now including k) plus remaining objects has not changed and is still >= M The initial call is SumOfSub(0,1,SumW).

36. No degenerate problems We assume that W[1] =M, otherwise we have a degenerate problem.

37. SumOfSub(s,k,r) { // two possible next states X[k] = 1 //1: take object k, we can: B k-1 = true if (s+W[k]==M) print X[1..k] // subset found else (if s+W[k]+W[k+1]<=M) // B k = true SumOfSub(s+W[k],k+1,r-W[k]) // do not take object k, // now we have to check whether the rest // of the objects still can reach M and // the lightest remaining object is not too heavy if (s + r - W[k] >= M // still can reach M and s+W[k+1] <=M // rest objects not too heavy ) SumOfSub(s,k+1,r-W[k]) }

38. SumOfSub(s,k,r) { // two possible next states X[k] = 1 //1: take object k, we can: B k-1 = true if (s+W[k]==M) print X[1..k] // subset found else (if s+W[k]+W[k+1]<=M) // B k = true SumOfSub(s+W[k],k+1,r-W[k]) // do not take object k, // now we have to check whether the rest // of the objects still can reach M and // the lightest remaining object is not too heavy if (s + r - W[k] >= M // still can reach M and s+W[k+1] <=M // rest objects not too heavy ) SumOfSub(s,k+1,r-W[k]) } Notice that the algorithm does not check k<=n. This is implicit in the fact that when SumOfSub is called s M, hence r>0 and thus k<=n.