1. Backtracking © Jeff Parker, 2009 Cogito, ergo spud: I think, therefore I yam.

2. 2 Outline Backtracking Problem - 8 Queens Demonstration of 4 Queens Code for 8 Queens problem How much will this cost? A better solution Turnpike Problem

3. 3 Backtracking – the 8 Queens Problem: place 8 queens on a chess board so that none attack each other General class of problems can be solved by backtracking

4. 4 Decreases size of search space If we were to examine all placements of queens, would take C(64,8) 64*63*62*61*60*59*58*57*56 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 4,426,165,368 Restrict queen i to col i means we only look at 8^8 positions 8 * 8 * 8 * 8 * 8 * 8 * 8 * 8 = (2^3) ^ 8 = 2 ^ 24 ~ 16,000,000 Restrict queen i to col i and avoid rows in use gives 8 * 7 * 6 * 5 * 4 * 3 * 2 * 1 = 40,320 possible moves Backtracking can reduce this far more. When we place the first queen we eliminate one of the 7 second slots, so we don't need to consider 2 * 6 * 5 * 4 * 3 * 2 * 1 = 1440 extensions In fact, for an 8x8 board we look at 114 partial boards The only complete boards we look at are solutions....

5. 5 Backtracking Demo for (r0 = 0; r0 < 4; r0++) if safe(r0, 0) { place queen at (r0, 0) for (r1 = 0; r1 < 4; r1++) if safe(r1, 1) { place queen at (r1, 1) for (r2 = 0; r2 < 4; r2++) In the slides to follow, read from left to right, top to bottom.

6. 6

7. 7

8. 8 A solution to the 4 queens problem

9. 9 Stack Tracing In effect, we are placing our partial solution on the system's procedure stack. We extend in a new stack frame. If we fail, we remove frame, and return to our position If we find a safe row, we try to extend. If no row works out, we pop the stack and retry in the previous column Recursion hides the stack - simply the procedure stack

10. 10 Backtracking Skeleton // Not every backtracking example has all points below: useful template boolean backtracking(some parameters) { if (we have a solution) report results and return true; if (there is no hope) return false; for (first position to last postion) { if (this looks like a legal postion) { Record position // Can I use this step and solve the rest of the problem? if (backtracking(parameters modified to reflect my new position)) // Success! Record position and return remember my position return true; else remove any traces from this call and try next iteration of loop return false; // If I reach here, I was not successful. Backtrack

11. 11 Backtracking for 8 Queens def check(lst, col, rowfree, upfree, downfree): """Try to extend a solution into col.""" print col, lst# Debugging: 3 [0, 3, 1, -1] if (col == len(lst)): print "Success!" printBoard(lst) return True for row in xrange(len(lst)): if (safe(lst, row, col, rowfree, upfree, downfree)): place(lst, row, col, rowfree, upfree, downfree) if (check(lst, col+1, rowfree, upfree, downfree)): return True remove(lst, row, col, rowfree, upfree, downfree) return False # Backtrack

12. 12 Why use Recursion? Why not use nested for loops, as suggested? for (r0 = 0; r0 < 4; r0++) if safe (r0, 0) { place queen at (r0, 0) for (r1 = 0; r1 < 4; r1++) if safe (r1, 1) { place queen at (r1, 1) for (r2 = 0; r2 < 4; r2++)... Does not scale to different sized boards You must duplicate identical code (place and remove). An error in one spot is hard to find

13. 13 4 Queens Results 0 [-1, -1, -1, -1] 1 [0, -1, -1, -1] 2 [0, 2, -1, -1] Backtrack 2 [0, 3, -1, -1] 3 [0, 3, 1, -1] Backtrack 1 [1, -1, -1, -1] 2 [1, 3, -1, -1] 3 [1, 3, 0, -1] 4 [1, 3, 0, 2] Success!. *. *.. *. *.

14. 14 3 Queens Results 0 [-1, -1, -1] 1 [0, -1, -1] 2 [0, 2, -1] Backtrack 1 [1, -1, -1] Backtrack 1 [2, -1, -1] 2 [2, 0, -1] Backtrack Could not solve for a board with side 3

15. 15 8 Queens Results *....... *.. *...... * *..... *.... *.. *..

16. 16 Finding All Solutions // No longer returns after success: keep trying void backtracking(some parameters) { if (we have a solution) report the position and return; if (there is no hope) return; for (first position to last postion) { if (this looks like a legal postion) { # if (backtracking(parameters modified)) # return true; backtracking(parameters modified) remove traces of this call //... and try the next iteration

17. 17 Find all Solutions def check(lst, col, rowfree, upfree, downfree): """Find All Solutions""" if (col == len(lst)): print "Success!" printBoard(lst) return True for row in xrange(len(lst)): if (safe(lst, row, col, rowfree, upfree, downfree)): place(lst, row, col, rowfree, upfree, downfree) #if (check(lst, col+1, rowfree, upfree, downfree)): # return True check(lst, col+1, rowfree, upfree, downfree) remove(lst, row, col, rowfree, upfree, downfree) return False # Backtrack

18. 18 Data Structures How do we store the board? We could use a two dimensional array of booleans. void place(int row, int column) { board[row][column] = QUEEN; } Operations needed for backtracking Check to see if queen is safe - safe Place a queen - place Remove Queen - remove Takes a fixed amount of time to set or remove. But which of the three operations above do we do most often? if (safe(...)): place(...) if (check(...)): return True remove(...)

19. 19 Python Profiler counts the calls 2166 function calls (2053 primitive calls) in 0.045 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 38 0.000 0.000 0.000 0.000 :0(insert) 696 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.044 0.044 :1( ) 876 0.009 0.000 0.011 0.000 Queens.py:19(safe) 113 0.002 0.000 0.003 0.000 Queens.py:29(place) 105 0.002 0.000 0.003 0.000 Queens.py:36(remove) 219 0.003 0.000 0.003 0.000 Queens.py:43(prettyPrint) 114/1 0.021 0.000 0.043 0.043 Queens.py:49(check) 1 0.000 0.000 0.000 0.000 Queens.py:7(printBoard) 1 0.000 0.000 0.043 0.043 Queens.py:81(solve) 0 0.000 0.000 profile:0(profiler) 1 0.000 0.000 0.045 0.045 profile:0(solve([-1, -1, -1, -1, -1, -1, -1, -1]))

20. 20 Testing a new position When we wish to test position (i, j), we attack over 20 squares. We don't need to test this col, nor the spots to our right. Worst case is 14 squares (x, j), (i, y), (i+x, j+x), (i+x, j-x) Still, there is a good deal of work to be done here....

21. 21 Alternative Store board as an array of row positions. The board above is [5, 3, 0, 4, 1,.....] Nothing new yet... Store an array of booleans rowFree Is this row free of queens?

22. 22 Smarter Storage Store 3 arrays of boolean rowfree, upfree, downfree Is this row under attack? Is this diagonal under attack?... Placement & removal is now more expensive: must touch 4 arrays However, when testing we only examine 3 array locations (col is implicit)

23. 23 Is this spot safe? # When testing we only examine 3 array locations (col is implicit) def safe(lst, row, col, rowfree, upfree, downfree): """Is it safe to place a queen at (row, col)?""" if (not rowfree[row]): return False if (not upfree[row+col]): return False if (not downfree[row-col+len(lst)-1]): return False return True

24. 24 Place and Remove def place (lst, row, col, rowfree, upfree, downfree): """Put a queeen in pos (row, col).""" lst[col] = row rowfree[row] = False upfree[row+col] = False downfree[row-col+len(lst)-1] = False def remove(lst, row, col, rowfree, upfree, downfree): """Remove a queeen from pos (row, col).""" lst[col] = -1 rowfree[row] = True upfree[row+col] = True downfree[row-col+len(lst)-1] = True

25. 25 Initialize the arrays def solve(lst): """Set things up for a run.""" rowfree = [ ] upfree = [ ] downfree = [ ] for x in xrange(len(lst)): rowfree.insert(x, True) for x in xrange(2*len(lst)- 1): upfree.insert(x, True) downfree.insert(x, True) if (not check(lst, 0, rowfree, upfree, downfree, 0)): print "Can not solve for a board with side", len(lst) solve([-1, -1, -1, -1])

26. 26 Profile the run import profile... profile.run('solve([-1, -1, -1, -1, -1, -1, -1, -1])') # ================ Output ================= 2166 function calls (2053 primitive calls) in 0.045 CPU seconds Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 38 0.000 0.000 0.000 0.000 :0(insert) 696 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.044 0.044 :1( ) 876 0.009 0.000 0.011 0.000 Queens.py:19(safe) 113 0.002 0.000 0.003 0.000 Queens.py:29(place) 105 0.002 0.000 0.003 0.000 Queens.py:36(remove)

27. 27 Summary Backtracking allows us to search a large solution space It organizes the search, and avoids wasting time Clever choice of data structure can save a great deal of time Rather than looking at a dozen squares in 8 Queens, we could test 3 locations Clever modification of algorithms can yield large savings Representation should be tuned to the requirements

28. 28 Turnpike def extend(cand, lst, depth): """Cand generates a subset of list. Try to add elt.""" """We assume that subset(differences(cand), lst)""" # Are we done? if (subset(lst, differences(cand))): print "Success!! ", cand, lst return True for x in lst: if ((x > 0) and (not (x in cand))): nwLst = cand[:] insrt(nwLst, x) if (subset(differences(nwLst), lst)): if (extend(nwLst, lst, depth+1)): return True return False

29. 29 Sample Run Extend [0] [] vs [2, 8, 10] Try extending by adding 2 to get [0, 2] Extend [0, 2] [2] vs [2, 8, 10] Try extending by adding 8 to get [0, 2, 8] Try extending by adding 10 to get [0, 2, 10] Extend [0, 2, 10] [2, 8, 10] vs [2, 8, 10] Success!! [0, 2, 10] [2, 8, 10]

30. 30 Turnpike def extend(cand, lst, depth): """Cand generates a subset of list. Try to add an elt.""" """We assume that subset(differences(cand), lst)""" prettyPrint(depth, "Extend") print cand, differences(cand), "vs", lst # Are we done? if (subset(lst, differences(cand))): print "Success!! ", cand, lst return True for x in lst: if ((x > 0) and (not (x in cand))): nwLst = cand[:] insrt(nwLst, x) prettyPrint(depth, "Try extending by adding") print x, "to get", nwLst if (subset(differences(nwLst), lst)): if (extend(nwLst, lst, depth+1)): return True return False

31. 31 Utility def insrt(lst, itm): """Insert item itm into sorted list lst""" spot = -1 for x in xrange(len(lst)): if (lst[x] < itm): spot = x else: break lst.insert(spot+1, itm) return lst def differences(a): """Take {0, 2, 4, 7, 10} generate {2, 2, 3, 3, 4, 5, 6, 7, 8, 10}""" lst = [] for x in xrange(len(a)): for y in xrange(x+1, len(a)): insrt(lst, a[y] - a[x]) return lst

32. 32 Subset def subset(a, b): """Assume a and b are sorted. Is a a subset of b?""" if (len(a) > len(b)): return False if (0 == len(a)): return True x = 0 for y in xrange(len(b)): if (a[x] < b[y]): return False if (a[x] == b[y]): x = x + 1 if (x == len(a)): return True return False

33. 33 Pretty Print def prettyPrint(depth, string): """Indent to show the levels of recursion""" for lp in xrange(depth): print "\t", print string, def extend(cand, lst, depth): """Cand generates a subset of list. Try to add an elt.""" prettyPrint(depth, "Extend") print cand, differences(cand), "vs", lst... if (extend(nwLst, lst, depth+1)): Extend [0] [] vs [2, 8, 10] Try extending by adding 2 to get [0, 2] Extend [0, 2] [2] vs [2, 8, 10] Try extending by adding 8 to get [0, 2, 8] Try extending by adding 10 to get [0, 2, 10] Extend [0, 2, 10] [2, 8, 10] vs [2, 8, 10] Success!! [0, 2, 10] [2, 8, 10]

34. 34 Improvements Extend [0] [] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 2 to get [0, 2] Extend [0, 2] [2] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 3 to get [0, 2, 3] Try extending by adding 4 to get [0, 2, 4] Extend [0, 2, 4] [2, 2, 4] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 3 to get [0, 2, 3, 4] Try extending by adding 5 to get [0, 2, 4, 5] Try extending by adding 6 to get [0, 2, 4, 6] Try extending by adding 7 to get [0, 2, 4, 7] Extend [0, 2, 4, 7] [2, 2, 3, 4, 5, 7] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7,... Try extending by adding 3 to get [0, 2, 3, 4, 7] Try extending by adding 5 to get [0, 2, 4, 5, 7] Try extending by adding 6 to get [0, 2, 4, 6, 7] Try extending by adding 8 to get [0, 2, 4, 7, 8] Try extending by adding 10 to get [0, 2, 4, 7, 10]

35. 35 Improvements Extend [0] [] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 2 to get [0, 2] Extend [0, 2] [2] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 3 to get [0, 2, 3] Try extending by adding 4 to get [0, 2, 4] Extend [0, 2, 4] [2, 2, 4] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12] Try extending by adding 3 to get [0, 2, 3, 4] Try extending by adding 5 to get [0, 2, 4, 5] Try extending by adding 6 to get [0, 2, 4, 6] Try extending by adding 7 to get [0, 2, 4, 7] Extend [0, 2, 4, 7] [2, 2, 3, 4, 5, 7] vs [2, 2, 2, 3, 3, 4, 5, 5, 6, 7,... Try extending by adding 3 to get [0, 2, 3, 4, 7] Try extending by adding 5 to get [0, 2, 4, 5, 7] Try extending by adding 6 to get [0, 2, 4, 6, 7] Try extending by adding 8 to get [0, 2, 4, 7, 8] Try extending by adding 10 to get [0, 2, 4, 7, 10]

36. 36 Profile Ordered by: standard name ncalls tottime percall cumtime percall filename:lineno(function) 342 0.003 0.000 0.003 0.000 :0(insert) 717 0.006 0.000 0.006 0.000 :0(len) 1 0.001 0.001 0.001 0.001 :0(setprofile) 1 0.000 0.000 0.027 0.027 :1( ) 38 0.005 0.000 0.020 0.001 Turnpike.py:18(differences) 32 0.002 0.000 0.003 0.000 Turnpike.py:26(subset) 32 0.000 0.000 0.000 0.000 Turnpike.py:42(prettyPrint) 6/1 0.002 0.000 0.027 0.027 Turnpike.py:48(extend) 342 0.009 0.000 0.014 0.000 Turnpike.py:7(insrt) 1 0.000 0.000 0.029 0.029 profile:0(extend([0], [2, 2, 2, 3, 3, 4, 5, 5, 6, 7, 8, 8, 10, 10, 12], 0)) 0 0.000 0.000 profile:0(profiler)