A backtrack data structure and algorithm CS420: Dancing Links A backtrack data structure and algorithm by Donald Knuth (wim bohm cs.colostate.edu) Except as otherwise noted, the content of this presentation is licensed under the Creative Commons Attribution 2.5 license.

Dancing Links Doubly linked list nodes have references x Doubly linked list nodes have references L to left node L and R to right node

Dancing Links Doubly linked list Remove x R[L[x]]=R[x] L[R[x]]=L[x] notice that x still, via its L and R, points at left and right nodes, and thus we can easily ... x

Dancing Links Doubly linked list Remove x R[L[x]]=R[x] L[R[x]]=L[x] Put x back R[L[x]]=x L[R[x]]=x x x

Using Dancing Links O(1) put x back operation Works in following BackTrack scenario . we have created a “space to be searched” as a global doubly linked data structure . we search this space by DFS, selecting: taking out certain options and putting them back in reverse order What does not work . adding completely new options . putting options back in other than reverse order

Why reverse order? x y x y remove x

Why reverse order? x y x y remove x x y remove y

What happens if x is put back first? y x y remove x x y remove y

What happens if x is put back first? y remove y x y put back x x points at y and y at x, but y is not in the chain!!

Exact sub-set / cover problem Given a matrix of 0-s and 1-s, find a subset of rows with exactly one 1 in each column of A Backtrack approach Pick a column c, pick a row r with 1 in c, remove columns c and j with 1 in r and rows with a 1 coinciding with 1-s in r a b c d e f g reducing the problem 1 0 0 1 0 1 1 0 eg c in row 1 2 1 0 0 1 0 0 1 selects subset {c,e,f} 3 0 1 1 0 0 1 0 so remove columns c,e,f 4 1 0 0 1 0 0 0 and row 1 and row 3 5 0 1 0 0 0 0 1 because it overlaps with row 1

Exact sub-set / cover problem Backtrack approach Pick column c, row 1 subset {c,e,f} which rows and cols disappear? a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1

Exact sub-set / cover problem Backtrack approach Pick column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick column a, row 2 {a,d,g} which rows, cols disappear? a b d g 2 1 0 1 1 4 1 0 1 0 5 0 1 0 1

Exact sub-set / cover problem Backtrack approach Pick column c, row 1 {c,e,f} Pick column a, row 2 {a,d,g} all rows/cols disappear what is the problem?

Exact sub-set / cover problem Backtrack approach Pick column c, row 1 {c,e,f} Pick column a, row 2 {a,d,g} all rows/cols disappear b is not covered, so backtrack

Exact sub-set / cover problem Backtrack Pick column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick column a, row 4 {a,d} which rows, cols disappear now? a b d g 2 1 0 1 1 4 1 0 1 0 5 0 1 0 1

Exact sub-set / cover problem Backtrack Pick column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick column a, row 4 {a,d} rows 2,4 cols a,d disappear b g 5 1 1

Exact sub-set / cover problem Backtrack Pick column c, row 1 {c,e,f} rows 1,3 and cols c,e,f disappear Pick column a, row 4 {a,d} rows 2,4 cols a,d Pick column b, row 5 {b,g} row 5 disappears Exact cover accomplished! matrix is empty

Exact sub-set / cover problem Subsets / rows 1,4,5 provide exact cover! a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1

Exact set cover The columns: elements of a universe The rows: subsets of elements in the universe Find a set of subsets that has each element exactly once Union of the set is the Universe Intersection of any two subsets is empty Finding an exact cover is “tough” NP-Complete e.g. when each subset has three elements Great candidate for backtrack search Representation: row column doubly linked sparse matrix only containing 1-s Use dancing links to remove and put back candidates

Pentominoes A pentomino is a size 5 n-omino composed of n congruent squares connected orthogonally by side (not point wise) There are 12 different pentominoes when rotation, and mirroring are allowed There are 18 pentominoes when only rotation is allowed

The 12 pentominoes F I L N P T U V W X Y Z

Early implementation Dana Scott wrote a backtrack program in 1958 for the Maniac (4000 instructions / sec) tiling a chessboard with a 2x2 hole in the middle with the 12 pentominoes, using each pentomino exactly once The code ran in ~3.5 hours: (50 million instructions)

One of the 65 solutions

Board shapes Tiling different board shapes Chessboard with 2x2 hole in middle or with 4 holes in arbitrary places Rectangles 6 x 10 5 x 12 4 x 15 3 x 20 3D boxes

Pentomino problems are exact cover problems Matrix with 72 columns 12 pentominoes and 60 positions in the board’s grid Each row has 6 1-s 1 for the pentomino, 5 for its positions Each row is a description of a pentomino in a certain position There are 1568 such rows

Translating puzzle to set cover Pentominoes matrix: 72 columns 1568 rows Let’s do a simpler puzzle 4 triominoes I: L: ┐: -: No rotation or flip Rectangle 3 x 4 # possible solutions?

Translating puzzle to set cover Pentominoes matrix: 72 columns 1568 rows Let’s do a simpler puzzle 4 triominoes I: L: ┐: -: No rotation or flip Rectangle 3 x 4 Possible solution:

Setcover matrix for simple puzzle 16 columns: 4 columns for the pieces 12 columns for the positions Rows: 4 I placements I L ┐ - (1,1) (1,2) (1,3) (1,4) (2,1) (2,2) (2,3) (2,4) (3,1) (3,2) (3,3) (3,4) 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 1 0 0 0 0 1 0 0 0 1 0 0 0 1 0 0 1 0 0 0 0 0 1 0 0 0 1 0 0 0 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1

Setcover matrix cont’ Rows 6 L placements I L ┐ - (1,1) (1,2) (1,3) (1,4) (2,1) (2,2) (2,3) (2,4) (3,1) (3,2) (3,3) (3,4) 0 1 0 0 1 0 0 0 1 1 0 0 0 0 0 0 0 1 0 0 0 1 0 0 0 1 1 0 0 0 0 0 0 1 0 0 0 0 1 0 0 0 1 1 0 0 0 0 0 1 0 0 0 0 0 0 1 0 0 0 1 1 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 1 1 0 0 1 0 0 0 0 0 0 0 0 1 0 0 0 1 1

Setcover matrix cont’ 6 ┐placements 6 – placements

Backtrack algorithm X sketch // A is the exact cover matrix if (empty(A)) return solution else { choose a column c choose a row r with Ar,c = 1 include r in solution for each column j with Ar,j = 1 { delete column j from A for each row i with Ai,j = 1 delete row i from A } repeat recursively on reduced A

algorithm X The subsets form a search tree Root: original problem Level k node: k subsets have been chosen and all columns (elements) in the subsets and all overlapping subsets (rows) have been removed Any systematic rule for choosing columns will find all solutions

Choosing c One way Better ways Pick pentominoes in alphabetical order F I L N P T U V W X Y Z Not good: F has 192 possible places, I then has ~32, very big search space ~2x212 Better ways Choose lexicographically first uncovered position, starting with (1,1), search space ~107 Scott realized that X has essentially three possible places centered at (2,3), (2,4) and (3,3). The rest leads to symmetrical solutions. Search space ~350,000 Knuth has a more general solution: pick column with minimal number of 1-s

Let’s dance Represent each 1 in A by a node with 5 links L(x), R(x) (left right) U(x), D(x) (up down) C(x) column header Each row is a doubly (L,R) linked circular list Each column is a doubly linked (U,D) circular list Each column has a column header node c with additional name N(c) and size S(c). The column headers form a circular row with a header h Each node points at its column header with link C

Our example h a 2 b 2 c 2 d 2 e 1 f 2 g 2 a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1

Search algorithm DLX search(int k){ // search is called with k=0 from outside if(R[h]=h) print O else { // O is the set of currently picked rows choose column c cover column c // pick element c for each row r = D[c], D[D[c]]… while r!=c { O[k]=r // pick a row with element c for each j = R[r], R[R[r]]… while j!=r cover column C[j] // all elements in the row are now covered search(k+1) for each j = L[r], L[L[r]]… while j!=r uncover column C[j] // uncover in reverse order } uncover column c and return; } }

cover column c // remove c from headers L[R[c]]= L[c]; R[L[c]]=R[c] // remove all rows in c’s column for each row r = D[c], D[D[c]]… while r!=c for each j = R[r], R[R[r]]… while j!=r { U[D[j]]=U[j]; D[U[j]]=D[j]; S[C[j]]--; }

uncover c // put rows back IN REVERSE ORDER // last out first back in for each row r = U[c], U[U[c]]… while r!=c for each j = L[r], L[L[r]]… while j!=r { S[C[i]]++; U[D[j]]=j; D[U[j]]=j; } // put header back L[R[c]]=c; R[L[c]]=c;

cover a: remove header h a 2 b 2 c 2 d 2 e 1 f 2 g 2 a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1

cover a: remove row 2 h a 2 b 2 c 2 d 1 e 1 f 2 g 1 b c d e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0 4 0 0 1 0 0 0 5 1 0 0 0 0 1

cover a; remove row 2 and 4 h a 2 b 2 c 2 d 0 e 1 f 2 g 1 b c d e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0 5 1 0 0 0 0 1 Notice the asymmetry in D2 (element D in row 2) versus D4 (element D in row 4). Putting back D2 first would change D4’s Links, but D4 is not in the set! This is why we need to uncover in exactly the reverse order, so we know that the element that is put back refers to elements in the set h a 2 b 2 c 2 d 0 e 1 f 2 g 1

cover a; last step: remove row 5 b c d e f g 1 0 1 0 1 1 0 3 1 1 0 0 1 0

Picking subset 2 leads to failure a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 b c e f 1 0 1 1 1 3 1 1 0 1 nothing left for e select a, row 2 cover a,d,g remove rows 2,4,5 select b, row 3 cover b,c,f remove rows 1,3

Picking subset 2 leads to failure a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 b c e f 1 0 1 1 1 3 1 1 0 1 select a, row 2 cover a,d,g select c, row 1 remoce rows 1,3 now what?

Picking subset 2 leads to failure a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 b c e f 1 0 1 1 1 3 1 1 0 1 nothing left for b select a, row 2 cover a,d,g select c, row 1 cover c,e,f

Picking subset 4 succeeds c e f 1 1 1 1 a b c d e f g 1 0 0 1 0 1 1 0 2 1 0 0 1 0 0 1 3 0 1 1 0 0 1 0 4 1 0 0 1 0 0 0 5 0 1 0 0 0 0 1 b c e f g 1 0 1 1 1 0 3 1 1 0 1 0 5 1 0 0 0 1 select c pick subset 1 empties A select a pick subset 4 cover a,d remove rows 2,4 select b picking subset 3 remove rows 1,3,5 fails again pick subset 5 cover b,g remove rows 3,5 Exact cover: subsets 4 {a,d}, 5 {b,g} and 1 {c,e,f}