Constraint Satisfaction Problems

Contents Representations Solving with Tree Search and Heuristics Constraint Propagation Tree Clustering

Posing a CSP A set of variables V1, …, Vn A domain over each variable D1,…,Dn A set of constraint relations C1,…,Cm between variables which indicate permitted combinations Goal is to find an assignment to each variable such than none of the constraints are violated

Constraint Graphs Nodes = variables Edges = constraints Example: map coloring A B C D A B C D

N-ary Constraint Graphs Example: Variables: X=[1,2] Y=[3,4] Z=[5,6] Constraints: X + Y = Z X Y Z Hyper graph X Y Z Primal constraint graph (Roman Barták, 1998 )

Making a Binary CSP Can convert n-ary constraint C into a unary constraint on new variable Vc Dc = cartesian product of vars in C Can convert n-ary CSP into a binary CSP Create var Vc for each constraint C (as above) Domain Dc = cartesian product – tuples that violate C Add binary equivalence constraints between new variables Vc, Vc’:C,C’ share var X  Vc,Vc’ must agree on X

Making a Unary Constraint Variables: X=[1,2] Y=[3,4] Z=[5,6] Constraints: X + Y = Z X Y Z XYZ [(1,3,5),(1,3,6),  (1,4,5),(1,4,6),  (2,3,5),(2,3,6)  (2,4,5),(2,4,6)] XYZ= (Roman Barták, 1998 )

Making a Unary Constraint Variables: X=[1,2] Y=[3,4] Z=[5,6] Constraints: X + Y = Z X XYZ Y Z XYZ= [(1,4,5), (2,3,5), (2,4,6)] (Roman Barták, 1998 )

Making a Binary CSP Variables: X=[1,2] Y=[3,4] Z=[5,6] W=[1,3] Constraints: X + Y = Z, W<Y X Y Z W XYZ WY Y Dual constraint graph (Roman Barták, 1998 )

Making a Binary CSP Variables: X=[1,2] Y=[3,4] Z=[5,6] W=[1,3] Constraints: X + Y = Z, W<Y X XYZ WY Y Dual constraint graph Y Z W (Roman Barták, 1998 )

Contents Representations Solving with Tree Search and Heuristics Constraint Propagation Tree Clustering

Generate and Test Generate each possible assignment to the variables and test if constraints are satisfied Exponential possibilities: O(d n) Simple but extremely wasteful!

DFS and Backtracking Depth first search Levels represent variables Branches off nodes represent a possible instantiations of variables Test against constraints after every variable instantiation and backtrack if violation Incrementally attempts to extend partial solution Whole subtrees eliminated at once

Example V1 red green blue V2 V3 red red green (*,*,*)

Example V1 red green blue V2 V3 red red green (*,*,*) (r,*,*)

Example V1 red green blue V2 V3 red red green (*,*,*) (r,*,*) (r,r,*)

Example V1 red green blue V2 V3 red red green (*,*,*) (r,*,*) (g,*,*) (r,r,*) (g,r,*)

Example V1 red green blue V2 V3 red red green (*,*,*) (r,*,*) (g,*,*) (r,r,*) (g,r,*) (g,r,r) (g,r,g)

Example V1 red green blue V2 V3 red red green (*,*,*) (r,*,*) (g,*,*) (r,r,*) (g,r,*) (b,r,*) (g,r,r) (g,r,g) (b,r,r) (b,r,g)

Forward Checking Backtracking is still wasteful A lot of time is spent searching in areas where no solution remains Ex. setting V4 to value X1 eliminates all possible values for V8 under the given constraints Can cause thrashing Forward checking removes restricted values from the domains of all uninstantiated variables If a domain becomes empty backtracking is done immediately

Heuristics The search can usually be sped up by searching intelligently: Most-constrained variable: Expand subtree of variables that have the fewest possible values within their domain first Most-constraining variable: Expand subtree of variables which most restrict others first Least-constraining value: Choose values that allow the most options for the remaining variables first

Contents Representations Solving with Tree Search and Heuristics Constraint Propagation Tree Clustering

Constraint Propagation A preprocessing step to shrink the CSP Constraints are used to gradually narrow down the possible values from the domains of the variables A singleton may result If the domains of each variable contain a single value we do not need to search

Arc Consistency Arc (Vi,Vj) in a constraint graph is arc consistent if for every value of Vi there is some value that is permitted for Vj Algorithm: Complexity O(ed3) do foreach edge (i,j) delete values from Di that cause Arc(Vi,Vj) to fail while deletions

Example V1 green V2 V3 red green blue green blue Consider edge (1,3)

Example V1 green V2 V3 red green blue green blue Consider edge (3,1)

Example V1 green V2 V3 red green blue green blue Consider edge (2,1)

Example V1 green V2 V3 red green blue green blue Consider edge (2,3)

Example V1 green V2 V3 red green blue green blue Consistent and a singleton!

Levels of Consistency Algorithms we have seen before are combinations of tree search and arc consistency: Generate and Test Backtracking Forward Checking Partial Lookahead Full Lookahead Really Full Lookahead TS BT = TS + AC 1/5 FC = TS + AC 1/4 PL = FC + AC 1/3 FL = FC + AC 1/2 RFL = FC + AC (Nadel, 1988)

Backtracking Given: check(i,Xi,j,Xj): true if Vi = Xi and Vj = Xj is permitted by constraints revise(i,j): true if Di is empty after making Arc(Vi,Vj) = true function BT(i,var) for(var[i]=Di) CONSISTENT = true for(j=1:i-1) CONISITENT = check(i,var[i],j,var[j]) end if CONSISTENT if i==n disp(var) else BT(i+1,var) function BT(i) EMPTY_DOMAIN = check_backward(i) if ~EMPTY_DOMAIN for(var[i]=Di) Di = var[i] if i==n disp(var) else BT(i+1) end function check_backward(i) for(j=1:i-1) if revise(i,j) return true return false

Forward Checking Similar to backtracking except more arc-consistency function FC(i) EMPTY_DOMAIN = check_forward(i) if ~EMPTY_DOMAIN for(var[i]=Di) Di = var[i] if i==n disp(var) else FC(i+1) end function check_forward(i) if i>1 for(j=i:n) if revise(j,i-1) return true return false Similar to backtracking except more arc-consistency

Levels of Consistency Generate and Test Backtracking Forward Checking Partial Lookahead Full Lookahead Really Full Lookahead TS BT = TS + AC 1/5 FC = TS + AC 1/4 PL = FC + AC 1/3 FL = FC + AC 1/2 RFL = FC + AC (Nadel, 1988)

A Stronger Degree of Consistency A graph is K-consistent if we can choose values for any K-1 variables that satisfy all the constraints, then for any Kth variable be able to assign it a value that satisfies the constraints A graph is strongly K-consistent if J-consistent for all J < K Node consistency is equivalent to strong 1-consistency Arc consistency is equivalent to strong 2-consistency

Towards Backtrack Free Search A graph that has strong n-consistency requires no search Acquiring strong n-consistency is exponential in the number of variables (Cooper, 1989) For a general graph that is strongly k-consistent (where k < n) backtracking cannot be avoided

Example V1 red green V2 V3 red green green blue (*,*,*) (r,*,*) (r,r,*) … Arc consistent, yet a search will backtrack!

Constraint Graph Width V1 V1 V2 V2 V3 V3 V1 V2 V3 V1 V3 V1 V2 V2 V3 V3 V2 V3 V1 V2 V1 1 1 1 2 1 2  1 The nodes of a constraint graph can be ordered The width of a node in an ordered graph is equal to the number of incoming arcs from higher up nodes The width of an ordered graph is the max width of its vertices The width of a constraint graph is the min width of each of its orderings

Backtrack Free Search Theorem: If a constraint graph is strongly K-consistent, and K is greater than its width, then there exists a search order that is backtrack free K>2 consistency algorithms add arcs requiring even greater consistency If a graph has width 1 we can use node and arc consistency to get strong 2-consistency without adding arcs All tree structured constraint graphs have width 1 (Freuder 1988)

Contents Representations Solving with Tree Search and Heuristics Constraint Propagation Tree Clustering

Tree Clustering Motivation Tree structured constraint graphs can be solved without backtracking We would like to turn non-tree graphs into trees by grouping variables The grouped variables themselves become smaller CSP’s Solving a CSP is exponential in the worst case so reducing the number of variables we consider at once is also important If we want the CSP for many queries it is worth investing more time in restructuring it (Dechter, 1988)

Redundancy Constraints in the dual graph are equalities Variables: A, B, C, D, E, F Constraints: (ABC), (AEF), (CDE), (ACE) A ABC AEF ACE CDE CE AC AE Join graph/tree ABC AEF C AC AE E ACE CDE CE

Tree Clustering If the dual graph cannot be reduced to a join tree we can still make it acyclic: Condition for acyclicity: A CSP is acyclic iff its primal graph is chordal and conformal Given a primal graph its dual can be made acyclic: Triangulate graph to make it chordal The maximal cliques are constraints/nodes in the new dual graph (Beeri, 1983)

Triangulation Use maximum cardinality search (m-ordering) to order the nodes Add an edge between any two nonadjacent nodes that are connected by nodes higher in the ordering (Tarjan, 1984)

The Algorithm Build the primal graph for the CSP Triangulate O(n2) Triangulate Use maximal cliques as new nodes in dual graph O(n) Remove any redundancies in the new graph

Example Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E) AD AC BD DE CE D A E C AD AC BD DE CE D A E C Still cyclic!

Example Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E) E C D A B Order: E, D, C, A, B E C D A B E C D A B

Example Variables: A, B, C, D, E Constraints: (A,C), (A,D), (B,D), (C,E), (D,E) E C D A B ACD BD CDE D CD ACD BD CDE D CD Acyclic!

Solving the CSP Solve each node of the tree as a separate small CSP This can be done in parallel The solutions to each node constitute the domain of that node in the tree O(d m) Use arc consistency to reduce the domains of each node Solve the entire CSP without backtracking

Appendix

Example CSP’s N-queens Map coloring Cryptoarithmatic Wireless network base station placement Object recognition from image features

Heuristic Repair Start with a random instantiation of variables, choose a variable and reassign it so that fewest constraints are violated Repeat this some number of times, if constraints still violated, restart with a new random instantiation Similar to GSAT

Graham’s Algorithm Given a dual constraint graph If Vi is a variable that appears in exactly one node then remove Vi If the variables in Ni are a subset of variables in another node Nj then remove Ni Repeat until neither applies Graph is acyclic if the result is the empty set Easy to verify (Graham, 1979)

Graham’s Algorithm Trees collapse to empty set Edges in dual graph are constraints on common vars Nodes in a tree share vars only with their parents Step 1 removes any unique vars from the children After step 1 the children are removed by step 2 since they are now subsets of the parent ab a b Step-1 ab ab Step-2 [ ] Step-1 ag bd

Graham’s Algorithm Cycles don’t collapse Step 1 will fail since cycles are created among nodes with vars that are shared among multiple nodes Step 2 must fail since if any node was a subset of another it would have to share all of its variables with at least one other node ab ag bg

Graham’s Algorithm  Primal Graph is Chordal Assume Graham’s algorithm succeeds Step 1 must have removed every node of the primal graph Assume there was a chordless cycle Let z be a node on the cycle that is first eliminated Let x, y be nodes on the cycle adjacent to z For step 1 to apply z must have belonged to only one constraint (which are cliques/hyperedges) With z gone the constraint is left with x and y Thus x and y are connected  contradiction!

Why m-ordering? We want an ordering that will not add any edges to chordal graphs Property P (for zero fill-in): If u < v < w If (u,w) is an edge If (v,w) is not an edge Then exists vertex x v < x (v, x) is an edge (w, x) is not an edge w u v x

Proof of Property P Assumptions Define property Q: Given chordal graph G Given an ordering a with Property P Define property Q: Let V0, V1, …, Vk be an unchorded path for which a(Vk) is maximum Vk > … > Vi+1 > Vi < … < V2 < V1 < Vk < V0 Not possible! w v u … order: u < v < w if no edge (v,w) then v, u, w satisfies property Q

M-ordering Satisfies P Suppose u < v < w w u v x