Eliminating non- binary constraints Toby Walsh Cork Constraint Computation Center

Eliminating non-binary constraints Two methods Two methods Encodings: Encodings: Replace with binary constraints by introducing new vars Decompositions: Decompositions: (For restricted class of non-binary constraints) Replace with binary constraints on same variables

Meta-motivation Theoretical results informative Comparing non-binary constraint propagation with binary Suggests where non- binary constraints are valuable

Bibliography Lots of valuable results! Lots of valuable results! Bacchus & van Beek, AAAI-98 Chen, PhD thesis, UAlberta, 2000 Dechter, AAAI-90 Mohr & Masini, ECAI-88 Regin, AAAI-94 Stergiou & Walsh, AAAI-99 & IJCAI-99 Gent, Stergiou & Walsh, Artificial Intelligence 2000 Bacchus, Chen, van Beek, Walsh, Artificial Intelligence 2002 ….

Let’s start with the easy case! Decomposable constraints: Non-binary constraints that can be represented by binary constraints with introducing new variables It’s a special case that sometimes occurs about which we can be (theoretically) quite precise

Binary decompositions Certain non-binary constraints decompose into binary constraints on same vars Certain non-binary constraints decompose into binary constraints on same vars Sometimes called “network decomposable” Sometimes called “network decomposable”

Binary decompositions Two examples: Two examples: all-different(x1,x2,x3) is x1\=x2, x1\=x3, x2\=x3 all-different(x1,x2,x3) is x1\=x2, x1\=x3, x2\=x3 monotone(x1,x2,x3) is x1 < x2 < x3 monotone(x1,x2,x3) is x1 < x2 < x3 One non-example: One non-example: even(x1+x2+x3) even(x1+x2+x3) Can you see why not? Can you see why not?

Binary decompositions Theoretical comparison direct Theoretical comparison direct compare pruning of vars in binary decomposition with that in non-binary compare pruning of vars in binary decomposition with that in non-binary Empirical experiments reinforce theory Empirical experiments reinforce theory decomposing non-binary constraints can add orders of magnitude to solution cost decomposing non-binary constraints can add orders of magnitude to solution cost

Binary decompositions Upper and lower bound on FC Upper and lower bound on FC nFC1 on non-binary > FC on decomposition > nFC0 on non-binary Gaps can be exponential Gaps can be exponential Consider n-ary all-different with n-1 values nFC1 takes (n-1) branches FC on decomposition takes (n-1)! branches

Binary decompositions GAC lower bound GAC lower bound GAC on non-binary > AC on decomposition Gap again can be exponential But if we decompose too much, GAC=AC! GAC upper bound GAC upper bound In general, GAC ~ NIC, GAC ~ PIC.. BUT if decomposition to clique, NIC > GAC

Binary decompositions Tighter results provable for stricter classes Tighter results provable for stricter classes Tree decomposable constraints Tree decomposable constraints constraint graph is tree constraint graph is tree Triangle preserving constraints Triangle preserving constraints non-binary constraints on all triangles non-binary constraints on all triangles

Binary decompositions Tree decomposable constraints Tree decomposable constraints e.g. monotone(x1,x2,x3) e.g. monotone(x1,x2,x3) GAC=AC GAC=AC not surprising as AC enough to solve! not surprising as AC enough to solve! Decomposition here doesn’t lose us anything Decomposition here doesn’t lose us anything but even one cycle is enough for GAC>AC but even one cycle is enough for GAC>AC

Binary decompositions Triangle preserving decomposition Triangle preserving decomposition e.g. all-different(x1,x2,x3), quasigroups,... GAC > PIC, gap can again be exponential GAC ~ SAC, strongPC PIC is very strong consistency to be achieving at each node PIC is very strong consistency to be achieving at each node GAC can do even better than this! GAC can do even better than this! decomposition carries a very large price decomposition carries a very large price

Binary decompositions Experimental results Experimental results quasigroup completion quasigroup completion quasigroup existence quasigroup existence Quasigroup is a Latin square Quasigroup is a Latin square completion is completing partially filled square completion is completing partially filled square existence is finding one with additional properties existence is finding one with additional properties

Binary decompositions Modelling the quasigroup problem Modelling the quasigroup problem n^2 vars, each with domain of size n n^2 vars, each with domain of size n Non-binary model Non-binary model 2n all-different constraints (one for each row and column 2n all-different constraints (one for each row and column Binary decomposition Binary decomposition 2n cliques of not-equals constraints 2n cliques of not-equals constraints

Binary decompositions Quasigroup completion Quasigroup completion Gomes & Selman report “heavy-tailed” distributions Gomes & Selman report “heavy-tailed” distributions Maintaining AC on binary decomposition Maintaining AC on binary decomposition problems often take long time to solve problems often take long time to solve Maintaining GAC on all-different Maintaining GAC on all-different almost all problems trivial almost all problems trivial

Binary decompositions Quasigroup existence Quasigroup existence best paper at IJCAI-93 best paper at IJCAI-93 of interest to design theory of interest to design theory Open results first proved by computer Open results first proved by computer in some cases, only ever proved by computer in some cases, only ever proved by computer Maintaining GAC very competitive Maintaining GAC very competitive compared to specialized model finders like FINDER, SEM compared to specialized model finders like FINDER, SEM

Binary encodings Not all non-binary constraint decompose into binary constraints Not all non-binary constraint decompose into binary constraints on the same set of variables on the same set of variables Consider again Consider again even(x1+x2+x3) even(x1+x2+x3) But binary CSPs NP-complete But binary CSPs NP-complete

Binary encodings Every non-binary constraint can be encoded into binary constraints Every non-binary constraint can be encoded into binary constraints using polynomial number of additional (hidden) variables using polynomial number of additional (hidden) variables Two popular encodings Two popular encodings hidden variable encoding hidden variable encoding dual encoding dual encoding

Binary encodings Hidden variable encoding Hidden variable encoding add hidden var for each non-binary constraint add hidden var for each non-binary constraint Dual encoding Dual encoding add hidden var for each non-binary constraint add hidden var for each non-binary constraint throw away original variables throw away original variables

Binary encodings Dual encoding Dual encoding consider c1:even(x1+x2), c2:odd(x2+x3) consider c1:even(x1+x2), c2:odd(x2+x3) {00,11} {01,10} c1 c2 R21 R21= or

Binary encodings Hidden variable encoding Hidden variable encoding consider c1:even(x1+x2), c2:odd(x2+x3) consider c1:even(x1+x2), c2:odd(x2+x3) {0,1} {00,11} {01,10} {0,1} x1x2x3 c1 c2 r1 r2 r1= or r2= or

Binary encodings Hidden var encoding -> dual encoding Hidden var encoding -> dual encoding compose relations, discard original vars compose relations, discard original vars {0,1} {00,11} {01,10} {0,1} x1x3 c1 c2 R21 R21 = r2 + r1

Binary encodings Double encoding Double encoding dual + hidden var encoding dual + hidden var encoding original vars + hidden vars original vars + hidden vars all constraints of dual and of hidden all constraints of dual and of hidden Also called “combined encoding” Also called “combined encoding”

Binary encodings Theoretical analysis complicated by: Theoretical analysis complicated by: encoding “builds” in GAC for hidden vars encoding “builds” in GAC for hidden vars must translate between (original and hidden) vars must translate between (original and hidden) vars pruning in dual can infer large arity nogoods in original pruning in dual can infer large arity nogoods in original

Binary encodings Hidden var encoding Hidden var encoding FC on hidden ~ nFC0 on original each can be exponentially better than the other FC+ propogates through hidden vars FC+ propogates through hidden vars FC+ on hidden = nFC1 on original

Binary encodings Hidden var encoding Hidden var encoding AC on hidden = GAC on original Before looking for efficient (specialized) GAC algorithm Before looking for efficient (specialized) GAC algorithm try AC on hidden var encoding try AC on hidden var encoding

Binary encodings No point doing NIC on hidden var encoding No point doing NIC on hidden var encoding NIC on hidden = AC on hidden due to star shaped topology of constraint graph Higher consistencies remain distinct Higher consistencies remain distinct strongPC on hidden > SAC on hidden > NIC, AC on hidden

Binary encodings Dual encoding Dual encoding FC on dual ~ nFC0 on original each can be exponentially better than the other Dual better for tight constraints Dual better for tight constraints domains for hidden vars then small domains for hidden vars then small

Binary encodings Dual encoding Dual encoding AC on dual > GAC on original BUT domains of hidden vars very large when non-binary constraints loose BUT domains of hidden vars very large when non-binary constraints loose AC on dual prohibitively expensive AC on dual prohibitively expensive

Binary encodings Dual v hidden encoding Dual v hidden encoding BT on dual = FC on hidden AC on dual > AC on hidden SAC on dual > SAC on hidden strongPC on dual = strongPC on hidden Path consistency is enough to get through the hidden star! Path consistency is enough to get through the hidden star!

Binary encodings Experimental results Experimental results crossword puzzles crossword puzzles Golomb rulers Golomb rulers random CSPs random CSPs random 3-SAT random 3-SAT

Binary encodings Crossword puzzles Crossword puzzles “Original” model “Original” model vars for letters, domains {A-Z} vars for letters, domains {A-Z} Dual model Dual model vars for words, domains = dictionary vars for words, domains = dictionary Dual at least 1,000 times faster on larger problems Dual at least 1,000 times faster on larger problems

Binary encodings Golomb ruler Golomb ruler set of ticks, xi on a ruler set of ticks, xi on a ruler all inter-tick distances, dij different all inter-tick distances, dij different Ternary constraints, dij=xi-xj Ternary constraints, dij=xi-xj encoded using hidden var/double encoding encoded using hidden var/double encoding Competitive with non-binary model Competitive with non-binary model runs on any binary CSP solver! runs on any binary CSP solver!

Binary encodings Random CSPs and SAT Random CSPs and SAT Bacchus and van Beek plot contours to show constraint tightness where encoding pays Bacchus and van Beek plot contours to show constraint tightness where encoding pays gives good predictions for performance on structured problems like crosswords, … gives good predictions for performance on structured problems like crosswords, … number of satisfying tuples in constraint is most important factor in such predictions number of satisfying tuples in constraint is most important factor in such predictions

Conclusions Non-binary v binary decompositions Non-binary v binary decompositions GAC on non-binary can be stronger than PIC on decomposition GAC on non-binary can be stronger than PIC on decomposition Non-binary v binary encodings Non-binary v binary encodings GAC on non-binary = AC on hidden GAC on non-binary = AC on hidden AC on dual > GAC on non-binary AC on dual > GAC on non-binary

Conclusions Non-binary v binary decompositions Non-binary v binary decompositions decomposition can add significantly to search cost decomposition can add significantly to search cost Non-binary v binary encodings Non-binary v binary encodings encoding pays in practice on tight constraints encoding pays in practice on tight constraints

Future directions? Mixed models Mixed models e.g. dual encoding for only some of the (non- binary) constraints e.g. dual encoding for only some of the (non- binary) constraints Optimization Optimization objective function typically non-binary objective function typically non-binary