Constraints and Search Toby Walsh Cork Constraint Computation Centre (4C) Logic & AR Summer School, 2002

Constraint satisfaction Constraint satisfaction problem (CSP) is a triple where: –V is set of variables –Each X in V has set of values, D_X Usually assume finite domain {true,false}, {red,blue,green}, [0,10], … –C is set of constraints Goal: find assignment of values to variables to satisfy all the constraints

Constraint solver Tree search –Assign value to variable –Deduce values that must be removed from future/unassigned variables Constraint propagation –If any future variable has no values, backtrack else repeat Number of choices –Variable to assign next, value to assign Some important refinements like nogood learning, non-chronological backtracking, …

Constraint propagation Enfrocing arc-consistency (AC) –A binary constraint r(X1,X2) is AC iff for every value for X1, there is a consistent value (often called support) for X2 and vice versa E.g. With 0/1 domains and the constraint X1 =/= X2 Value 0 for X1 is supported by value 1 for X2 Value 1 for X1 is supported by value 0 for X2 … –A problem is AC iff every constraint is AC

Tree search Backtracking (BT) Forward checking (FC) Maintaining arc-consistency (MAC) Limited discrepany search (LDS) Non-chronological backtracking & learning

Modelling case study: orthogonal Latin squares Or constraint programming isnt purely declarative!

Modelling decisions Many different ways to model even simple problems –Its not pure declarative programming! Combining models can be effective –Channel between models Need additional constraints –Symmetry breaking –Implied (but logically) redundant

Orthogonal Latin squares Find a pair of Latin squares –Every cell has a different pair of elements Generalized form: –Find a set of m Latin squares –Each possible pair is orthogonal

Orthogonal Latin squares Two 4 by 4 Latin squares No pair is repeated

History of (orthogonal) Latin squares Introduced by Euler in 1783 –Also called Graeco-Latin or Euler squares No orthogonal Latin square of order 2 –There are only 2 (non)-isomorphic Latin squares of order 2 and they are not orthogonal

History of (orthogonal) Latin squares Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2 –Constructions exist for 4n and for 2n+1 –Took till 1900 to show conjecture for n=1 –Took till 1960 to show false for all n>1 6 by 6 problem also known as the 36 officer problem … Can a delegation of six regiments, each of which sends a colonel, a lieutenant-colonel, a major, a captain, a lieutenant, and a sub- lieutenant be arranged in a regular 6 by 6 array such that no row or column duplicates a rank or a regiment?

More background Lams problem –Existence of finite projective plane of order 10 –Equivalent to set of 9 mutually orthogonal Latin squares of order 10 –In 1989, this was shown not to be possible after 2000 hours on a Cray (and some major maths) Orthogonal Latin squares are used in experimental design –To ensure no dependency between independent variables

A simple 0/1 model Suitable for integer programming –Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise –Avoiding advice never to use more than 3 subscripts! Constraints –Each row contains one number in each square Sum_jl Xijkl = 1 Sum_il Xijkl = 1 –Each col contains one number in each square Sum_jk Xijkl = 1 Sum_ik Xijkl = 1

A simple 0/1 model Additional constraints –Every pair of numbers occurs exactly once Sum_kl Xijkl = 1 –Every cell contains exactly one pair of numbers Sum_ij Xijkl = 1 Is there any symmetry?

Symmetry removal Important for solving CSPs –Especially for proofs of optimality? Orthogonal Latin square has lots of symmetry –Permute the rows –Permute the cols –Permute the numbers 1 to n in each square How can we eliminate such symmetry?

Symmetry removal Fix first row … Fix first column Eliminates all this symmetry?

What about a CSP model? Exploit large finite domains possible in CSPs –Reduce number of variables –O(n^4) -> ? Exploit non-binary constraints –Problem states that squares contain pairs that are all different –All-different is a non-binary constraint our solvers can reason with efficiently

CSP model 2 sets of variables –Skl = i if the 1st element in row k col l is i –Tkl = j if the 2nd element in row k col l is j How do we specify all pairs are different? –All distinct (k,l), (k,l) if Skl = i and Tkl = j then Skl=/ i or Tkl =/ j O(n^4) loose constraints, little constraint propagation! What can we do?

CSP model Introduce auxiliary variables –Fewer constraints, O(n^2) –Tightens constraint graph => more propagation –Pkl = i*n + j if row k col l contains the pair i,j Constraints –2n all-different constraints on Skl, and on Tkl –All-different constraint on Pkl –Channelling constraint to link Pkl to Skl and Tkl

CSP model v O/1 model CSP model –3n^2 variables –Domains of size n, n and n^2+n –O(n^2) constraints –Large and tight non- binary constraints 0/1 model –n^4 variables –Domains of size 2 –O(n^4) constraints –Loose but linear constraints Use IP solver!

Solving choices for CSP model Variables to assign –Skl and Tkl, or Pkl? Variable and value ordering How to treat all-different constraint –GAC using Regins algorithm O(n^4) –AC using the binary decomposition

Good choices for the CSP model Experience and small instances suggest: –Assign the Skl and Tkl variables –Choose variable to assign with Fail First (smallest domain) heuristic Break ties by alternating between Skl and Tkl –Use GAC on all-different constraints for Skl and Tkl –Use AC on binary decomposition of large all- different constraint on Pkl

Performance n0-1 model Fails t/sec CSP model AC Fails t/sec CSP model GAC Fails t/sec ? *

General methodology? Choose a basic model Consider auxiliary variables –To reduce number of constraints, improve propagation Consider combined models –Channel between views Break symmetries Add implied constraints –To improve propagation

2ns case study: Langfords problem

Langfords problem Find a sequence of 8 numbers –Each number [1,4] occurs twice –Two occurrences of i are i numbers apart Unique solution –

Langfords problem L(k,n) problem –To find a sequence of k*n numbers [1,n] –Each of the k successive occrrences of i are i apart –We just saw L(2,4) Due to the mathematician Dudley Langford –Watched his son build a tower which solved L(2,3)

Langfords problem L(2,3) and L(2,4) have unique solutions L(2,4n) and L(2,4n-1) have solutions –L(2,4n-2) and L(2,4n-3) do not –Computing all solutions of L(2,19) took 2.5 years! L(3,n) –No solutions: 0<n<8, 10<n<17, 20,.. –Solutions: 9,10,17,18,19,.. A Sequence: 0,0,1,1,0,0,26,150,0,0,17792,108144,0,0, , , 0,0, ,

Basic model What are the variables?

Basic model What are the variables? Variable for each occurrence of a number X11 is 1st occurrence of 1 X21 is 1st occurrence of 2.. X12 is 2nd occurrence of 1 X22 is 2nd occurrence of 2.. Value is position in the sequence

Basic model What are the constraints? –Xij in [1,n*k] –Xij+1 = i+Xij –Alldifferent([X11,..Xn1,X12,..Xn2,..,X1k,..Xnk ])

Recipe Create a basic model –Decide on the variables Introduce auxiliary variables –For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver –Variable, value ordering

Break symmetry Does the problem have any symmetry?

Break symmetry Does the problem have any symmetry? –Of course, we can invert any sequence!

Break symmetry How do we break this symmetry?

Break symmetry How do we break this symmetry? –Many possible ways –For example, for L(3,9) Either X92 < 14 (2nd occurrence of 9 is in 1st half) Or X92=14 and X82<14 (2nd occurrence of 8 is in 1st half)

Recipe Create a basic model –Decide on the variables Introduce auxiliary variables –For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver –Variable, value ordering

What about dual model? Can we take a dual view?

What about dual model? Can we take a dual view? Of course we can, its a permutation!

Dual model What are the (dual) variables? –Variable for each position i What are the values?

Dual model What are the (dual) variables? –Variable for each position i What are the values? –If use the number at that position, we cannot use an all-different constraint –Each number occurs not once but k times

Dual model What are the (dual) variables? –Variable for each position i What are the values? –Solution 1: use values from [1,n*k] with the value i*n+j standing for the ith occurrence of j –Now want to find a permutation of these numbers subject to the distance constraint

Dual model What are the (dual) variables? –Variable for each position i What are the values? –Solution 2: use as values the numbers [1,n] –Each number occurs exactly k times –Fortunately, there is a generalization of all-different called the global cardinality constraint (gcc) for this

Global cardinality constraint Gcc([X1,..Xn],l,u) enforces values used by Xi to occur between l and u times –All-different([X1,..Xn]) = Gcc([X1,..Xn],1,1) Regins algorithm enforces GAC on Gcc in O(n^2.d) –Regins papers are tough to follow but this seems to beat his algorithm for all-different!?

Dual model What are the constraints? –Gcc([D1,…Dk*n],k,k) –Distance constraints?

Dual model What are the constraints? –Gcc([D1,…Dk*n],k,k) –Distance constraints: Di=j then Di+j+1=j

Combined model Primal and dual variables Channelling to link them –What do the channelling constraints look like?

Combined model Primal and dual variables Channelling to link them –Xij=k implies Dk=i

Solving choices? Which variables to assign? –Xij or Di

Solving choices? Which variables to assign? –Xij or Di, doesnt seem to matter Which variable ordering heuristic? –Fail First or Lex?

Solving choices? Which variables to assign? –Xij or Di, doesnt seem to matter Which variable ordering heuristic? –Fail First very marginally better than Lex How to deal with the permutation constraint? –GAC on the all-different –AC on the channelling –AC on the decomposition

Solving choices? Which variables to assign? –Xij or Di, doesnt seem to matter Which variable ordering heuristic? –Fail First very marginally better than Lex How to deal with the permutation constraint? –AC on the channelling is often best for time

Conclusions Modelling is an art but there are patterns –Develop basic model Decide on the variables and their values –Use auxiliary variables to represent constraints compactly/efficiently –Consider dual, combined and 0/1 models –Break symmetry –Add implied constraints –Customize solver for your model