1. Case study 3: orthogonal Latin squares Modelled by Barbara Smith

2. Modelling decisions Modelling decisions Many different ways to model even simple problems Many different ways to model even simple problems Combining models can be effective Combining models can be effective Channel between models Channel between models Need additional constraints Need additional constraints Symmetry breaking Symmetry breaking Implied (but logically) redundant Implied (but logically) redundant

3. Orthogonal Latin squares Find a pair of Latin squares Find a pair of Latin squares Every cell has a different pair of elements Generalized form: Generalized form: Find a set of m Latin squares Each possible pair is orthogonal

4. Orthogonal Latin squares 1 2 3 4 1 2 3 4 2 1 4 3 3 4 1 2 3 4 1 2 4 3 2 1 4 3 2 1 2 1 4 3 11 22 33 44 11 22 33 44 23 14 41 32 23 14 41 32 34 43 12 21 34 43 12 21 42 31 24 13 42 31 24 13 Two 4 by 4 Latin squares Two 4 by 4 Latin squares No pair is repeated No pair is repeated

5. History of (orthogonal) Latin squares Introduced by Euler in 1783 Introduced by Euler in 1783 Also called Graeco-Latin or Euler squares Also called Graeco-Latin or Euler squares No orthogonal Latin square of order 2 No orthogonal Latin square of order 2 There are only 2 (non)-isomorphic Latin squares of order 2 and they are not orthogonal There are only 2 (non)-isomorphic Latin squares of order 2 and they are not orthogonal

6. History of (orthogonal) Latin squares Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2 Euler conjectured in 1783 that there are no orthogonal Latin squares of order 4n+2 Constructions exist for 4n and for 2n+1 Constructions exist for 4n and for 2n+1 Took till 1900 to show conjecture for n=1 Took till 1900 to show conjecture for n=1 Took till 1960 to show false for all n>1 Took till 1960 to show false for all n>1 6 by 6 problem also known as the 36 officer problem 6 by 6 problem also known as the 36 officer problem … Can a delegation of six regiments, each of which sends a colonel, a lieutenant-colonel, a major, a captain, a lieutenant, and a sub-lieutenant be arranged in a regular 6 by 6 array such that no row or column duplicates a rank or a regiment?

7. More background Lams problem Lams problem Existence of finite projective plane of order 10 Existence of finite projective plane of order 10 Equivalent to set of 9 mutually orthogonal Latin squares of order 10 Equivalent to set of 9 mutually orthogonal Latin squares of order 10 In 1989, this was shown not to be possible after 2000 hours on a Cray (and some major maths) In 1989, this was shown not to be possible after 2000 hours on a Cray (and some major maths) Orthogonal Latin squares are used in experimental design Orthogonal Latin squares are used in experimental design To ensure no dependency between independent variables To ensure no dependency between independent variables

8. A simple 0/1 model Suitable for integer programming Suitable for integer programming Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise Xijkl = 1 if pair (i,j) is in row k column l, 0 otherwise Avoiding advice never to use more than 3 subscripts! Avoiding advice never to use more than 3 subscripts! Constraints Constraints Each row contains one number in each square Each row contains one number in each square Sum_jl Xijkl = 1 Sum_il Xijkl = 1 Each col contains one number in each square Each col contains one number in each square Sum_jk Xijkl = 1 Sum_ik Xijkl = 1

9. A simple 0/1 model Additional constraints Additional constraints Every pair of numbers occurs exactly once Every pair of numbers occurs exactly once Sum_kl Xijkl = 1 Every cell contains exactly one pair of numbers Every cell contains exactly one pair of numbers Sum_ij Xijkl = 1 Is there any symmetry?

10. Symmetry removal Important for solving CSPs Important for solving CSPs Especially for proofs of optimality? Especially for proofs of optimality? Orthogonal Latin square has lots of symmetry Orthogonal Latin square has lots of symmetry Permute the rows Permute the rows Permute the cols Permute the cols Permute the numbers 1 to n in each square Permute the numbers 1 to n in each square How can we eliminate such symmetry? How can we eliminate such symmetry?

11. Symmetry removal Fix first row Fix first row 11 22 33 … Fix first column Fix first column112332.. Eliminates all this symmetry? Eliminates all this symmetry?

12. What about a CSP model? Exploit large finite domains possible in CSPs Exploit large finite domains possible in CSPs Reduce number of variables Reduce number of variables O(n^4) -> ? O(n^4) -> ? Exploit non-binary constraints Exploit non-binary constraints Problem states that squares contain pairs that are all different Problem states that squares contain pairs that are all different All-different is a non-binary constraint our solvers can reason with efficiently All-different is a non-binary constraint our solvers can reason with efficiently

13. CSP model 2 sets of variables 2 sets of variables Skl = i if the 1st element in row k col l is i Skl = i if the 1st element in row k col l is i Tkl = j if the 2nd element in row k col l is j Tkl = j if the 2nd element in row k col l is j How do we specify all pairs are different? How do we specify all pairs are different? All distinct (k,l), (k,l) All distinct (k,l), (k,l) if Skl = i and Tkl = j then Skl=/ i or Tkl =/ j if Skl = i and Tkl = j then Skl=/ i or Tkl =/ j O(n^4) loose constraints, little constraint propagation! What can we do?

14. CSP model Introduce auxiliary variables Introduce auxiliary variables Fewer constraints, O(n^2) Fewer constraints, O(n^2) Tightens constraint graph => more propagation Tightens constraint graph => more propagation Pkl = i*n + j if row k col l contains the pair i,j Pkl = i*n + j if row k col l contains the pair i,j Constraints Constraints 2n all-different constraints on Skl, and on Tkl 2n all-different constraints on Skl, and on Tkl All-different constraint on Pkl All-different constraint on Pkl Channelling constraint to link Pkl to Skl and Tkl Channelling constraint to link Pkl to Skl and Tkl

15. CSP model v O/1 model CSP model CSP model 3n^2 variables 3n^2 variables Domains of size n, n and n^2+n Domains of size n, n and n^2+n O(n^2) constraints O(n^2) constraints Large and tight non- binary constraints Large and tight non- binary constraints 0/1 model 0/1 model n^4 variables Domains of size 2 O(n^4) constraints Loose but linear constraints Use IP solver!

16. Solving choices for CSP model Variables to assign Variables to assign Skl and Tkl, or Pkl? Skl and Tkl, or Pkl? Variable and value ordering Variable and value ordering How to treat all-different constraint How to treat all-different constraint GAC using Regins algorithm O(n^4) GAC using Regins algorithm O(n^4) AC using the binary decomposition AC using the binary decomposition

17. Good choices for the CSP model Experience and small instances suggest: Experience and small instances suggest: Assign the Skl and Tkl variables Assign the Skl and Tkl variables Choose variable to assign with Fail First (smallest domain) heuristic Choose variable to assign with Fail First (smallest domain) heuristic Break ties by alternating between Skl and Tkl Break ties by alternating between Skl and Tkl Use GAC on all-different constraints for Skl and Tkl Use GAC on all-different constraints for Skl and Tkl Use AC on binary decomposition of large all- different constraint on Pkl Use AC on binary decomposition of large all- different constraint on Pkl

18. Performance n 0-1 model Fails t/sec CSP model AC Fails t/sec CSP model GAC Fails t/sec 4 4 0.11 2 0.18 2 0.38 5 1950 4.05 295 1.39 190 1.55 6 ? ? 640235 657 442059 773 7* 20083 59.8 91687 51.1 57495 66.1

19. Dual CSP model 4 subscripts in 0/1 model are interchangeable 4 subscripts in 0/1 model are interchangeable Suggests a dual model Suggests a dual model DSij = k if the pair (i,j) occurs in row k DSij = k if the pair (i,j) occurs in row k DTij = l if the pair (i,j) occurs in the row l DTij = l if the pair (i,j) occurs in the row l DPij = k*n + l if the pair (i,j) occurs in row i col j DPij = k*n + l if the pair (i,j) occurs in row i col j Dual constraints to the primal model Dual constraints to the primal model

20. Combined model Primal and dual model together Primal and dual model together Channelling constraints to link them Channelling constraints to link them But new search decisions But new search decisions Do we assign both primal and dual variables? Do we assign both primal and dual variables? How do handle dual constraints (AC, GAC …)? How do handle dual constraints (AC, GAC …)? … Other dualities Other dualities Any choice of 2 subscripts from 4 Any choice of 2 subscripts from 4 Diminishing returns Diminishing returns

21. Dual performance NPrimal Fails t/sec Dual all vars Fails t/sec Dual primal vars Fails t/sec 4 2 0.38 0 0.29 0 0.046 5 190 1.55 94 0.37 67 0.34 6 442059 773 33868 146 26936 78.5 7* 57495 66.1 978 3.8 1529 5.5

22. Conclusions Many ways to model even simple problems Many ways to model even simple problems Introduce auxiliary variables Introduce auxiliary variables Reduce number of constraints, improve propagation Reduce number of constraints, improve propagation Combining models often beneficial Combining models often beneficial Channelling constraints link models Channelling constraints link models Need to deal with symmetry Need to deal with symmetry Dont always use GAC on all-different constraints Dont always use GAC on all-different constraints

23. General methodology? Choose a basic model Choose a basic model Consider auxiliary variables Consider auxiliary variables To reduce number of constraints, improve propagation To reduce number of constraints, improve propagation Consider combined models Consider combined models Channel between views Channel between views Break symmetries Break symmetries Add implied constraints Add implied constraints To improve propagation To improve propagation

24. Case study 4: template design Again model due to Barbara Smith

25. Introduction Prob002 at www.csplib.org Prob002 at www.csplib.orgwww.csplib.org Problem comes from a printing firm Problem comes from a printing firm Cat food labels that need to be printed using templates Cat food labels that need to be printed using templates Several designs (tuna, chicken, …) go on each template Several designs (tuna, chicken, …) go on each template Different demand for each flavour Different demand for each flavour Aside: where did cats get the taste for tuna? Aside: where did cats get the taste for tuna?

26. Template design problem For Francescas benefit For Francescas benefit How else can I get a cat picture into my talk? How else can I get a cat picture into my talk?

27. Basic CSP model Assume number of templates is fixed Assume number of templates is fixed Variables Variables Pij = number of slots on template i for design j Pij = number of slots on template i for design j Ri = run length for template i Ri = run length for template i Constraints Constraints Sum_j Pij = s, number of slots on each template Sum_j Pij = s, number of slots on each template Sum_i Pij * Ri >= dj, total production equals demand Sum_i Pij * Ri >= dj, total production equals demand

28. Basic CSP model Optimization problem Optimization problem Introduce variable to minimize Introduce variable to minimize Production = Sum_i Ri Production = Sum_i Ri Solved as sequence of decision problems Solved as sequence of decision problems Production < l1, Production < l2 … l1 set to minimum number of printings with 1 template

29. Symmetry Does the model have any symmetry? Does the model have any symmetry? If so, how can we eliminate it? If so, how can we eliminate it?

30. Symmetry The templates are indistinguishable and can be permuted The templates are indistinguishable and can be permuted Swap all designs on one template with all those on a second template Swap all designs on one template with all those on a second template Break this symmetry by distinguishing the templates Break this symmetry by distinguishing the templates R1 <= R2 <= R3 … R1 <= R2 <= R3 …

31. Symmetry Designs j, k with the same demand are indistinguishable Designs j, k with the same demand are indistinguishable We can break this symmetry We can break this symmetry [P1j,P2j,P3j,…] <lex [P1k,P2k,P3k,…] [P1j,P2j,P3j,…] <lex [P1k,P2k,P3k,…] Efficient GAC algorithm for lex ordering constraint Efficient GAC algorithm for lex ordering constraint

32. Sort of symmetry? Symmetries can be subtle to spot! Symmetries can be subtle to spot! Consider designs j and j with demand for j less than for j Consider designs j and j with demand for j less than for j Suppose we produce more of j than j Suppose we produce more of j than j We could swap j and j and still have solution We could swap j and j and still have solution Prevent this with constraint on production Prevent this with constraint on production Sum_i Pij Ri <= Sum_i Pij Ri Sum_i Pij Ri <= Sum_i Pij Ri

33. Implied constraints We should always look for implied constraints we can add to model We should always look for implied constraints we can add to model Encourage constraint propagation Encourage constraint propagation

34. Implied constraints 2 templates 2 templates R1+R2 = Production R1+R2 = Production R1 <= R2 R1 <= R2 Hence Hence R1 <= Production/2 R1 <= Production/2 R2 >= Production/2 R2 >= Production/2

35. Implied constraints 3 templates 3 templates R1 + R2 + R3 = Production R1 + R2 + R3 = Production R1 <= R2 <= R3 R1 <= R2 <= R3 Hence Hence R1 <= Production/3 R1 <= Production/3 R2 <= Production/2 R2 <= Production/2 R3 >= Production/3 R3 >= Production/3…

36. Solving choices Variable ordering Variable ordering As with Golomb ruler, assign variables to construct solution systematically As with Golomb ruler, assign variables to construct solution systematically Assign all designs on one template before moving on to a second template Assign all designs on one template before moving on to a second template Encourages constraint propagation on runlength constraints Encourages constraint propagation on runlength constraints

37. Performance Basic model Basic model Difficult to solve 2 or 3 template problems Difficult to solve 2 or 3 template problems Full model Full model Problem solved quickly Problem solved quickly Can solve much larger problems than feasible with the basic model Can solve much larger problems than feasible with the basic model Optimality can still be tough! Optimality can still be tough!

38. Conclusions Basic model often obvious Basic model often obvious To refine such a model we need: To refine such a model we need: Consider dual/combined models Consider dual/combined models Symmetries eliminated Symmetries eliminated Implied constraints Implied constraints Variable ordering heuristics Variable ordering heuristics Hopefully you can start to see patterns in what we do! Hopefully you can start to see patterns in what we do!