University of Toronto Mechanical & Industrial Engineering An Introduction to Constraint Programming J. Christopher Beck Dept. of Mechanical & Industrial Engineering University of Toronto Canada

University of Toronto Mechanical & Industrial Engineering

University of Toronto Mechanical & Industrial Engineering Crystal Maze Place the numbers 1 through 8 in the nodes such that: –Each number appears exactly once ? ? ? ? ? ? ?? –No connected nodes have consecutive numbers You have 5 minutes!

University of Toronto Mechanical & Industrial Engineering Constraint Satisfaction Problem (CSP) Given: –V, a set of variables {v 0, v 1, …, v n } –D, a set of domains {D 0, D 1, …, D n } –C, a set of constraints {c 0, c 1, …, c m } Each constraint, c i, has a scope c i (v 0, v 2, v 4, v 117, …), the variables that it constrains

University of Toronto Mechanical & Industrial Engineering Constraint Satisfaction Problem (CSP) A constraint, c i, is a mapping from the elements of the Cartesian product of the domains of the variables in its scope to {T,F} –c i (v 0, v 2, v 4, v 117, …) maps: (D 0 X D 2 X D 4 X D 117 X … )  {T,F} A constraint is satisfied if the assignment of the variables in its scope are mapped to T

University of Toronto Mechanical & Industrial Engineering Constraint Satisfaction Problem (CSP) In a solution to a CSP: –each variable is assigned a value from its domain: v i = d i, d i є D i –each constraint is satisfied

University of Toronto Mechanical & Industrial Engineering Crystal Maze Place the numbers 1 through 8 in the nodes such that: –Each number appears exactly once ? ? ? ? ? ? ?? –No connected nodes have consecutive numbers

University of Toronto Mechanical & Industrial Engineering Modeling Crystal Maze Each node  a variable {1, …, 8}  values in the domain of each variable No consecutive numbers  a constraint –(v i, v j )  |v i – v j | > 1 All values used  all-different constraint

University of Toronto Mechanical & Industrial Engineering Heuristic Search ? ? ? ? ? ? ?? 18 {1, 2, 3, 4, 5, 6, 7, 8}

University of Toronto Mechanical & Industrial Engineering Inference/Propagation ? ? ? ? ? ? ?? 18 {1, 2, 3, 4, 5, 6, 7, 8}

University of Toronto Mechanical & Industrial Engineering Inference/Propagation ? ? ? ? ? ? ?? 18 {3, 4, 5, 6} {3, 4, 5, 6, 7}{2, 3, 4, 5, 6}

University of Toronto Mechanical & Industrial Engineering Generic CP Algorithm Assert Constraint Propagators Start Success Solution? Make Heuristic Decision Backtrack Technique Failure Nothing to retract? Dead-end?

University of Toronto Mechanical & Industrial Engineering The Core of CP Modeling –How to represent the problem Heuristic search –How to branch –How much effort to find a good branch Backtracking Inference/propagation

University of Toronto Mechanical & Industrial Engineering Modeling How do you represent the problem with your tools? Like MIP, it’s an art –Not as “advanced” on the theory side as MIP modeling Different models of the same problem can have radically different performance

University of Toronto Mechanical & Industrial Engineering Modeling Often the first model is easy but poor –rich constraint language, “natural” –slow, scales poorly Need to think about: –symmetry, “what would inference do?”, search heuristics, adding redundancy, multiple viewpoints, … May have to build custom constraints

University of Toronto Mechanical & Industrial Engineering Heuristic Search Traditional view: –variable-ordering & value ordering More general view: –add constraints that partition the space Domain independent vs. domain dependent –min domain/degree, adaptive weights, … –e.g., heuristics for scheduling

University of Toronto Mechanical & Industrial Engineering The Importance of Heuristics N = 30, |d| = 8, 45 constraints, 26 tuples/constraint (64-26)/64 nogoods => constraint tightness ~ 0.6 Forward checking search means for 10 problems lexical min-domain Searchtree 353,599 nodes 44 nodes Time 7.36s 0.01s

University of Toronto Mechanical & Industrial Engineering Backtracking Need to follow alternative branches –dead-end or a possible sub-optimal solution Chronological backtracking (aka DFS) –simplest, most popular, dumbest Constraint-directed backjumping –keep track of info to provably jump over some branches Randomized restart –nice mathematical model

University of Toronto Mechanical & Industrial Engineering Inference/Propagation Probably the primary reason for the problem solving power of CP –main mechanism to reduce search Most mathematically/theoretically mature sub-area of CP 300+ “global constraints” in current catalog –domain-independent to specialized –stamp collecting?

University of Toronto Mechanical & Industrial Engineering Heuristic Search or Propagation? ? ? ? ? ? ? ?? 18 {1, 2, 3, 4, 5, 6, 7, 8} There must be a 1 or 8 here. Why?

University of Toronto Mechanical & Industrial Engineering Arc Consistency Fundamental notion in CP! Given: c 1 (v 1,v 2 ) –a binary constraint –e.g., v 1 < v 2 Given: D 1 = D 2 = {0, 1, …, 5}

University of Toronto Mechanical & Industrial Engineering Arc Consistency c 1 is arc consistent iff –for all values d 1 є D 1 there exists a value d 2 є D 2 such that c 1 (v 1 =d 1,v 2 =d 2 )  T –And similarly for all values d 2 є D 2 V1V2 < {0,1,2,3,4,5}

University of Toronto Mechanical & Industrial Engineering What Now? V1V2 < {0,1,2,3,4,5} V3 < < {0,1,2,3,4,5}

University of Toronto Mechanical & Industrial Engineering Generalized Arc Consistency (GAC) Given: c 1 (v 1,..., v m ) C 1 is GAC iff –for all variables d i, for all values d i є D i there exists a tuple of values [d j є D j ], j≠i such that C 1 (v i =d i,[v j =d j ])  T E.g., c 1 (v 1,v 2,v 3,v 4 ) –for every value in d 1 є D 1 there must be a triple [d 2 є D 2, d 3 є D 3, d 4 є D 4 ] s.t. c 1 (v 1 =d 1, v 2 =d 2, v 3 =d 3, v 4 =d 4 )  T

University of Toronto Mechanical & Industrial Engineering All-different Constraint all-diff(v 1, v 2, …, v n ) –all v i ’s must take on a different value –logically equivalent to Useful in many, many combinatorial problems –travelling salesman, timetabling, …

University of Toronto Mechanical & Industrial Engineering All-Diff vs. Clique of ≠ Logically, all-diff(v 1, v 2, …, v n ) = def v i ≠ v j for 1 ≤ i < j ≤ n D 1 =D 2 =D 3 ={1,3} Establish AC (or GAC) for –v 1 ≠ v 2, v 1 ≠ v 3, v 2 ≠ v 3 –all-diff(v 1,v 2,v 3 ) V1V2 ≠ {1,3} V3 ≠ ≠ {1,3} V1V2 {1,3} V3 {1,3} all-diff

University of Toronto Mechanical & Industrial Engineering All-Diff vs. Clique of ≠ So GAC on all-diff is stronger than AC on a clique of ≠ But what about the computational complexity? All-diff is a “global constraint” –GAC on some global constraints is NP- complete –What about all-diff?

University of Toronto Mechanical & Industrial Engineering GAC Propagation on All-Diff Value graph Matchings & graph theory Main Theorem Algorithm Sketch

University of Toronto Mechanical & Industrial Engineering Value Graph Example –D 1 = {B,C,D,E}, D 2 = {B,C}, D 3 = {A,B,C,D}, D 4 = {B,C} –all-diff(v 1,…,v 4 ) A B C D E

University of Toronto Mechanical & Industrial Engineering Definitions Given a graph G(V,E) A matching is a set of disjoint edges –disjoint = no common end-points M covers a set if all vertices in S are end-points of an edge in M A vertex is M-free if it isn’t covered by M

University of Toronto Mechanical & Industrial Engineering Value Graph In terms of matchings, provide a characterization of an assignment that satisfies the all- diff constraint A B C D E

University of Toronto Mechanical & Industrial Engineering Theorem [Regin94] G is a value graph All-diff(v 1,…,v n ) is GAC iff every edge in G belongs to a matching in G covering {v 1,…,v n }

University of Toronto Mechanical & Industrial Engineering A B C D E

University of Toronto Mechanical & Industrial Engineering A B C D E

University of Toronto Mechanical & Industrial Engineering A B C D E

University of Toronto Mechanical & Industrial Engineering Finding Initial Max-Matching Find a maximum matching –O(n√m), n = |V|, m =  |D i | –well-known graph theory algorithm A B C D E

University of Toronto Mechanical & Industrial Engineering Finding More Max-Matchings 1 Orient edges: –In M – from variables to values –Not in M – from values to variables Find strongly connected components –depth-first search – if you re-enter a node it is in a strongly connected component M-alternating cycle

University of Toronto Mechanical & Industrial Engineering 2 4 B C A B C D E

University of Toronto Mechanical & Industrial Engineering Finding More Max-Matchings 2 Find an M-alternating path starting from an uncovered vertex Breadth first search

University of Toronto Mechanical & Industrial Engineering A 1 E 3 D A B C D E

University of Toronto Mechanical & Industrial Engineering GAC Propagation All edges –in M –in an M-alternating circuit, or –in an M-alternating path are consistent Remove the rest

University of Toronto Mechanical & Industrial Engineering A B C D E A B C D E

University of Toronto Mechanical & Industrial Engineering Value Graph D 1 = {B,C,D,E}, D 2 = {B,C}, D 3 = {A,B,C,D}, D 4 = {B,C} All-different: –D 1 = {B,C,D,E}, D 2 = {B,C}, D 3 = {A,B,C,D}, D 4 = {B,C}

University of Toronto Mechanical & Industrial Engineering GAC All-Diff Propagation Summary Find a max-matching –O(n√m), n = |V|, m =  |D i | Find all M-alternating circuits –DFS to find strongly connected components: O(n+m) Find all M-alternating paths starting from an M-free vertex –Breadth-first search: O(m)

University of Toronto Mechanical & Industrial Engineering Global Constraints Any number of variables Accompanying efficient propagation algorithm –to enforce some level of consistency (not necessarily GAC) A custom global constraint becomes both an algorithmic and modeling object –extending the language –inference algorithm can be arbitrary –compositionality

University of Toronto Mechanical & Industrial Engineering Differences with MIP Richer, extensible language Traditionally no explicit use of lower bounds –no optimization function constraint satisfaction problems –less focus on relaxations –less true now: LBs “inside” global constraints

University of Toronto Mechanical & Industrial Engineering CP Engineering Process Model, remodel, remodel, … –if you are lucky, you’re done –abstraction? Domain-specific heuristics Custom global constraints Hybrids –CP/LP, CP/MIP, CP/Metaheuristics

University of Toronto Mechanical & Industrial Engineering When to Choose CP Finding a feasible solution is hard –intricate set of complicated constraints No tight relaxations Existing global constraints Strong back-propagation