An Overview of Constraint Programming MSORM 2000 Tutorial November 20/21, 2000 Martin Henz, National University of Singapore

MSORM 2000 Constraint Programming Tutorial2 Material and Other References  Tutorial Material  Slides (distributed in conference bag)  Henz, Müller: An Overview of Finite Domain Constraint Programming (distributed in conference bag)  Programs and slides available at  Oz and Mozart  Programming language Oz  developed by Programming Systems Lab  Programming system Mozart  developed by Mozart Consortium  more information at

MSORM 2000 Constraint Programming Tutorial3 Initial Remarks  Here, we focus on Finite Domain Constraint Programming CP(FD); CP initially conceived as framework CLP(X) [Jaffar, Lassez 1987]  Communication problems CS  OR; Program  Program [Lustig, Puget 1999]  Within CP(FD), we focus on constraint-based tree search  Programming language  software framework

MSORM 2000 Constraint Programming Tutorial4 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling I II

MSORM 2000 Constraint Programming Tutorial5 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial6 Constraint Programming in a Nutshell SEND MORE MONEY

MSORM 2000 Constraint Programming Tutorial7 Constraint Programming in a Nutshell SEND + MORE = MONEY

MSORM 2000 Constraint Programming Tutorial8 SEND + MORE = MONEY Assign distinct digits to the letters S, E, N, D, M, O, R, Y such that S E N D + M O R E = M O N E Y holds.

MSORM 2000 Constraint Programming Tutorial9 SEND + MORE = MONEY Assign distinct digits to the letters S, E, N, D, M, O, R, Y such that S E N D + M O R E = M O N E Y holds. Solution =

MSORM 2000 Constraint Programming Tutorial10 Modeling Formalize the problem as a constraint problem:  number of variables: n  constraints: c 1,…,c m   n  problem: Find a = (v 1,…,v n )   n such that a  c i, for all 1  i  m

MSORM 2000 Constraint Programming Tutorial11 A Model for MONEY  number of variables: 8  constraints: c 1 = {(S,E,N,D,M,O,R,Y)   8 | 0  S,…,Y  9 } c 2 = {(S,E,N,D,M,O,R,Y)   8 | 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y}

MSORM 2000 Constraint Programming Tutorial12 A Model for MONEY (continued)  more constraints c 3 = {(S,E,N,D,M,O,R,Y)   8 | S  0 } c 4 = {(S,E,N,D,M,O,R,Y)   8 | M  0 } c 5 = {(S,E,N,D,M,O,R,Y)   8 | S…Y all different }

MSORM 2000 Constraint Programming Tutorial13 Solution for MONEY c 1 = {(S,E,N,D,M,O,R,Y)   8 | 0  S,…,Y  9 } c 2 = {(S,E,N,D,M,O,R,Y)   8 | 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y} c 3 = {(S,E,N,D,M,O,R,Y)   8 | S  0 } c 4 = {(S,E,N,D,M,O,R,Y)   8 | M  0 } c 5 = {(S,E,N,D,M,O,R,Y)   8 | S…Y all different } Solution: (9,5,6,7,1,0,8,2)   8

MSORM 2000 Constraint Programming Tutorial14 Elements of Constraint Programming Exploiting constraints during tree search  Use propagation algorithms for constraints  Employ branching algorithm  Execute exploration algorithm

MSORM 2000 Constraint Programming Tutorial15 S E N D + M O R E = M O N E Y S   E   N   D   M   O   R   Y   0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial16 S E N D + M O R E = M O N E Y Propagate S  {0..9} E  {0..9} N  {0..9} D  {0..9} M  {0..9} O  {0..9} R  {0..9} Y  {0..9} 0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial17 S E N D + M O R E = M O N E Y Propagate S  {1..9} E  {0..9} N  {0..9} D  {0..9} M  {1..9} O  {0..9} R  {0..9} Y  {0..9} 0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial18 S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} Propagate 0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial19 S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 Branch 0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial20 0  S,…,Y  9 S  0 M  0 S…Y all different 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 Propagate

MSORM 2000 Constraint Programming Tutorial21 S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 Branch S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 5 E  5 S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8}

MSORM 2000 Constraint Programming Tutorial22 S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 Propagate S  {9} E  {6..7} N  {7..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 5 E  5 S  {9} E  {5} N  {6} D  {7} M  {1} O  {0} R  {8} Y  {2}

MSORM 2000 Constraint Programming Tutorial23 S E N D + M O R E = M O N E Y Complete Search Tree S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 S  {9} E  {6..7} N  {7..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 5 E  5 S  {9} E  {5} N  {6} D  {7} M  {1} O  {0} R  {8} Y  {2} E = 6 E  6

MSORM 2000 Constraint Programming Tutorial24 Relation to Integer Programming  More general notion of problem; constraints can be any relation, not just arithmetic or even just linear arithmetic constraints  De-emphasize optimization (optimization as after-thought)  Focus on software engineering  no push-button solver, but glass-box or no-box  experimentation platforms  extensive support for “performance debugging”

MSORM 2000 Constraint Programming Tutorial25 Constraint Programming Systems Role: support elements of constraint programming  Provide propagation algorithms for constraints  all different (e.g. wait for fixing)  summation (e.g. interval consistency)  Allow choice of branching algorithm (e.g. first-fail)  Allow choice of exploration algorithm (e.g. depth-first search)

MSORM 2000 Constraint Programming Tutorial26 Using Oz proc {Money Root} S E N D M O R Y in Root=sol(s:S e:E n:N d:D m:M o:O r:R y:Y) Root ::: 0#9 {FD.distinct Root} S \=: 0 M \=: *S + 100*E + 10*N + D *M + 100*O + 10*R + E =: 10000*M *O + 100*N + 10*E + Y {FD.distribute ff Root} end {ExploreAll Money} Modeling Propagation Branching Exploration MONEY Demo

MSORM 2000 Constraint Programming Tutorial27 The Art of Constraint Programming  Choose model  Choose propagation algorithms  Choose branching algorithm  Choose exploration algorithm

MSORM 2000 Constraint Programming Tutorial28 Programming Systems for Finite Domain Constraint Programming  Finite domain constraint programming libraries  PECOS [Puget 1992]  ILOG Solver [Puget 1993]  Finite domain constraint programming languages  CHIP [Dincbas, Hentenryck, Simonis, Aggoun 1988]  SICStus Prolog [Haridi, Carlson 1995]  Oz [Smolka and others 1995]  CLAIRE [Caseau, Laburthe 1996]  OPL [van Hentenryck 1998]

MSORM 2000 Constraint Programming Tutorial29 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial30 Constraint Problems A finite domain constraint problem consists of:  number of variables: n  constraints: c 1,…,c m   n The problem is to find a = (v 1,…,v n )   n such that a  c i, for all 1  i  m

MSORM 2000 Constraint Programming Tutorial31 Constraint Solving Given: a satisfiable constraint C and a new constraint C’. Constraint solving means deciding whether C  C’ is satisfiable. Example: C: n > 2 C’: a n + b n = c n

MSORM 2000 Constraint Programming Tutorial32 Constraint Solving Constraint solving is not possible for general constraints. Constraint programming separates constraints into  basic constraints: complete constraint solving  non-basic constraints: propagation (incomplete); search needed

MSORM 2000 Constraint Programming Tutorial33 Basic Constraints in Finite Domain Constraint Programming  Basic constraints are conjunctions of constraints of the form X  S, where S is a finite set of integers.  Constraint solving is done by intersecting domains. Example: C = ( X  {1..10}  Y  {9..20} ) C’ = ( X  {9..15}  Y  {14..30} )  In practice, we keep a solved form, storing the current domain of every variable.

MSORM 2000 Constraint Programming Tutorial34 Basic Constraints and Propagators S  {1..9} E  {0..9} N  {0..9} D  {0..9} M  {1..9} O  {0..9} R  {0..9} Y  {0..9} all_different(S,E,N,D, M,O,R,Y) 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial35 Basic Constraints and Propagators S  {1..9} E  {0..9} N  {0..9} D  {0..9} M  {1} O  {0..9} R  {0..9} Y  {0..9} all different(S,E,N,D, M,O,R,Y) 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial36 Basic Constraints and Propagators S  {2..9} E  {0,2..9} N  {0,2..9} D  {0,2..9} M  {1} O  {0,2..9} R  {0,2..9} Y  {0,2..9} all different(S,E,N,D, M,O,R,Y) 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial37 Basic Constraints and Propagators S  {2..9} E  {0,2..9} N  {0,2..9} D  {0,2..9} M  {1} O  {0} R  {0,2..9} Y  {0,2..9} all different(S,E,N,D, M,O,R,Y) 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y and so on

MSORM 2000 Constraint Programming Tutorial38 Basic Constraints and Propagators S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} all different(S,E,N,D, M,O,R,Y) 1000*S + 100*E + 10*N + D *M + 100*O + 10*R + E = 10000*M *O + 100*N + 10*E + Y

MSORM 2000 Constraint Programming Tutorial39 Issues in Propagation  Expressivity: What kind of information can be expressed as propagators?  Completeness: What behavior can be expected from propagation?  Efficiency: How much computational resources does propagation consume?

MSORM 2000 Constraint Programming Tutorial40 Completeness of Propagation  Given: Basic constraint C and propagator P.  Propagation is complete, if for every variable x and every value v in the domain of x, there is an assignment in which x=v that satisfies C and P.  Complete propagation is also called domain-consistency or arc-consistency.

MSORM 2000 Constraint Programming Tutorial41 Completeness of Propagation  General arithmetic constraints are undecidable (Hilbert’s Tenth Problem).  Propagation cannot always exhibit all inconsistencies.  Example: c 1 : n > 2 c 2 : a n + b n = c n

MSORM 2000 Constraint Programming Tutorial42 Example: Complete All Different  C: w  {1,2,3,4} x  { 2,3,4} y  { 2,3,4} z  { 2,3,4}  P: all_different(w,x,y,z)

MSORM 2000 Constraint Programming Tutorial43 Example: Complete All Different  C: w  {1,2,3,4} x  { 2,3,4} y  { 2,3,4} z  { 2,3,4}  P: all_different(w,x,y,z)  Most efficient known algorithm: O(|X| 2 d max 2 ) Regin [1994], using graph matching

MSORM 2000 Constraint Programming Tutorial44 Basic Constraints vs. Propagators  Basic constraints  are conjunctions of constraints of the form X  S, where S is a finite set of integers  Enjoy complete constraint solving  Propagators  can be arbitrarily expressive (arithmetic, symbolic)  implementation typically fast but incomplete

MSORM 2000 Constraint Programming Tutorial45 Propagation vs Branching Obvious trade-off: Complex propagation algorithms result in fewer, but more expensive nodes in the search tree. Example: MONEY with alldiff and sum : only test fixed assignment alldiff : wait for fixed variables sum : interval cons. alldiff and sum : domain consistency

MSORM 2000 Constraint Programming Tutorial46 Some Propagator Classes  Symbolic propagators  Arithmetic propagators  Scheduling propagators  Reification

MSORM 2000 Constraint Programming Tutorial47 Symbolic Propagators Example: The “Element” Propagator Oz: {FD.element I Vector X} Meaning: X is the I th element of Array Example: A = [ ] I  {0,9} X  {0,9} Propagation in both directions: I  {1,3}  X  {5,7} X  {7}  I  {4}

MSORM 2000 Constraint Programming Tutorial48 Arithmetic Propagators General arithmetic equations: I 1 *X 11 *…*X 1m 1 + … + I n *X n1 *…*X nm n =: 0 : = =: \=:

MSORM 2000 Constraint Programming Tutorial49 Scheduling Propagators Details later. Scheduling constraint classes:  resource constraints  precedence constraints

MSORM 2000 Constraint Programming Tutorial50 Reified Constraints Reflecting the validity of a constraint in a 0/1 variable Example: Reified arithmetic equations X = (I 1 *X 11 *…*X 1m 1 + … + I n *X n1 *…*X nm n =: 0) : = =: \=:

MSORM 2000 Constraint Programming Tutorial51 Excursion: From Propagation to Problem Solving see Part II

MSORM 2000 Constraint Programming Tutorial52 Branching Algorithms Constraint programming systems provide  libraries of predefined branching algorithms  programming support for user-defined branching algorithms

MSORM 2000 Constraint Programming Tutorial53 Branching for MONEY proc {Money Root} S E N D M O R Y in Root=sol(s:S e:E n:N d:D m:M o:O r:R y:Y) Root ::: 0#9 {FD.distinct Root} S \=: 0 M \=: *S + 100*E + 10*N + D *M + 100*O + 10*R + E =: 10000*M *O + 100*N + 10*E + Y {FD.distribute ff Root} end {ExploreAll Money} Modeling Propagation Branching Exploration

MSORM 2000 Constraint Programming Tutorial54 Basic Choice Points choice x =: y end x < y x >= y

MSORM 2000 Constraint Programming Tutorial55 Choice Point Sequences choice x =: y end choice z = 1 [] z = 2 end x <: y x >=: y z = 1z = 2z = 1z = 2

MSORM 2000 Constraint Programming Tutorial56 Examples of Branching Algorithms  Enumeration: Choose variable, choose value  naive enumeration: choose variables and values in a fixed sequence  first-fail enumeration: choose a variable with minimal domain size  Domain-splitting: choice X =: Mid end  Task sequencing for scheduling (later)

MSORM 2000 Constraint Programming Tutorial57 Excursion: Branching Algorithms in Oz see Part II

MSORM 2000 Constraint Programming Tutorial58 Exploration Algorithms Combinations of aspects such as:  Order of exploration  Interaction  Optimization  Visualization

MSORM 2000 Constraint Programming Tutorial59 Exploration for MONEY proc {Money Root} S E N D M O R Y in Root=sol(s:S e:E n:N d:D m:M o:O r:R y:Y) Root ::: 0#9 {FD.distinct Root} S \=: 0 M \=: *S + 100*E + 10*N + D *M + 100*O + 10*R + E =: 10000*M *O + 100*N + 10*E + Y {FD.distribute ff Root} end {ExploreAll Money} Modeling Propagation Branching Exploration

MSORM 2000 Constraint Programming Tutorial60 Order of Exploration  Depth-first search  Iterative Deepening  Limited discrepancy search [Harvey/Ginsberg 95]

MSORM 2000 Constraint Programming Tutorial61 Interaction  First-solution search  All solution search  Last solution search  Search with user interaction

MSORM 2000 Constraint Programming Tutorial62 Optimization  Branch-and-bound  Restart optimization

MSORM 2000 Constraint Programming Tutorial63 Visualization Example: Oz Explorer [Schulte 1997] Oz Explorer combines  visualization  first/all solution / user interaction  branch-and-bound optimization  depth-first search

MSORM 2000 Constraint Programming Tutorial64 Excursion: Exploration Algorithms in Oz see Part II

MSORM 2000 Constraint Programming Tutorial65 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial66 ACC 1997/98: A Success Story of Constraint Programming  Integer programming + enumeration, 24 hours Nemhauser, Trick: Scheduling a Major College Basketball Conference, Operations Research, January 1998, 46(1)  Constraint programming, less than 1 minute Henz: Scheduling a Major College Basketball Conference - Revisited, Operations Research, January 2001, 49(1)

MSORM 2000 Constraint Programming Tutorial67 Round Robin Tournament Planning Problems  n teams, each playing a fixed number of times r against every other team  r = 1: single, r = 2: double round robin.  Each match is home match for one and away match for the other  Dense round robin:  At each date, each team plays at most once.  The number of dates is minimal.

MSORM 2000 Constraint Programming Tutorial68 The ACC 1997/98 Problem  9 teams participate in tournament  Dense double round robin:  there are 2 * 9 dates  at each date, each team plays either home, away or has a “bye”  Alternating weekday and weekend matches

MSORM 2000 Constraint Programming Tutorial69 The ACC 1997/98 Problem (cont’d)  No team can play away on both last dates.  No team may have more than two away matches in a row.  No team may have more than two home matches in a row.  No team may have more than three away matches or byes in a row.  No team may have more than four home matches or byes in a row.

MSORM 2000 Constraint Programming Tutorial70 The ACC 1997/98 Problem (cont’d)  Of the weekends, each team plays four at home, four away, and one bye.  Each team must have home matches or byes at least on two of the first five weekends.  Every team except FSU has a traditional rival. The rival pairs are Clem-GT, Duke-UNC, UMD- UVA and NCSt-Wake. In the last date, every team except FSU plays against its rival, unless it plays against FSU or has a bye.

MSORM 2000 Constraint Programming Tutorial71 The ACC 1997/98 Problem (cont’d)  The following pairings must occur at least once in dates 11 to 18: Duke-GT, Duke-Wake, GT- UNC, UNC-Wake.  No team plays in two consecutive dates away against Duke and UNC. No team plays in three consecutive dates against Duke UNC and Wake.  UNC plays Duke in last date and date 11.  UNC plays Clem in the second date.  Duke has bye in the first date 16.

MSORM 2000 Constraint Programming Tutorial72 The ACC 1997/98 Problem (cont’d)  Wake does not play home in date 17.  Wake has a bye in the first date.  Clem, Duke, UMD and Wake do not play away in the last date.  Clem, FSU, GT and Wake do not play away in the fist date.  Neither FSU nor NCSt have a bye in the last date.  UNC does not have a bye in the first date.

MSORM 2000 Constraint Programming Tutorial73 Nemhauser/Trick Solution  Enumerate home/away/bye patterns  explicit enumeration (very fast)  Compute pattern sets  integer programming (below 1 minute)  Compute abstract schedules [Schreuder 1992]  integer programming (several minutes)  Compute concrete schedules  explicit enumeration (approx. 24 hours)

MSORM 2000 Constraint Programming Tutorial74 Constraint Programming Solution  Enumerate home/away/bye patterns  constraint programming (very fast)  Compute pattern sets  constraint programming (below 10 seconds)  Assign teams to patterns  constraint programming (below 30 seconds)  Compute concrete schedules  constraint programming (below 20 seconds)

MSORM 2000 Constraint Programming Tutorial75 Modeling ACC 97/98 as Constraint Satisfaction Problem Variables  9 * 18 variables taking values from {0,1} that express which team plays home when. Example: H UNC, 5 =1 means UNC plays home on date 5.  away, bye similar, e.g. A UNC, 5 or B UNC, 5  9 * 18 variables taking values from {0,1,...,9} that express against which team which other team plays. Example:  UNC, 5 =1 means UNC plays team 1 (Clem) on date 5

MSORM 2000 Constraint Programming Tutorial76 Assessment  Naive branching algorithm suffices  Naive exploration algorithm suffices  Mission-critical component: Propagation algorithms!  Element propagator for assigning patterns to teams  All-different propagator for opponents in one season  Symmetric all-different propagator for opponents in one round  More references (see )  Constraint-based Round Robin Scheduling, ICLP 99  Global Constraints for Round Robin Tournaments, draft

MSORM 2000 Constraint Programming Tutorial77 Friar Tuck  Constraint programming tool for sport scheduling, ACC 97/98 just one instance  Used by sports leagues in US and Europe  Convenient entry of constraints through GUI  Friar Tuck 1.1 available for Unix and Windows 95/98 ( )  Implementation language: Oz using Mozart Friar Tuck Demo

MSORM 2000 Constraint Programming Tutorial78 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Concluding Part I  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling I II

MSORM 2000 Constraint Programming Tutorial79 Assessment: Don’t Use It! Don’t use constraint programming for:  Problems for which there are known efficient algorithms or heuristics. Example: Traveling salesman.  Problems for which integer programming works well, such as many discrete assignment problems.  Problems with weak constraints and a complex optimization function, such as many timetabling problems.

MSORM 2000 Constraint Programming Tutorial80 Assessment: Do Use It! Use constraint programming for:  Problems for which integer programming does not work (linear models too large) and for which there are no efficient heuristic solutions available.  Problems with tight constraints, where propagation can be employed. Example: ACC 97/98.  Problems for which strong branching algorithms exist. Example: Scheduling with unary resources.  Problems that need customized branching or search algorithms, extensive performance debugging or tight software integration

MSORM 2000 Constraint Programming Tutorial81 Myths Debunked  A fad! Constraint programming has been used successfully in a number of application areas, most spectacularly in scheduling  Universal! More failure stories than success stories.  All new! Many ideas come from AI search and Operations Research. In particular, the important ideas for scheduling come from OR.  Artificial Intelligence! Mostly quite earthly algorithms. Constraint programming systems provide software framework for these algorithms to interact with each other.

MSORM 2000 Constraint Programming Tutorial82 The Future  Constraint programming will become a standard technique in OR for solving combinatorial problems, along with local search and integer programming.  Constraint programming techniques will be tightly integrated with integer programming and local search.  Specific propagation algorithms will be developed to cater for important application domains.

MSORM 2000 Constraint Programming Tutorial83 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial84 Excursion: From Propagation to Problem Solving Start Oz Programming Interface

MSORM 2000 Constraint Programming Tutorial85 Excursion: Branching Algorithms in Oz Start Oz Programming Interface

MSORM 2000 Constraint Programming Tutorial86 Excursion: Exploration Algorithms in Oz Start Oz Programming Interface

MSORM 2000 Constraint Programming Tutorial87 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial88 Constraint Programming Techniques  Symmetry Breaking  Redundant Constraints  Parameterized Problems

MSORM 2000 Constraint Programming Tutorial89 Symmetry Breaking Often, the most efficient model admits many different solutions that are essentially the same (“symmetric” to each other). Symmetry breaking tries to improve the performance of search by eliminating such symmetries.

MSORM 2000 Constraint Programming Tutorial90 Redundant Constraints  Pruning of original model is often not sufficient  Adding redundant constraints sometimes helps

MSORM 2000 Constraint Programming Tutorial91 Example: Grocery Puzzle A kid goes into a grocery store and buys four items. The cashier charges $7.11. The kid pays and is about to leave when the cashier calls the kid back, and says “Hold on, I multiplied the four items instead of adding them; I’ll try again… Gosh, with adding them the price still comes to $7.11”! What were the prices of the four items?

MSORM 2000 Constraint Programming Tutorial92 Model for Grocery Puzzle  Variables A, B, C, D represent prices of items in cents.  Constraints:  A + B + C + D = 711  A * B * C * D = 711 * 100 * 100 * 100

MSORM 2000 Constraint Programming Tutorial93 Additional Constraints B  C  D 79 is prime factor of 711. Thus without loss of generality: A divisible by 79.  Redundant constraint  Symmetries

MSORM 2000 Constraint Programming Tutorial94 Solution to Grocery Puzzle Grocery plus Redundancy Grocery plus Redundancy plus Symmetry

MSORM 2000 Constraint Programming Tutorial95 Example: Fractions Find distinct non-zero digits such that the following equation holds: A D G + + = 1 B C E F H I

MSORM 2000 Constraint Programming Tutorial96 Model for Fractions  One variable for each letter, similar to MONEY  Constraint A*E*F*H*I + D*B*C*H*I + G*B*C*E*F = B*C*E*F*H*I

MSORM 2000 Constraint Programming Tutorial97 Additional Constraints A D G  B C E F H I A 3 *  1 B C G 3 *  1 H I  Symmetries  Redundant constraints

MSORM 2000 Constraint Programming Tutorial98  Constraint A*E*F*H*I + D*B*C*H*I + G*B*C*E*F = B*C*E*F*H*I  Symmetries A*E*F >= D*B*C D*H*I >= G*E*F  Redundant Constraints 3*A >= B*C 3*G =< H*I Fractions plus Symmetries Fractions plus Symmetries plus Redundancies

MSORM 2000 Constraint Programming Tutorial99 Redundant Constraints Adding redundant constraints sometimes results in dramatic performance improvements.

MSORM 2000 Constraint Programming Tutorial100 Performance of Symmetry Breaking  All solution search: Symmetry breaking usually improves performance; often dramatically  One solution search: Symmetry breaking may or may not improve performance

MSORM 2000 Constraint Programming Tutorial101 Parameterized Problems  Modeling facilities in Oz are embedded in advanced programming language.  Problem scripts can be result of computation.  Applications:  Embedding problem solving in complex applications  Parameterized problems Excursion: N Queens Start Oz Programming Interface

MSORM 2000 Constraint Programming Tutorial102 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Excursions: Constraint Programming in Oz  Constraint Programming Techniques  Constraint-based Scheduling

MSORM 2000 Constraint Programming Tutorial103 Review Constraint programming is a framework for integrating three families of algorithms  Propagation algorithms  Branching algorithms  Exploration algorithms

MSORM 2000 Constraint Programming Tutorial104 S E N D + M O R E = M O N E Y S  {9} E  {4..7} N  {5..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} S  {9} E  {5..7} N  {6..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 4 E  4 S  {9} E  {6..7} N  {7..8} D  {2..8} M  {1} O  {0} R  {2..8} Y  {2..8} E = 5 E  5 S  {9} E  {5} N  {6} D  {7} M  {1} O  {0} R  {8} Y  {2} E = 6 E  6

MSORM 2000 Constraint Programming Tutorial105 Optimization  Modeling: define optimization function  Propagation algorithms: identify propagation algorithms for optimization function  Branching algorithms: identify branching algorithms that lead to good solutions early  Exploration algorithms: extend existing exploration algorithms to achieve optimization

MSORM 2000 Constraint Programming Tutorial106 Optimization: Example SEND + MOST = MONEY

MSORM 2000 Constraint Programming Tutorial107 SEND + MOST = MONEY Assign distinct digits to the letters S, E, N, D, M, O, T, Y such that S E N D + M O S T = M O N E Y holds and M O N E Y is maximal.

MSORM 2000 Constraint Programming Tutorial108 Modeling Formalize the problem as a constraint optimization problem:  Number of variables: n  Constraints: c 1,…,c m   n  Optimization constraints: d 1,…,d m   n  2  n Given a solution a, and an optimization constraint d i, the constraint d i (a)   n contains only assignments that are better than a.

MSORM 2000 Constraint Programming Tutorial109 A Model for MONEY  number of variables: 8  constraints: c 1 = {(S,E,N,D,M,O,T,Y)   8 | 0  S,…,Y  9 } c 2 = {(S,E,N,D,M,O,T,Y)   8 | 1000*S + 100*E + 10*N + D *M + 100*O + 10*S + T = 10000*M *O + 100*N + 10*E + Y}

MSORM 2000 Constraint Programming Tutorial110 A Model for MONEY (continued)  more constraints c 3 = {(S,E,N,D,M,O,T,Y)   8 | S  0 } c 4 = {(S,E,N,D,M,O,T,Y)   8 | M  0 } c 5 = {(S,E,N,D,M,O,T,Y)   8 | S…Y all different }  optimization constraint: d : (s,e,n,d,m,o,t,y)  {(S,E,N,D,M,O,T,Y)   8 | 10000*m *o + 100*n + 10*e + y < 10000*M *O + 100*N + 10*E + Y }

MSORM 2000 Constraint Programming Tutorial111 Propagation Algorithms Identify a propagation algorithm to implement the optimization constraints Example: SEND + MOST = MONEY d : (s,e,n,d,m,o,t,y)  {(S,E,N,D,M,O,T,Y))   8   8 | 10000*m *o + 100*n + 10*e + y < 10000*M *O + 100*N + 10*E + Y } Given a solution a, choose propagation algorithm for d(a).

MSORM 2000 Constraint Programming Tutorial112 Branching Algorithms Identify a branching algorithm that finds good solutions early. Example: SEND + MOST = MONEY Idea: Naive enumeration in the order M, O, N, E, Y. Try highest values first.

MSORM 2000 Constraint Programming Tutorial113 Exploration Algorithms Modify exploration such that for each solution a, corresponding optimization constraints are added. Two strategies:  branch-and-bound: continue as in original exploration  restart optimization: after each solution, start from the root.

MSORM 2000 Constraint Programming Tutorial114 Using Oz proc {Money Root} S E N D M O R Y in Root=sol(s:S e:E n:N d:D m:M o:O r:R y:Y) Root ::: 0#9 {FD.distinct Root} S \=: 0 M \=: *S + 100*E + 10*N + D *M + 100*O + 10*R + E =: 10000*M *O + 100*N + 10*E + Y {FD.distribute ff Root} end proc {OptimizationConstraint Old New} sol(s:S1 e:E1 n:N1 d:D1 m:M1 o:O1 t:T1 y:Y1) = Old sol(s:S2 e:E2 n:N2 d:D2 m:M2 o:O2 t:T2 y:Y2) = New in 10000*M *O *N1 + 10*E1 + Y1 <: 10000*M *O *N2 + 10*E2 + Y2 end {Explorer.object script(Money OptimizationConstraint)}

MSORM 2000 Constraint Programming Tutorial115 Experiments with MONEY MONEY Search MONEY Optimization

MSORM 2000 Constraint Programming Tutorial116 Scheduling  Scheduling Problems  Propagation Algorithms for Resource Constraints  Branching Algorithms  Other Constraints  Exploration Algorithms  Literature

MSORM 2000 Constraint Programming Tutorial117 Scheduling Problems Assign starting times (and sometimes durations) to tasks, subject to  resource constraints,  precedence constraints,  idiosyncratic and other constraints, and  an optimization function, often to minimize overall schedule duration.

MSORM 2000 Constraint Programming Tutorial118 Example: Building a Bridge Show Constraints  Resource constraints Example: a1 and a2 use excavator, cannot overlap in time  Precedence constraints Example: p1 requires a3, a3 must end before p1 starts Animate SolutionGantt Chart

MSORM 2000 Constraint Programming Tutorial119 Modeling  Indices: Tasks = [beg a1 a2 a3 a4 a5 a6 p1]  Constants: duration of tasks Duration = d(beg:0 a1:4 a2:2...)  Variables: represent each task with a finite domain variable representing its starting time. Start = {FD.record start Tasks 0#MaxTime} Start.t is finite domain variable representing the starting time of task t

MSORM 2000 Constraint Programming Tutorial120 Precedence Constraints For each two tasks t1, t2, where t1 must precede t2, introduce a constraint Start.t1 + Duration.t1  Start.t2

MSORM 2000 Constraint Programming Tutorial121 Resource Constraints For each two tasks t1, t2 that require a unary resource r, we have the constraint Start.t1 + Duration.t1  Start.t2  Start.t2 + Duration.t2  Start.t1 But: many constraints and weak propagation. Thus, introduce global resource constraints.

MSORM 2000 Constraint Programming Tutorial122 Global Resource Constraints in Oz {Schedule.serializedDisj [ [a1 a2 a3 a4 a5 a6] % excavator [p1 p2] % pile driver... ] Start Duration} User chooses among Schedule.serialized Schedule.serializedDisj Schedule.taskIntervals

MSORM 2000 Constraint Programming Tutorial123 Propagation: Using Disjunction For all tasks t1, t2 using the same resource: Start.t1 + Duration.t1  Start.t2  Start.t2 + Duration.t2  Start.t1 Weakest but fastest form of propagation for unary resources.

MSORM 2000 Constraint Programming Tutorial124 Propagation: Edge finding For a given task t and set of tasks S, all sharing the same unary resource, find out whether t can occur  before all tasks in S,  after all tasks in S,  between two tasks in S, and infer corresponding basic constraints.

MSORM 2000 Constraint Programming Tutorial125 Propagation: Task Intervals Notation: est(t): earliest starting time lct(t): latest completion time For two tasks t1 and t2 where est(t1)  est(t2) and lct(t1)  lct(t2), the task interval I(t1,t2) is defined: I(t1,t2) = { t | est(t1)  est(t) and lct(t)  lct(t2) } Perform edge finding on all task intervals.

MSORM 2000 Constraint Programming Tutorial126 Task Intervals: Example I(A,B) = {A,B,C} I(C,D) = {B,C,D} A B C D

MSORM 2000 Constraint Programming Tutorial127 Which Edge Finder? As usual, trade-off between run-time of propagation algorithm and strength of propagation. Edge finding can be expensive depending on which tasks are considered. Recent edge finders have complexity n 2 for one propagation step, where n is the number of tasks using the resource. Edge finding based on task intervals can be stronger than these, but typically have complexity n 3.

MSORM 2000 Constraint Programming Tutorial128 Demo: Propagation for Unary Resources Bridge

MSORM 2000 Constraint Programming Tutorial129 Branching Algorithms: Serializers  Simple enumeration techniques such as first-fail are hopeless for scheduling.  Use unary resources to guide branching.  For two tasks t1, t2 sharing the same resource, use the constraints Start.t1 + Duration.t1  Start.t2 and Start.t2 + Duration.t2  Start.t1 for branching.

MSORM 2000 Constraint Programming Tutorial130 Which Tasks To Serialize?  Resource-oriented serialization: Serialize all tasks of one resource completely, before tasks of another resource are serialized.  Most-used-resource serialization  Global slack  Local slack  Task-oriented serialization: Choose two suitable tasks at a time, regardless of resources.  Slack-based task serialization (see later)

MSORM 2000 Constraint Programming Tutorial131 Most-used-resource Serialization “Most-used-resource” serialization: serialize first the resource that is used the most. Let T be a set of tasks running on resource r. demand(T) =  t  T Duration.t Let S be the set of all tasks using resource r. demand(r) := demand(S) Serialize the resource r with maximal demand(r) first.

MSORM 2000 Constraint Programming Tutorial132 Slack-based Resource Serialization Let T be a set of tasks running on resource r. supply(T) = lct(T) - est(T) demand(T) =  t  T Duration.t slack(T) = supply(T) - demand(T) Let S be the set of all tasks using resource r. slack(r) := slack(S), S is set of all task running on r. Global slack serialization: Serialize resource with smallest slack first

MSORM 2000 Constraint Programming Tutorial133 Local-Slack Resource Serialization Let I r be all task intervals on r. The local slack is defined as min {slack(I) | I  I r } Local slack serialization: Serialize resource with smallest local slack first. Use global slack for tie-breaking.

MSORM 2000 Constraint Programming Tutorial134 Which Tasks Are Serialized? Ideas:  Use edge finding to look for possible first tasks (and last tasks)  Choose tasks according to their est, lst ( lct, ect ).

MSORM 2000 Constraint Programming Tutorial135 Task-oriented Serialization Among all tasks using all resources, select a pair of tasks according to local/global slack criteria and other considerations, regardless what resources are serialized already. Provides more fine-grained control at the expense of runtime for finding candidate task pairs among all tasks.

MSORM 2000 Constraint Programming Tutorial136 Demo: Serialization Algorithms Bridge

MSORM 2000 Constraint Programming Tutorial137 Exploration Algorithms  Usually: branch-and-bound using minimization of the starting time of a “ last ” task minimize a[end].start subject to {...}  Lower-bounding: Prove the non-existence of a solution with Start.last  n, add constraint Start.last > n ; increase n by .  Upper-bounding: Find solution with Start.last = n, add constraint Start.last < n ; decrease n by .

MSORM 2000 Constraint Programming Tutorial138 Literature on CP & Scheduling  Van Hentenryck: The OPL optimization programming language, 1999  Constraint Programming Tutorial of Mozart, 2000 ( )  Applegate, Cook: A computational study of the job-shop scheduling problem, 1991  Carlier, Pinson: An algorithm for solving the job- shop scheduling problem, 1989  Various papers by Laburthe, Caseau, Baptiste, Le Pape, Nuijten, see “Overview” paper

MSORM 2000 Constraint Programming Tutorial139 Overview  Constraint Programming in a Nutshell  Elements of Constraint Programming  Case study: ACC 97/98 Basketball  Constraint Programming Techniques  Excursions: Constraint Programming in Oz  Constraint-based Scheduling  Some Final Thoughts

MSORM 2000 Constraint Programming Tutorial140 Assessment: Don’t Use It! Don’t use constraint programming for:  Problems for which there are known efficient algorithms or heuristics. Example: Traveling salesman.  Problems for which integer programming works well, such as many discrete assignment problems.  Problems with weak constraints and a complex optimization function, such as many timetabling problems.

MSORM 2000 Constraint Programming Tutorial141 Assessment: Do Use It! Use constraint programming for:  Problems for which integer programming does not work (linear models too large) and for which there are no efficient solutions available.  Problems with tight constraints, where propagation can be employed. Example: ACC 97/98.  Problems for which strong branching algorithms exist. Example: Scheduling with unary resources.  Problems that need customized branching or search algorithms, extensive performance debugging or tight software integration

MSORM 2000 Constraint Programming Tutorial142 Myths Debunked  A fad! Constraint programming has been used successfully in a number of application areas, most spectacularly in scheduling  Universal! More failure stories than success stories.  All new! Many ideas come from AI search and Operations Research. In particular, the important ideas for scheduling come from OR.  Artificial Intelligence! Mostly quite earthly algorithms. Constraint programming systems provide software framework for these algorithms to interact with each other.

MSORM 2000 Constraint Programming Tutorial143 The Future  Constraint programming will become a standard technique in OR for solving combinatorial problems, along with local search and integer programming.  Constraint programming techniques will be tightly integrated with integer programming and local search.  Specific propagation algorithms will be developed to cater for important application domains.