1. Constraint Programming An Appetizer Christian Schulte schulte@imit.kth.se Laboratory of Electronics and Computer Systems Institute of Microelectronics and Information Technology KTH – Royal Institute of Technology Stockholm, Sweden

2. Christian Schulte, IMIT, KTH2 2003-10-02 Constraint Programming Solving combinatorial problems start with a first toy problem

3. Christian Schulte, IMIT, KTH3 2003-10-02 Send More Money (SMM) Find distinct digits for letters, such that SEND + MORE =MONEY

4. Christian Schulte, IMIT, KTH4 2003-10-02 Constraint Model for SMM Variables: S,E,N,D,M,O,R,Y  {0,…,9} Constraints: distinct(S,E,N,D,M,O,R,Y) 1000×S+100×E+10×N+D + 1000×M+100×O+10×R+E = 10000×M+1000×O+100×N+10×E+Y S0M0S0M0

5. Christian Schulte, IMIT, KTH5 2003-10-02 Solving SMM Find values for variables such that all constraints satisfied

6. Christian Schulte, IMIT, KTH6 2003-10-02 Finding a Solution Enumerate assignments: poor! Constraint programming  compute with possible values  prune inconsistent values constraint propagation  search branch:define search tree explore:explore search tree for solution

7. Overview

8. Christian Schulte, IMIT, KTH8 2003-10-02 Overview Constraint propagation Search Summary: Applications & Principles Application examples resource scheduling instruction scheduling Constraint Programming 2G1515

9. Constraint Propagation

10. Christian Schulte, IMIT, KTH10 2003-10-02 Important Concepts Constraint store Basic constraint Propagator Non-basic constraint Constraint propagation

11. Christian Schulte, IMIT, KTH11 2003-10-02 Constraint Store Stores basic constraints map variables to possible values x  {3,4,5} y  {3,4,5}

12. Christian Schulte, IMIT, KTH12 2003-10-02 Constraint Store Stores basic constraints map variables to possible values x  {3,4,5} y  {3,4,5} finite domain constraints

13. Christian Schulte, IMIT, KTH13 2003-10-02 Constraint Store Stores basic constraints map variables to possible values Domains: finite sets, real intervals, trees, … x  {3,4,5} y  {3,4,5}

14. Christian Schulte, IMIT, KTH14 2003-10-02 Propagators Implement non-basic constraints distinct(x 1,…,x n ) x + 2×y = z

15. Christian Schulte, IMIT, KTH15 2003-10-02 Propagators Amplify store by constraint propagation x  {3,4,5} y  {3,4,5} xyxy y>3

16. Christian Schulte, IMIT, KTH16 2003-10-02 Propagators Amplify store by constraint propagation x  {3,4,5} y  {3,4,5} xyxy y>3

17. Christian Schulte, IMIT, KTH17 2003-10-02 Propagators Amplify store by constraint propagation x  {3,4,5} y  {4,5} xyxy y>3

18. Christian Schulte, IMIT, KTH18 2003-10-02 Propagators Amplify store by constraint propagation x  {3,4,5} y  {4,5} xyxy y>3

19. Christian Schulte, IMIT, KTH19 2003-10-02 Propagators Amplify store by constraint propagation x  {4,5} y  {4,5} xyxy y>3

20. Christian Schulte, IMIT, KTH20 2003-10-02 Propagators Amplify store by constraint propagation Disappear when done (entailed) no more propagation possible x  {4,5} y  {4,5} xyxy y>3

21. Christian Schulte, IMIT, KTH21 2003-10-02 Propagators Amplify store by constraint propagation Disappear when done (entailed) no more propagation possible x  {4,5} y  {4,5} xyxy

22. Christian Schulte, IMIT, KTH22 2003-10-02 Computation Space Store with connected propagators x  {4,5} y  {4,5} xyxy y>3

23. Christian Schulte, IMIT, KTH23 2003-10-02 Propagation for SMM Results in store S=9 E  {4,…,7}N  {5,…,8} D  {2,…,8} M=1 O=0 R  {2,…,8} Y  {2,…,8} Propagation alone not sufficient! create simpler sub-problems branching

24. Search

25. Christian Schulte, IMIT, KTH25 2003-10-02 Important Concepts Branching Exploration Branching heuristics Best solution search

26. Christian Schulte, IMIT, KTH26 2003-10-02 Branching Yields spaces with additional constraints Enables further constraint propagation x  {4,5} y  {4,5} xyxy y>3 x  {4} y  {4} xyxy y>3 x  {5} y  {4,5} xyxy y>3 x=4 x4x4

27. Christian Schulte, IMIT, KTH27 2003-10-02 Branching Strategy Pick variable x with at least two values Pick value n from domain of x Branch with x=n and x  n Part of model

28. Christian Schulte, IMIT, KTH28 2003-10-02 Search Iterate propagation and branching Orthogonal: branching  exploration Nodes:  Unsolved  Failed  Succeeded

29. Christian Schulte, IMIT, KTH29 2003-10-02 SMM: Solution SEND + MORE =MONEY 9567 + 1085 =10652

30. Christian Schulte, IMIT, KTH30 2003-10-02 Heuristics for Branching Which variable least possible values (first-fail) application dependent heuristic Which value minimum, median, maximum x=m or x  m split with median m x<m or x  m Problem specific

31. Christian Schulte, IMIT, KTH31 2003-10-02 SMM: Solution With First-fail SEND + MORE =MONEY 9567 + 1085 =10652

32. Christian Schulte, IMIT, KTH32 2003-10-02 Send Most Money (SMM++) Find distinct digits for letters, such that and MONEY maximal SEND + MOST =MONEY

33. Christian Schulte, IMIT, KTH33 2003-10-02 Best Solution Search Naïve approach: compute all solutions choose best Branch-and-bound approach: compute first solution add “betterness” constraint to open nodes next solution will be “better” prunes search space

34. Christian Schulte, IMIT, KTH34 2003-10-02 Branch-and-bound Search Find first solution

35. Christian Schulte, IMIT, KTH35 2003-10-02 Branch-and-bound Search Explore with additional constraint

36. Christian Schulte, IMIT, KTH36 2003-10-02 Branch-and-bound Search Explore with additional constraint

37. Christian Schulte, IMIT, KTH37 2003-10-02 Branch-and-bound Search Guarantees better solutions

38. Christian Schulte, IMIT, KTH38 2003-10-02 Branch-and-bound Search Guarantees better solutions

39. Christian Schulte, IMIT, KTH39 2003-10-02 Branch-and-bound Search Last solution best

40. Christian Schulte, IMIT, KTH40 2003-10-02 Branch-and-bound Search Proof of optimality

41. Christian Schulte, IMIT, KTH41 2003-10-02 Modelling SMM++ Constraints and branching as before Order among solutions with constraints so-far-best solution S,E,N,D,M,O,T,Y current node S,E,N,D,M,O,T,Y constraint added 10000×M+1000×O+100×N+10×E+Y <

42. Christian Schulte, IMIT, KTH42 2003-10-02 SMM++: Branch-and-bound SEND + MOST =MONEY 9782 + 1094 =10876

43. Christian Schulte, IMIT, KTH43 2003-10-02 SMM++: All Solution Search SEND + MOST =MONEY 9782 + 1094 =10876

44. Summary

45. Christian Schulte, IMIT, KTH45 2003-10-02 Summary Modeling variables with domain constraints to state relations branching strategy solution ordering Solving constraint propagation constraint branching search tree exploration applications principles

46. Christian Schulte, IMIT, KTH46 2003-10-02 Application Areas Timetabling Scheduling Crew rostering Resource allocation Workflow planning and optimization Gate allocation at airports Sports-event scheduling Railroad: track allocation, train allocation, schedules Automatic composition of music Genome sequencing Frequency allocation …

47. Christian Schulte, IMIT, KTH47 2003-10-02 Application Examples Scheduling resources machines, personal, … constraint programming showcase Instruction scheduling used in compilation for modern microprocessors with latencies

48. Christian Schulte, IMIT, KTH48 2003-10-02 Principles Models for constraint propagation properties and guarantees Strong constraint propagation global constraints with strong algorithmic methods mantra: search kills, search kills, search k… Branching strategies Exploration strategies

49. Christian Schulte, IMIT, KTH49 2003-10-02 SMM: Strong Propagation SEND + MORE =MONEY 9567 + 1085 =10652

50. Christian Schulte, IMIT, KTH50 2003-10-02 Used Techniques Artificial intelligence Operations research Algorithms Programming languages

51. Christian Schulte, IMIT, KTH51 2003-10-02 Why Does CP Matter? Middleware for combining smart algorithmic components scheduling graphs flows … plus essential extra constraints

52. Christian Schulte, IMIT, KTH52 2003-10-02 Significance Constraint programming identified as a strategic direction in computer science research [ACM Computing Surveys, December 1996] Applications are ubiquitous knowledgeable people are not!

53. Christian Schulte, IMIT, KTH53 2003-10-02 Relation to Logic Programming Logic programming horn clauses + trees with unification Constraint logic programming horn clauses + constraint domain Constraint programming X + constraint domain with focus on constraints refined search language

54. Scheduling Resources Modeling Propagation Strong propagation

55. Christian Schulte, IMIT, KTH55 2003-10-02 Scheduling Resources: Given Tasks duration resource Precedence constraints determine order among two tasks Resource constraints at most one task per resource [disjunctive, non-preemptive scheduling]

56. Christian Schulte, IMIT, KTH56 2003-10-02 Scheduling: Bridge Example

57. Christian Schulte, IMIT, KTH57 2003-10-02 Scheduling: Solution Start time for each task All constraints satisfied Earliest completion time minimal make-span

58. Christian Schulte, IMIT, KTH58 2003-10-02 Scheduling: Model Variable for start-time of task a start(a) Precedence constraint: a before b start(a) + dur(a)  start(b)

59. Christian Schulte, IMIT, KTH59 2003-10-02 Propagating Precedence start(a)  {0,…,7} start(b)  {0,…,5} a b a before b

60. Christian Schulte, IMIT, KTH60 2003-10-02 Propagating Precedence start(a)  {0,…,7} start(b)  {0,…,5} a b a b a before b start(a)  {0,…,2} start(b)  {3,…,5}

61. Christian Schulte, IMIT, KTH61 2003-10-02 Scheduling: Model Variable for start-time of task a start(a) Precedence constraint: a before b start(a) + dur(a)  start(b) Resource constraint: a before b or b before a

62. Christian Schulte, IMIT, KTH62 2003-10-02 Scheduling: Model Variable for start-time of task a start(a) Precedence constraint: a before b start(a) + dur(a)  start(b) Resource constraint: start(a) + dur(a)  start(b) or b before a

63. Christian Schulte, IMIT, KTH63 2003-10-02 Scheduling: Model Variable for start-time of task a start(a) Precedence constraint: a before b start(a) + dur(a)  start(b) Resource constraint: start(a) + dur(a)  start(b) or start(b) + dur(b)  start(a)

64. Christian Schulte, IMIT, KTH64 2003-10-02 Reified Constraints Use control variable b  {0,1} c  b=1 Propagate c holds   propagate b=1  c holds  propagate b=0 b=1 holds  propagate c b=0 holds  propagate  c

65. Christian Schulte, IMIT, KTH65 2003-10-02 Reified Constraints Use control variable b  {0,1} c  b=1 Propagate c holds   propagate b=1  c holds  propagate b=0 b=1 holds  propagate c b=0 holds  propagate  c not easy!

66. Christian Schulte, IMIT, KTH66 2003-10-02 Reification for Disjunction Reify each precedence [start(a) + dur(a)  start(b)]  b 0 =1 and [start(b) + dur(b)  start(a)]  b 1 =1 Model disjunction b 0 + b 1  1

67. Christian Schulte, IMIT, KTH67 2003-10-02 Model Is Too Naïve Local view individual task pairs O(n 2 ) propagators for n tasks Global view all tasks on resource single propagator smarter algorithms possible

68. Christian Schulte, IMIT, KTH68 2003-10-02 Edge Finding Find ordering among tasks (“edges”) For each subset of tasks {a}  B assume: a before B deduce information fora and B assume: B before a deduce information fora and B join computed information can be done in O(n 2 )

69. Christian Schulte, IMIT, KTH69 2003-10-02 Summary Modeling easy but not always efficient constraint combinators (reification) global constraints smart heuristics More on constraint-based scheduling Baptiste, Le Pape, Nuijten. Constraint-based Scheduling, Kluwer, 2001.

70. Instruction Scheduling

71. Christian Schulte, IMIT, KTH71 2003-10-02 Instruction Scheduling Optimized object code by compiler Minimum length instruction schedule precedence latency resources per basic block Best paper CP 2001, Peter van Beek and Kent Wilken, Fast Optimal Scheduling for Single-issue Processors with Arbitrary Latencies, 2001.

72. Christian Schulte, IMIT, KTH72 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC

73. Christian Schulte, IMIT, KTH73 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC instructions

74. Christian Schulte, IMIT, KTH74 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC dependency with latency

75. Christian Schulte, IMIT, KTH75 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC instructions i, j

76. Christian Schulte, IMIT, KTH76 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC latency l(i, j)

77. Christian Schulte, IMIT, KTH77 2003-10-02 Problem r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC Find issue time s(i) 1. i  j  s(i)  s(j) 2. s(i)+l(i,j)  s(j) Minimize max s(i)

78. Christian Schulte, IMIT, KTH78 2003-10-02 Problem Is also Model r1  ar1  ar2  br2  b r 1  r 1 + r 2 r3  cr3  c r 1  r 1 + r 3 E 13 3 AB DC Find issue time s(i) 1. i  j  s(i)  s(j) 2. s(i)+l(i,j)  s(j) Minimize max s(i)

79. Christian Schulte, IMIT, KTH79 2003-10-02 Model All issue times must be distinct use single distinct constraint (as in SMM) is resource constraint or unit duration Latency constraints precedence constraints (as before) difference: duration  latency

80. Christian Schulte, IMIT, KTH80 2003-10-02 Making It Work Only propagate bounds information relevant for distinct Add redundant constraints regions: additional structure in DAG successor and predecessor constraints [special case of edge-finding]

81. Christian Schulte, IMIT, KTH81 2003-10-02 Results Tested with gcc SPEC95 FP Large basic blocks up to 1000 instructions Optimally solved less than 0.6% compile time increase limited static improvement (< 0.1%) better dynamic impact (loops) Far better than ILP approach

82. 2G1515

83. Christian Schulte, IMIT, KTH83 2003-10-02 Course Overview 2G1515 As to be expected, no surprises: applications principles pragmatics limitations

84. Christian Schulte, IMIT, KTH84 2003-10-02 Organization 12 lectures Assignments (practical + principles) Exam Material to be developed as we go… with some textbook excerpts + research papers

85. Christian Schulte, IMIT, KTH85 2003-10-02 More Info Check my webpage frequently… www.imit.kth.se/~schulte/