Case study 5: balanced academic curriculum problem Joint work with Brahim Hnich and Zeynep Kiziltan (CPAIOR 2002)

Outline Introduction Introduction Balanced Academic Curriculum Problem (BACP) Balanced Academic Curriculum Problem (BACP) Alternate Models Alternate Models Combined Models Combined Models Experimental Results Experimental Results Conclusions Conclusions

Introduction Modelling optimisation problems as CSPs requires many skills Modelling optimisation problems as CSPs requires many skills Quite often alternate models are tried Quite often alternate models are tried models that add implied constraints models that add implied constraints models that add symmetry-breaking constraints models that add symmetry-breaking constraints models that use different variable representation models that use different variable representation models that use different constraint formulation models that use different constraint formulation combined models combined models solved with the same solution method solved with the same solution method solved with different solution methods solved with different solution methods

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering

BACP: description Periods Courses Each course has associated a number of credits Each course has associated a number of credits some courses can have other courses as prerequisite some courses can have other courses as prerequisite a minimum amount of credits per period is required a minimum amount of credits per period is required a maximum amount of credits per period is allowed a maximum amount of credits per period is allowed a minimum number of courses per period is required a minimum number of courses per period is required a maximum number of courses per period is allowed a maximum number of courses per period is allowed

BACP: objective Assign a period to every course such that the following constraints are satisfied: Assign a period to every course such that the following constraints are satisfied: the minimum and maximum academic load for each period the minimum and maximum academic load for each period the minimum and maximum number of courses for each period the minimum and maximum number of courses for each period the prerequisite relationships the prerequisite relationships An optimal balanced curriculum minimises the maximum academic load for all periods An optimal balanced curriculum minimises the maximum academic load for all periods

Model 1: variables 0..1 Courses Periods Cur1

Model 1: constraints Each course is taught Each course is taught  j  periods Cur1[i,j] = 1 Min and max courses per period Min and max courses per period c   i  courses Cur1[i,j]  d

Model 1: constraints Each course is taught Each course is taught  j  periods Cur1[i,j] = 1 Min and max courses per period Min and max courses per period c   i  courses Cur1[i,j]  d Academic load in each period more messy Academic load in each period more messy Each course comes with a weight Each course comes with a weight Is there a way of dealing with this constraint efficiently? Is there a way of dealing with this constraint efficiently?

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering

Refine Model 1: auxiliary vars Add auxiliary variables to compute academic load in each period Add auxiliary variables to compute academic load in each period Load(j) =  i  courses credit(i)*Cur1[i,j]

Model 1: variables 0..1 Courses Periods  = Cur1 0..maxint Load Periods

Load(j) =  i  courses credit(i)*Cur1[i,j] c   i  courses Cur1[i,j]  d a  Load(j)  b Computing load of period j: Restricting the load of period j: Restricting the number of courses of period j: Objective function: minimize max(Load) Prerequisite constraints btw i and j:  k  periods: k> 1 Cur1[j,k]   r=1 Cur1[i,r] k-1 Model 1: constraints Good News: all constraints are linear: we can also employ ILP methods Bad News: prerequisite constraints: too many constraints high arity-constraints We refer to model 1 as CP1 when CP solver is used and ILP when ILP solver is used

Breaking symmetry Shouldn’t we be worrying about breaking symmetry? Shouldn’t we be worrying about breaking symmetry?

Breaking symmetry Shouldn’t we be worrying about breaking symmetry? Shouldn’t we be worrying about breaking symmetry? Not really Not really Time periods are not usually symmetrical due to pre-requisites Time periods are not usually symmetrical due to pre-requisites Similarly courses are only symmetrical if they carry the same load and pre-requisites Similarly courses are only symmetrical if they carry the same load and pre-requisites

Alternative models? Model 1 has lots of 0/1 variables Model 1 has lots of 0/1 variables Can’t we exploit larger domains that CSPs give us? Can’t we exploit larger domains that CSPs give us?

Model 2: variables in periods courses 0..maxint Load Periods Cur2

Model 2: constraints Load(j) =  i  courses credit(i)*B i,j atleast(j, Cur2,c) atmost(j,Cur2,d) a  Load(j)  b Computing load of period j: Restricting the load of period j: Restricting the number of courses of period j: Objective function: minimize max(Load) Prerequisite constraint btw i and j: Cur2[i] < Cur2[j] Good News: use of global constraints efficient statement of prerequisite constraints Bad News: load computation: more Boolean variables more reification constraints We refer to model 2 as CP2 B i,j  Cur2[i]=j, for all i

Model CP1 vs. Model CP2 CP2 Good News: use of global constraints to restriction number of courses statement of prerequisite constraints Bad News: statement of academic load constraint CP1 Good News: academic load constraint easily stated by a weighted column sum Bad News: statement of prerequisite constraints

Combined model CP1+CP2 CP2 Good News: use of global constraints to restriction number of courses statement of prerequisite constraints Bad News: statement of academic load constraint CP1 Good News: academic load constraint easily stated by a weighted column sum Bad News: statement of prerequisite constraints Cur1[i,j]=1  Cur2[i]=j - Channelling constraints Bad News - Increased number of variables

Model ILP vs Model CP2 CP2 Good News: use of global constraints to restriction number of courses statement of prerequisite constraints Bad News: statement of academic load constraint ILP Good News: academic load constraint easily stated by a weighted column sum all constraints are linear Bad News: statement of prerequisite constraints

Hybrid model ILP+CP2 CP2 Good News: statement of prerequisite constraints ILP Good News: academic load constraint easily stated by a weighted column sum all constraints are linear Bad News: statement of prerequisite constraints Cur1[i,j]=1  Cur2[i]=j - Channelling constraints Bad News - Increased number of variables

Finding optimal solution We used three real-life instances We used three real-life instances CP1+CP2 is the quickest CP1+CP2 is the quickest due to the increase in the amount of pruning that compensated the increase in variables and constraints due to the increase in the amount of pruning that compensated the increase in variables and constraints ILP+CP2 is second best model ILP+CP2 is second best model better than CP2 in all instance better than CP2 in all instance better than ILP in two instances better than ILP in two instances CP model is essential in reducing the search space while the ILP model with its relaxation is essential for bounding and guiding the search CP model is essential in reducing the search space while the ILP model with its relaxation is essential for bounding and guiding the search

Proving optimality Proving Optimality Proving Optimality ILP+CP2 is quicker than ILP on two instances, and is the quickest on these instances ILP+CP2 is quicker than ILP on two instances, and is the quickest on these instances ILP+CP2 benefits from CP model in reducing the search space, and from the relaxation of the ILP model ILP+CP2 benefits from CP model in reducing the search space, and from the relaxation of the ILP model It is tough for all CP models (CP1, CP2, CP1+CP2) to prove optimality It is tough for all CP models (CP1, CP2, CP1+CP2) to prove optimality

Yet other alternate models Viewing BACP as a set partitioning problem the set of courses ought to be partitioned into n subsets, one for each periods the set of courses ought to be partitioned into n subsets, one for each periods n set variables that are subsets of the set of courses n set variables that are subsets of the set of courses the union of all set variables is the set of courses the union of all set variables is the set of courses all set variables must be pair-wise disjoint all set variables must be pair-wise disjoint all constraints of BACP can easily be stated using weighted cardinality, and cardinality constraints all constraints of BACP can easily be stated using weighted cardinality, and cardinality constraints except, the prerequisite constraints! except, the prerequisite constraints! To enhance the model based on set variables, we can also channel into the model CP2 To enhance the model based on set variables, we can also channel into the model CP2

Conclusions Lesson 1: When constraints are difficult to specify in a particular model, we should consider channelling into a second model in which these constraints are easier to specify and reason about Lesson 1: When constraints are difficult to specify in a particular model, we should consider channelling into a second model in which these constraints are easier to specify and reason about Lesson 2: Lesson 2: CP models may excel in finding optimal or near-optimal solutions CP models may excel in finding optimal or near-optimal solutions ILP models may be better for proving optimality ILP models may be better for proving optimality

Conclusions Lesson 3: CP and ILP tools should provide efficient primitives for channelling between models Lesson 3: CP and ILP tools should provide efficient primitives for channelling between models Lesson 4: careful integration of different models can result in better models despite the increase in the number of variables and constraints Lesson 4: careful integration of different models can result in better models despite the increase in the number of variables and constraints

Case study 6: Steel Mill Slab Design Problem Joint work with Alan Frisch and Ian Miguel

Overview The slab design problem The slab design problem An example An example Model A Model A Model B Model B A dual model A/B A dual model A/B Results Results Conclusion/Future Work Conclusion/Future Work

Background This work is based on the problem as stated in: “Variable Sized Bin Packing with Color Constraints”, Dawande, Kalagnanam and Sethuraman Approximation algorithms guaranteed to be within some bound of an optimal solution.

Motivation Many problems exhibit flexibility in portions of their structure. Many problems exhibit flexibility in portions of their structure. Example: the number required of a certain type of variable. Example: the number required of a certain type of variable. Flexibility must be resolved during the solution process. Flexibility must be resolved during the solution process. Slab design is a representative example of this type of problem. Slab design is a representative example of this type of problem.

The Slab Design Problem The mill can make  different slab sizes. The mill can make  different slab sizes. Given j input orders with: Given j input orders with: A colour (route through the mill). A colour (route through the mill). A weight. A weight. Pack orders onto slabs such that the total slab capacity is minimised, subject to: Pack orders onto slabs such that the total slab capacity is minimised, subject to: Capacity constraints. Capacity constraints. Colour constraints. Colour constraints.

Slab Design Constraints Capacity: Capacity: Total weight of orders assigned to a slab cannot exceed slab capacity. Total weight of orders assigned to a slab cannot exceed slab capacity. Colour: Colour: Each slab can contain at most p of k total colours. Each slab can contain at most p of k total colours. Reason: expensive to cut slabs up to send them to different parts of the mill. Reason: expensive to cut slabs up to send them to different parts of the mill.

An Example Slab Sizes: {1, 3, 4} (  = 3) Orders: {o a, …, o i } (j = 9) Colours: {red, green, blue, orange, brown} (k = 5) p = 2 abcdefghi

An Example Solution f gi e cd b h a 6 Slabs: (size 4)(size 3)(size 1) (size 3)(size 1) 2

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering

Basic model What are the variables? What are the variables? Need to assign sizes to slabs Need to assign sizes to slabs Complicated because we don’t even know how many slabs will be used Complicated because we don’t even know how many slabs will be used

Model A – Redundant Variables Slabs cannot be more than orders Slabs cannot be more than orders Assuming greatest order weight does not exceed maximum slab size Assuming greatest order weight does not exceed maximum slab size A list, S, of slab variables: {s 1, …, s j }. A list, S, of slab variables: {s 1, …, s j }. Domains size . Domains size . Solution quality: Solution quality:

Slab Variable Redundancy Some slab variables may be redundant: Some slab variables may be redundant: 0 is added to the domain of each s i. 0 is added to the domain of each s i. If s i is not necessary to solve the problem, s i = 0. If s i is not necessary to solve the problem, s i = 0.

Slab Variable Symmetry Slab variables are indistinguishable. Slab variables are indistinguishable. So model A suffers from symmetry: So model A suffers from symmetry: Counteract with binary symmetry-breaking constraints: s 1  s 2, s 2  s 3, etc. Counteract with binary symmetry-breaking constraints: s 1  s 2, s 2  s 3, etc.

Basic model What are the variables? What are the variables? Need to assign orders to slabs Need to assign orders to slabs Want to compute weight on each slab efficiently Want to compute weight on each slab efficiently

Model A Order Matrix (Oa) oaoa obob ococ odod s1s s2s s3s s4s4 0000

More Slab Symmetry Slab variables assigned the same size are indistinguishable Slab variables assigned the same size are indistinguishable If s i and s i+1 have the same size, the corresponding rows of the order matrix are lexicographically ordered. If s i and s i+1 have the same size, the corresponding rows of the order matrix are lexicographically ordered. E.g  E.g  0110.

Colour constraints Need to count how many colours used on a slab Need to count how many colours used on a slab Counting hard Counting hard Channel into a matrix of 0/1 variables and sum rows Channel into a matrix of 0/1 variables and sum rows

Colour Matrix: colourMa RedGreenBlueOrange s1s s2s s3s s4s Channelling:

Model A of the Example Problem oaoa obob ococ odod oeoe ofof ogog ohoh oioi oaoa obob ococ odod oeoe ofof ogog ohoh oioi s1s1 1 s2s2 1 … RedGreenBlueOrangeBrown s1s1 1 s2s2 1 …

A Solution: Model A oaoa obob ococ odod oeoe ofof ogog ohoh oioi oaoa obob ococ odod oeoe ofof ogog ohoh oioi s1s s2s s3s s4s … RedGreenBlueOrangeBrown s1s s2s s3s s4s …00000 s 1 = 4, s 2 = 3, s 3 = 3, s 4 = 3, s i = 0 (5  i  9)

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering

Model A Implied Constraints Combined weight of input orders is a lower bound on optimisation variable: Combined weight of input orders is a lower bound on optimisation variable: Lower bound on number of slabs required: Lower bound on number of slabs required: With symmetry-breaking constraints, decomposes into unary constraints on slab variables.

Model A Implied Constraints (2) AssWt i is the weight of orders assigned to s i. AssWt i is the weight of orders assigned to s i. Prune domains by reasoning about reachable values via dynamic programming [Trick, 2001]. Prune domains by reasoning about reachable values via dynamic programming [Trick, 2001]. Incorporate both size and colour information. Incorporate both size and colour information. More powerful if done during search (future work). More powerful if done during search (future work). Minimum number of slabs required: Minimum number of slabs required:

Model A Implied Constraints (3) Waste i = s i – AssWt i Waste i = s i – AssWt i the unused portion of a slab. the unused portion of a slab. Upper bound on total waste: Upper bound on total waste: Assume each order is assigned to an individual slab, with smallest size able to hold it. Assume each order is assigned to an individual slab, with smallest size able to hold it. Sum waste in each case: leads to upper bound for optimisation variable. Sum waste in each case: leads to upper bound for optimisation variable. Upper bound on Waste i is the worst of these cases. Upper bound on Waste i is the worst of these cases.

Model B – Abstraction 2-phase approach: 2-phase approach: 1. Construct/solve an abstraction of the problem. 2. Solve independent sub-problems, assigning a subset of the orders to slabs of a common size. Solving phase 2 sub-problems either: Solving phase 2 sub-problems either: Provides a solution to the original problem, or: Provides a solution to the original problem, or: Identifies new constraints which restrict set of solutions at phase 1. Identifies new constraints which restrict set of solutions at phase 1.

Model B, Phase 1 Number of slab sizes is fixed. Number of slab sizes is fixed. A list, Z, of slab size variables, {z 1, z 2, …}. A list, Z, of slab size variables, {z 1, z 2, …}. Domains: {0, …, j} number of slabs of corresponding sized used. Domains: {0, …, j} number of slabs of corresponding sized used. Solution quality: Solution quality:

Model B, Phase 1 Order Matrix (Ob) oaoa obob ococ odod z1z z3z z4z4 1000

Model B, Phase 1 Colour Matrix (ColourMb) RedGreenBlueOrange z1z z3z z4z Channelling:

A Solution: Model B, Phase oaoa obob ococ odod oeoe ofof ogog ohoh oioi oaoa obob ococ odod oeoe ofof ogog ohoh oioi z1z z3z z4z RedGreenBlueOrangeBrown z1z z3z z4z z 1 = 0, z 3 = 3, z 4 = 1

Model B Implied Constraints Unary constraints on order matrix:

Model B, Phase 2 Model B, Phase 1 is ambiguous. Model B, Phase 1 is ambiguous. A Phase 1 solution does provide: A Phase 1 solution does provide: Number and sizes of slabs required. Number and sizes of slabs required. Size of slab each order is assigned to. Size of slab each order is assigned to. Quality of final solution. Quality of final solution. Phase 1 solution used to construct much simpler, independent, phase 2 sub- problems. Phase 1 solution used to construct much simpler, independent, phase 2 sub- problems.

Model B, Phase 2 Sub-problems oaoa obob ococ odod oeoe ofof ogog ohoh oioi oaoa obob ococ odod oeoe ofof s1s s2s s3s Slabs of size 3 1 Slab of size 4 ogog ohoh oioi s1s1 111

The Price of Ambiguity Model B, Phase 1 is ambiguous. Model B, Phase 1 is ambiguous. Phase 2 sub-problems may be inconsistent. Phase 2 sub-problems may be inconsistent. Due to interaction between weight/colour constraints. Due to interaction between weight/colour constraints oaoa obob ococ odod p = 1 oaoa obob ococ odod s1s1 ???? s2s2 ???? 2 Slabs of size 4

Conflict Recording Not simply underestimation of optimisation variable: Not simply underestimation of optimisation variable: May be incorrect combination of slab sizes. May be incorrect combination of slab sizes. Or wrong assignment of orders to sizes. Or wrong assignment of orders to sizes. Solution: Solution: Isolate reasons for failure. Isolate reasons for failure. Post constraints at phase 1. Post constraints at phase 1. Solve phase 1 again. Solve phase 1 again. Example: Example: o a = 4  o b = 4  o c = 4  o d = 4  z 4 > 2 o a = 4  o b = 4  o c = 4  o d = 4  z 4 > 2

A Model B Solution Cycle Phase 1 Phase 2 Solution Constraints

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering

Combined Model A/B Model A and model B, phase 1. Model A and model B, phase 1. Variables: Variables: Explicit slab variables (s i ) and slab-size variables (z i ). Explicit slab variables (s i ) and slab-size variables (z i ). Order matrices referring to explicit slabs (Oa) and to slab-sizes (Ob). Order matrices referring to explicit slabs (Oa) and to slab-sizes (Ob). Both types of colour matrix. Both types of colour matrix.

Channelling Constraints Constraints for individual models as previously described. Constraints for individual models as previously described. Channelling constraints between the models maintain consistency, aid pruning. Channelling constraints between the models maintain consistency, aid pruning. Between S and Z: Between S and Z: (Number of occurrences of i in S) = z i. (Number of occurrences of i in S) = z i. Between order matrices and S: Between order matrices and S: Oa[i, j] = 1  Ob[i, s j ] = 1. Oa[i, j] = 1  Ob[i, s j ] = 1.

A/B Search Strategy Instantiate model A variables first: Instantiate model A variables first: Channelling constraints ensure model B variables instantiated. Channelling constraints ensure model B variables instantiated. Analogous to pure model A approach. Analogous to pure model A approach. Instantiate model B variables first: Instantiate model B variables first: Channelling constraints do not force instantiation of model A variables. Channelling constraints do not force instantiation of model A variables. Model A variables are constrained though. Model A variables are constrained though. Analogous to pure model B approach. Analogous to pure model B approach.

A/B Search Strategies 2 Other search strategies exploit more complete view offered by model A/B. Other search strategies exploit more complete view offered by model A/B. Interleave instantiation of variables from 2 basic models: Interleave instantiation of variables from 2 basic models: Obtain most efficient pruning of the search space. Obtain most efficient pruning of the search space.

Results OrdersOptimalModel AModel AB : 14, 0.1s 78: 486, 0.2s 77: 1841, 0.6s 83: 14, 0.1s 78: 452, 0.2s 77: 1714, 0.6s : 14, 0.1s 80: 451, 0.2s 79: 1536, 0.5s 83: 14, 0.1s 80: 407, 0.2s 79: 1447, 0.4s : 16, 0.1s 89: 819, 0.2s 88: 934, 0.3s 87: 8797, 1.2s 95: 16, 0.1s 89: 726, 0.3s 88: 841, 0.4s 87: 8612, 2.2s : 21, 0.1s 94: 5108, 1.1s 93: 5619, 1.2s 92: 17734, 3.6s 95: 21, 0.2s 94: 4950, 1.3s 93: 5456, 1.5s 92: 17190, 4.7s

Results OrdersOptimalModel AModel AB : 17, 0.1s 101: 5112, 0.9s 100: 5305, 0.9s 99: 92441, 17.8s 107: 17, 0.1s 101: 3673, 0.9s 100: 3866, 0.9s 99: 89618, 23.5s : 23, 0.1s 105: 13074, 2.6s 104: 26757, 5.5s 103: , 50.2s 107: 23, 0.1s 105: 11556, 2.9s 104: 24792, 6.6s 103: , 67.1s : 19, 0.2s 111: 1012, 0.4s 110: , 253.4s 119: s 111: 977, 0.4s 110: , 350.3s

Model B Results? On these problems, many solutions at phase 1. On these problems, many solutions at phase 1. Cycle is therefore lengthy. Cycle is therefore lengthy. Improve efficiency: Improve efficiency: Model phase 1 as a dynamic CSP. Model phase 1 as a dynamic CSP. Reduce arity of recorded constraints. Reduce arity of recorded constraints. Phase 1 heuristics Phase 1 heuristics

Other Models Set variables: Set variables: Each represents a slab Each represents a slab Domain is set of orders assigned. Domain is set of orders assigned. Activity-based dynamic CSP: Activity-based dynamic CSP: Model A slab variables used. Model A slab variables used. Only `activated’ according to remaining capacity of activated slabs. Only `activated’ according to remaining capacity of activated slabs.

Conclusions Results only on small instances. Results only on small instances. All models need further development: All models need further development: More implied constraints. More implied constraints. Better heuristics Better heuristics Explore new models: Explore new models: Set variables Set variables

Recipe Create a basic model Decide on the variables Introduce auxiliary variables For messy/loose constraints Consider dual, combined or 0/1 models Break symmetry Add implied constraints Customize solver Variable, value ordering