Column Generation n ENGG 6070 n Instructor: Shawki M Areibi n Students: Limin Ma, Hao Qin

What is Column Generation ? Column generation is a technique that is widely used to solve large scale integer programs

Prototype of Linear Programs of Large-scale Systems z * = Max z = c 1 x 1 + c 2 x 2 + … +c n x n, Subject to: a i1 x 1 + a i2 x 2 + … + a in x n = b i ( i = 1, 2, …, m) (1) x n  0 ( j = 1, 2, …, n) where n is very large, may be more than millions.

Example : Cutting stock problem A paper company must produce various sizes of its paper products from its raw rolls to meet customer demand.

Example : Cutting stock problem If customers need b i rolls of sizes L i, how the company meet the demand using the fewest number of rolls ?

Example : Cutting stock problem A key obstacle : In general, there are exponentially many patterns x n to cut raw rolls: x1x1 x2x2 xnxn

Independent subsystemPrimal block angularDual block angular Bordered angular Block triangularStaircase Figure 1 Structure of Large-Scale Problems

The Decomposition Algorithm Decompose the large scale system in many subsystem for computation. For Independent subsystem, it can be written as: Minimize subject to: = b i (i = 1, 2, …,t) = b i (i = t+1, t+2, …,u) = b i (i = u+1, u+2, …,m) x j >=0 (j = 1, 2, …, n)

Why Decomposition ? Provide significant computational savings, since the computations for linear programs are quite sensitive to m, the number of constraints, in practice proportionally to m 3. So it can save 1/m 2 times. Each of the independent subproblem can be treated separately. Data can be gathered, analyzed and stored separately.

Because of the large number of variables and columns, directly solution by the simplex method may be inappropriate. Simply generating all the coefficients data a ij usually prohibits this approach. Column generation extends the technique introduced in the decomposition algorithm, of using the simplex method (SM), but generating the coefficient data only as needed. Consequently, new data could be generated by solving this linear program with an appropriate objective function. Why Column generation ?

The Approach of Column Generation As in decomposition, assume a priori that certain variables, say x J+1, x J+2, …, x n are nonbasic and restrict their values to zero. The problem becomes : z J = Max c 1 x 1 + c 2 x 2 + … + c J x J, (1) subject to: a i1 x 1 + a i2 x 2 + … + a iJ x J = bi ( i = 1, 2, …, m) (2) x j ≥ 0 ( j = 1, 2, …, J) The original problem (1) is called a master problem (MP), whereas problem (2) is called the restricted master problem (RMP).

Suppose that the RMP has been solved by SM and that  J 1,  J 2, …,  J m are the optimal shadow prices. It is optimal if the simplex optimality condition holds, that is, if the original problem has been solved without specifying all of the a ij data or solving the full MP. then the simplex method, when applied to MP, would introduce variable x s into the basis. The new RMP can be solved by the simplex method and the entire procedure can be repeated. If

This procedure avoids solving the full MP; instead it alternately solves a RMP. It is referred to as a subprobelm. The method is specified in flow-chart in Figure 2. Its efficiency is predicted upon: Obtaining an optimal solution before many columns have been added to RMP. Being able to solve the subproblem effectively. The Significance of Column Generation

The column generation method might generate a non-integer solution. For applications (with large demands), rounding to nearest integers produces satisfactory answers. If an exact optimal is needed, one can apply the B&B technique. Remark :

Type I : Airline crew scheduling (ACS) Type II : The cutting stock (CS) problem Two Major Types of Column Generation

Type I column generation Uses an auxiliary model (AM) to identify an “attractive” set of columns, defining a RMP that optimizes over these explicitly defined columns. The RMP accepts these columns and does not interact further with the AM. Two Types of Column Generation

Type I : Airline crew scheduling (ACS) The objective is to minimize total crew cost. Airline schedules give the type of aircraft and its departure and arrival times for each flight segment, a flight from one city to another. A crew works a “rotation,” which services a series of segments and may last several days. Each rotation must satisfy a number of constraints imposed by official agencies, union rules and airline restrictions. The cost of rotation includes lodging and other travel expenses, flight time and deadheading

Defining a ij =1 if rotation j ( j  J ) serves segment i (i  I ), 0 otherwise and employing decision variables x j =1 if rotation j is prescribed, 0 otherwise ( j  J ), Problem ACS may be formulated as: Min Z ACS = (4.1) s.t. i  I (4.2) x j  {0, 1} j  J (4.3) The algorithm of ACS

Challenge : there may be millions of feasible rotations for a commercial airline so that even generating them may require a prohibitive amount of run time. Solution : 1. Invokes an AM to generate a set of feasible rotations. Each generated column aj ( j  J ) consists of elements aij =1 (i  Ij ) for each segment i served by rotation j, and aij =0 (i  Ij ). 2. Solves the set partitioning problem with the columns defined explicitly by the AM.

Experience Result : It is typically not possible to assure that a restricted set of rotations includes a subset that comprises an optimal solution, so the overall approach is a heuristic. Nevertheless, experience has shown that it is possible to prescribe good solutions if the set partitioning problem can be solved effectively for a large number of generated rotations.

Type II column generation Type II column generation has two classic applications: i) The cutting stock (CS) problem; ii) The vehicle routing problem;

The Cutting Stock (CS) Problem. A company stocks rolls of sheet metal in standard lengths L m (m  M). Assume that the company has an unlimited number of each standard length and that will cut these standard lengths into shorter lengths to satisfy customers orders. Customers require b i pieces of length i (i  I ), where I is the set of cut lengths ordered. Note that order I can be satisfied as long as l i  L m for some m. Standard length L m costs c m (m  M) and the total cost of filling orders includes the costs of all standard lengths cut to fill the orders. The cutting stock problem is to fill customer orders at minimum total cost.

Let a ij denote the number of pieces of length i (i  I ) in cutting pattern j ; and c j, the cost of the standard roll used in pattern j. Thus, the index j implies m as well as I j, the subset of cut lengths included in the cutting pattern. To be feasible, cutting pattern j must observe the limitation imposed by the standard length L m (m  M): The Cutting Stock (CS) Problem.

The algorithm Decision variables x j =number of cutting patterns of type j prescribed ( j  J ) to present Problem CS as formulated: Min (5.1) s.t. i  I (5.2) x j  0, Integer j  J. (5.3)

The algorithm Linear relaxation for integer constraints: The requirement of the cutting stock problem is known to be tight. This heuristic is not guaranteed to prescribe the optimal integer solution. To guarantee such a solution, the linear relaxation may be used to obtain a lower bound at each node in a branch-and- bound (B&B) search tree.

The primary difficulty is that the number of cutting patterns, |J |, may be extremely large, so that solving the LP problem with all |J | columns could take a prohibitive amount of time. The algorithm

Instead of listing all |J | columns explicitly and pricing each out to identify the column that should enter solution at each simplex iteration, the RMP generates a limited set of columns and must be optimized over these columns to obtain dual variables that are used to define an dual objective function. Defining w i =dual variable associated with constraint i of type (5) (i. I ) and c m =cost of standard length L m (m. M) and employing decision variables y i =number of length i in a cutting pattern j. The algorithm

It can be formulated as: Min (6.1) s.t. (6.2) y i  0, Integer i  I (6.3) If Z * POP  0 (m  M), the current RMP solution is optimal. Otherwise, if Z * POP  0, the dual solution generates an improving column, which is assigned a new column number j ( j  j +1), an objective function coefficient c j = cm and coefficient in each row i (i  I )a ij = y* i (i  I ). The dual solution

The primary limitation of the Type I method is that the AM generates columns without interacting with the RMP. It normally has to generating a large set of columns with the hope of including the subset that comprises the optimal solution Type II methods, which do involve interaction between the RMP and SP(s), allow columns to be identified and introduced as needed Comparison between Type I and Type II

Other Problem Can be Solved by Column Generation : Generalized assignment problem The vehicle routing problem Single machine scheduling Assembly system design with tool changes. Problems that can be solved by the Dantzig- Wolfe decomposition.

Technical considerations

i) Formulation issues a) RMP formulation The goal is to formulate a decomposition that will allow the solution to the RMP to serve as a tight bound on the value of the optimal integer solution to facilitate the B&B search: (1) They may provide tighter bounds than more compact formulations can; (2) They may avoid symmetry that requires B&B to expend considerable run time to enumerate a large number of equivalent solutions; (3) They are amenable to decomposition, perhaps separating into a number of independent SPs that may be solved easily; and (4) They may provide the only formulation possible.

b). SP formulation SP(s) also play a crucial role through their structure, symmetry, complexity and whether of not they exhibit the Integrality Property. The integer variables in an IP typically become decision variables in one or more SPs. Formulations that have the block diagonal structure are attractive because they result in small, independent SPs that are typically more effectively solved.

ii) Algorithm design issues a) Branching strategies Branching must not destroy SP structure and should result in siblings that represent balanced sets of solutions. Balancing is important because it can be expected to result in a tighter bound at each sibling node, facilitating solution. b) Subproblem solution strategies SP solution strategies select the column that will enter the RMP and a method to solve each SP. One strategy would be to solve all SPs and select the best improving column to enter the RMP.

iii) Implementation issues a) Pool management 1. Determine an initial BFS for the RMP 2. Initialize the column pool to be empty 3. Optimize the RMP over columns that are known explicitly 4. Delete nonbasic generated columns in the RMP that have positive reduced costs 5. Select a subset of remaining pool columns with negative reduced costs, add them to RMP and go to step (3)

6. Eliminate any remaining columns from the pool 7. Use a heuristic to solve the SP(s), If improving columns are generated, add them to the pool and go to step (5) 8. If no columns are generated in step (7), use an optimizing algorithm to solve the SP(s) If improving columns are generated, add them to the pool and go to step (5) 9. Stop

b) Degeneracy Degeneracy in the RMP may be problematic for several reasons. First, a number of pivots ( and corresponding SP solutions to generate entering columns) may be required before the search can progress to a better, adjacent extreme point. Second, special steps may be required to avoid a large number of pivots (called stalling) and the worst-case, cycling. Thirdly, if the primal RMP has a degenerate optimum, the associated dual has alternative optimal solutions and the “best” set of optimal dual variables must be identified so that they can be passed into the SPs and the primal basic feasible solution that satisfies the LP optimality criterion may be identified.

Several issues must be resolved by future research to enhance existing capabilities. First, as in any LP problem, degeneracy causes inefficiency by requiring a number of pivot operations to advance beyond a degenerate extreme point and may lead to stalling and cycling. Second, SP solution strategies may require a large number of SPs to be solved for each column that is entered into the solution of the RMP. Conclusions and recommendations for future research

Thank you !