Column Generation By Soumitra Pal Under the guidance of Prof. A. G. Ranade

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Cutting Stock Problem Given larger raw paper rolls Get final rolls of smaller widths 10 53

Cutting Stock Problem (2) Raw width (W) = 10 No of finals given below iWidth (w i )Quantity (b i ) Minimize total no of raws reqd. to be cut

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

5x 1 - 8x 2 ≥ x 1 - 4x 2 ≥ x 1 - 2x 2 ≥ x 1 + 2x 2 ≥ -50 Min -x 1 -x 2 Objective/ Cost fn Constraints

(10,0) (13,5) (12,10) (8,15) (0,10) 5x 1 - 8x 2 ≥ x 1 - 4x 2 ≥ x 1 - 2x 2 ≥ x 1 + 2x 2 ≥ -50

-x1-x2 = -10-x1-x2 = -20 -x1-x2 = -30 Cost decreases in this direction

(10,0) (13,5) (12,10) (8,15) (0,10) Simplex method

(10,0,130,50,30,0) (13,5,110,10,0,0) (12,10,60,0,0,10) (8,15,0,0,10,40) (0,10,0, 60,60,70) (0,0,80, 100,80,50) Basic & non-basic 5x 1 - 8x 2 ≥ x 1 - 4x 2 ≥ x 1 - 2x 2 ≥ x 1 + 2x 2 ≥ -50

If all elements of it are non-negative, it is already optimal Otherwise choose the non-basic variable corresponding to most negative value to enter the basic

How to find out outgoing basic variable?

Few steps of simplex algorithm Compute reduced cost If all >= 0, optimal stop. Otherwise choose the most negative component Corresponding variable enters basis Compute the ratios for finding out leaving basis Update basic variables and B for next step

Smart way of doing

yd Start

yd Reduced cost vector

yd Selection of non-basic variable

yd y vector

yd Selection of leaving basic variable

Preparation for next step yd

yd Entering variable in 2 nd iteration

yd Leaving variable in 2 nd iteration

yd Preparation for 3 rd iteration

yd Final iteration

Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

First step to solution - Formulation Kantorovich formulation

Kantorovich formulation example 1234

LP relaxation Integer programming is hard Convert it to a linear program 0 ≤ y k ≤ 1

LP relaxation (2) LP relaxation is poor For our example, optimal LP objective is The optimal integer objective solution value is 157 Gap is 36.5 It can be as bad as ½ of the integer solution Can we get a better LP relaxation?

Another Formulation Gilmore-Gomory Formulation

Example formulation 3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10

LP relaxation of Gilmore-Gomory LP relaxation is better For our example, Integer objective solution, 157 Gap is 0.3 Conjecture: Gap is less than 2 for practical cutting stock problems

Issues The number of columns are exponential Optimal value is still not integer Gilmore-Gomory proposed an ingenuous way of working with less columns Branch-and-price to solve the other problem

Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Column Generation In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis This decision is made from the reduced cost vector The component with most negative value determines the entering variable

Column Generation (2) In terms of columns, we need to find That is equivalent to finding a such that For cutting stock problem, c j =1, hence it is equivalent to With implicit constraint Pricing sub-problem

Column Generation (3) Any New Columns? STOP (LP Optimal) Solve Restricted Master Problem (RMP) Solve Pricing Problem Update RMP with New Columns No Yes

Example formulation

Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds Delayed column generation –Restricted master problem –Sub-problem to generate column –Repeated till optimal

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Integer solution A formulation of the problem which gives ‘good’ LP relaxation Solved the LP relaxation using column generation But, the LP relaxation is not the ultimate goal, we need integer solution In the example x 2 is 39.5 One way is to rounding

What to do if there are multiple fractional variables? The method commonly followed is branch -and-bound technique of solving integer programs Basic fundae –create multiple sub-problems with extra constraints –solve the sub-problems using column generation –take the best of solutions to the sub problems

Key is the rules of dividing into sub problems (this is called branching strategy) Things to look into –Branching rule does not destroy column generation sub problem property –Efficiency of the process Ingenuity of the formulation and branching strategy Use of running bounds to prune (fathomed) This technique is called branch-and-price We see another formulation next

Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds Delayed column generation –Restricted master problem –Sub-problem to generate column –Repeated till optimal Branch-and-price –Basic fundae

Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

Flow formulation Bin packing problem Similar to the cutting stock problem binitems

w2w2 w2w2 w2w2 w2w2 w1w1 w1w1 w1w1 loss w2w2 w2w Example Bin capacity W = 5 Item sizes – w 1 =3 and w 2 =2

Formulation equations Problem is equivalent to finding minimum flow between nodes 0 & W

Issue of symmetry in the graph 3 rules to reduce symmetry 1.Arcs are specified according to decreasing width 2.A bin can never start with a loss 3.Number of consecutive arcs corresponding to a single item size must not exceed number of orders

w2w2 w2w2 w2w2 w2w2 w1w1 w1w1 w1w1 loss Invalid as per rule 2 Invalid as per rule 1 w2w2 w2w2 w2w2 w1w1 loss

Branch-and-price The sub-problem finds an longest unit flow path where cost of each arc is given by y vector Branching heuristics –Xij >= ceil(xij) –Xij <= floor(xij)

Few other points Loss constraints from the LP relaxation value is equated with the sum of loss variables With the above constraint it can be shown that the solution will be always integral The algorithm uses this fact by incrementally adding the lower bound This is done finite no. of times No of new constraints are added is finite Algorithm runs in pseudo-polynomial time

Conclusions & Future work Column generation is a success story in large scale integer programming The flexibility to work with a few columns make real life problems tractable Ingenuous formulations and branching heuristics are used to improve efficiency Need to explore more applications with different heuristics

Acknowledgements Valerio de Carvalho for borrowing some of his explanations exactly fp.mcs.anl.gov/otc/Guide/CaseStudies/s implex/applet/SimplexTool.html for the java applethttp://www- fp.mcs.anl.gov/otc/Guide/CaseStudies/s implex/applet/SimplexTool.html AMPL & CPLEX tools My guide

Thank you!