1. Column Generation: Theory, Algorithms and Applications Dr Natashia Boland Department of Mathematics and Statistics The University of Melbourne

2. Outline  Linear and integer programs  A shipping problem: column generation formulation  Column generation and why it works  Successful applications  Difficulties  Towards a cure: duality  A general framework for stabilisation  Progress in column generation and the future

3. Linear Programs x i = i th decision variable, i = 1,…,n min c 1 x 1 + c 2 x 2 +…+ c n x n s.t. a 11 x 1 + a 12 x 2 +…+ a 1n x n  b 1 a 21 x 1 + a 22 x 2 +…+ a 2n x n  b 2 a m1 x 1 + a m2 x 2 +…+ a mn x n  b m linear objective function m linear constraints x i integer, i = 1,…,n Integer

4. A Shipping Problem*  Delivering product to customers around the world  Shipping contracts entered into for next 12 months  Contract specifies ship s can be used at most once in any m s consecutive months – the spread  Contract specifies min and max limits on number of times ship used over year Operations Research Group Pty Ltd J. Horton, T. Surendonk, E. Jago * See Boland and Surendonk, 2001

5. Exercising a Shipping Contract JanFebMarAprMayJunJulAugSep Spread = 3 months Spread = 4 months CUSTOMER DEMAND

6. Exercising a Shipping Contract JanFebMarAprMayJunJulAugSep Spread = 3 months Spread = 4 months CUSTOMER DEMAND 1 2 X

7. An Integer Programming Model x ist = 1,if ship s delivers to customer i in period t 0,otherwise min  cost is x ist i,s,t  capacity s x ist  demand of customer i in period t s s.t.

8. Contract Constraints i, t min s   x ist  max s  ( x ist + x is(t+1) + … + x is(t+m s -1) )  1 i For each ship s for all t=1,… spread of m s periods

9. Outcomes  Implemented using top-grade commercial integer programming solver (ILOG Cplex)  16 customers, about 30 ships  Ran for many hours – out of memory  Best solution found might be as bad as 16% from optimal WHAT’S GOING WRONG?

10. Integrality Gap Integer program min cx s.t. Ax  b x  {0,1} n Value of integer program >> Value of relaxation Linear programming relaxation min cx s.t. Ax  b 0  x  1

11. Re-formulation: Ship Combinations JanFebMarAprMayJunJulAugSep  capacity s x ist  demand of customer i in period t s Which combination of ships should be used to meet customer i ’s demand in period t ?

12. A Column Generation Model y ikt = 1,if combination k used for customer i in period t 0,otherwise For customer i, period t K it = set of all combinations of ships that can meet customer’s demand in that period For k  K it

13. The Model min  (  cost is ) y ikt i,t,k  K it  y ikt = 1 k  K it s.t. sksk customer i demand met in period t min s   y ikt  max s i,t,k: s  k  ( y ikt + y ik(t+1) + … + y ik(t+m s -1) )  1 i,k: s  k contract constraints for ship s for all t=1,…

14. Issues  There are a very large number of variables - how can we solve even the linear programming relaxation?  Why will this reformulation help? ANSWERS: Column generation The linear program better approximates the integer program - WHY?

15. Column Generation 1.Select a small number of variables and solve the linear program using only these. 2.Find an unused variable which, if included, would most improve the objective value, or determine that there is none - the linear program has been solved: stop. 3.Include the variable in the linear program, re- solve it, and go to step 2.  Model this as an optimization problem Solve the column generation subproblem

16. The Shipping Subproblem For each customer i and period t, find a combination of ships that will most improve the objective:  capacity s v s  demand of customer i in period t s v s = 1,if ship s used in combination 0,otherwise min  f(cost is, s ) v s s.t. dual variables from linear program

17. Shipping Column Generation 1.Select a small number of combinations for each customer and period. Solve the linear program using only these. 2. For each customer and period, solve the shipping subproblem: if its value < cut-off include the optimal combination in the linear program. 3. If no new combinations included in step 2, the linear program has been solved: stop. 4.Re-solve the linear program and go to step 2.

18. Outcomes  Implemented using ILOG Cplex (and AMPL)  Solved in less than 2 minutes  Step 2 performed 34 times  1700 combinations generated  Solution to linear program was integer optimal WHAT’S GOING RIGHT?!

19. Dantzig-Wolfe Decomposition Integer program min cx s.t. Ax  b Dx  g x  {0,1} n Value integer program >> Value linear relaxation 0  x  1 Linear relaxation D-W Decomposition min cx s.t. Ax  b x  convDy  g y  {0,1} n Value integer program  Value D-W decomposition  contract demand

20. Geometry           demand linear relaxation D-W relax. contract

21. Successful Applications: A Brief History Cutting stock problems Air crew scheduling Aircraft fleeting and routing Crew rostering Vehicle routing Global shipping Multi-item lot-sizing Optical telecommunications network design Cancer radiation treatment using IMRT      1960 1980 1990 2000

22. Difficulties with Column Generation  Tailing off – slow convergence  Getting integrality – column generation within branch-and-bound and in combination with cutting planes  Columns with set-up costs

23. Difficulties with Column Generation  Tailing off – slow convergence  Getting integrality – column generation within branch-and-bound and in combination with cutting planes  Columns with set-up costs

24. A Dual Approach Integer program min cx s.t. Ax = b Dx  g x  {0,1} n Lagrangian Relaxation min cx s.t. x  Y Value integer program  Value Lagrangian relaxation Y + (b – Ax) For all : = z( )

25. Properties of the Lagrangian Relaxation  z( ) is a piecewise affine concave function  To find the best lower bound, solve Lagrangian Dual max z( ) s.t.   m  To find a subgradient of z( ), solve the Lagrangian relaxation

26. Nonsmooth Methods  Kelley’s cutting plane  Subgradient optimisation (Held & Karp)  Projection methods (Conn & Cornuejols)  Method of analytic centres (Goffin et al.)  Box-step method (Marsten)  Bundle methods (Lemarechal, Kiwiel)  Linear-norm bundle methods (Kim et al.)  Penalty-step method (du Merle, Desrosiers et al.)  Partial-step method (Neame) = column generation!

27. A Common Framework Iteratively solve max ( min { cx + (b – Ax) : x  Y i } ) s.t.   m + s i ( ) where Y i  Y Joint work with Danny Ralph & Phil Neame stabilisation function

28. Stabilisation Functions Box-step Bundle Bundle – 1-norm du Merle et al.

29. Better Theory and Algorithms  An algorithm for general stabilisation functions  General conditions on stabilisation functions under which the algorithm will -Converge -Terminate finitely  Termination of earlier methods: special cases  These imply conditions on the parameters defining the stabilisation functions  New (hybrid) stabilisation functions

30. Numerical Improvements  Binary cutting stock problem  Average results over 20 problems drawn at random from 3 problem classes CPU Time (s) Standard Col. Gen. Box-step Variant Bundle 1-norm Hybrid 16151820 319117121113 216123111113

31. Progress in Column Generation  Tailing off – slow convergence  Getting integrality – column generation within branch-and-bound and in combination with cutting planes*  Columns with set-up costs** * See Barnhart, Boland et al. 1998 & Boland et al. 2000 ** See Vanderbeck 2000  