1. Column Generation By Soumitra Pal Under the guidance of Prof. A. G. Ranade

2. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

3. Cutting Stock Problem Given larger raw paper rolls Get final rolls of smaller widths 10 53

4. Cutting Stock Problem (2) Raw width (W) = 10 No of finals given below iWidth (w i )Quantity (b i ) 139 2579 3690 4927 Minimize total no of raws reqd. to be cut

5. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

6. 5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50 Min -x 1 -x 2 Objective/ Cost fn Constraints

7. (10,0) (13,5) (12,10) (8,15) (0,10) 5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50

8. -x1-x2 = -10-x1-x2 = -20 -x1-x2 = -30 Cost decreases in this direction

9. (10,0) (13,5) (12,10) (8,15) (0,10) Simplex method

13. (10,0,130,50,30,0) (13,5,110,10,0,0) (12,10,60,0,0,10) (8,15,0,0,10,40) (0,10,0, 60,60,70) (0,0,80, 100,80,50) Basic & non-basic 5x 1 - 8x 2 ≥ -80 -5x 1 - 4x 2 ≥ -100 -5x 1 - 2x 2 ≥ -80 -5x 1 + 2x 2 ≥ -50

15. If all elements of it are non-negative, it is already optimal Otherwise choose the non-basic variable corresponding to most negative value to enter the basic

16. How to find out outgoing basic variable?

17. Few steps of simplex algorithm Compute reduced cost If all >= 0, optimal stop. Otherwise choose the most negative component Corresponding variable enters basis Compute the ratios for finding out leaving basis Update basic variables and B for next step

18. Smart way of doing

19. yd Start

20. yd Reduced cost vector

21. yd Selection of non-basic variable

22. yd y vector

23. yd Selection of leaving basic variable

24. Preparation for next step yd

25. yd Entering variable in 2 nd iteration

26. yd Leaving variable in 2 nd iteration

27. yd Preparation for 3 rd iteration

28. yd Final iteration

29. Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value

30. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

31. First step to solution - Formulation Kantorovich formulation

32. Kantorovich formulation example 1234

33. LP relaxation Integer programming is hard Convert it to a linear program 0 ≤ y k ≤ 1

34. LP relaxation (2) LP relaxation is poor For our example, optimal LP objective is 120.5 The optimal integer objective solution value is 157 Gap is 36.5 It can be as bad as ½ of the integer solution Can we get a better LP relaxation?

35. Another Formulation Gilmore-Gomory Formulation

36. Example formulation 3*1 + 5*0 + 6*1 + 9*0 = 9 ≤ 10

37. LP relaxation of Gilmore-Gomory LP relaxation is better For our example, 156.7 Integer objective solution, 157 Gap is 0.3 Conjecture: Gap is less than 2 for practical cutting stock problems

38. Issues The number of columns are exponential Optimal value is still not integer Gilmore-Gomory proposed an ingenuous way of working with less columns Branch-and-price to solve the other problem

39. Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds

40. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

41. Column Generation In pricing step of simplex algorithm one basic variable leaves and one non-basic variable enters the basis This decision is made from the reduced cost vector The component with most negative value determines the entering variable

42. Column Generation (2) In terms of columns, we need to find That is equivalent to finding a such that For cutting stock problem, c j =1, hence it is equivalent to With implicit constraint Pricing sub-problem

43. Column Generation (3) Any New Columns? STOP (LP Optimal) Solve Restricted Master Problem (RMP) Solve Pricing Problem Update RMP with New Columns No Yes

44. Example formulation

48. Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds Delayed column generation –Restricted master problem –Sub-problem to generate column –Repeated till optimal

49. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

50. Integer solution A formulation of the problem which gives ‘good’ LP relaxation Solved the LP relaxation using column generation But, the LP relaxation is not the ultimate goal, we need integer solution In the example x 2 is 39.5 One way is to rounding

51. What to do if there are multiple fractional variables? The method commonly followed is branch -and-bound technique of solving integer programs Basic fundae –create multiple sub-problems with extra constraints –solve the sub-problems using column generation –take the best of solutions to the sub problems

52. Key is the rules of dividing into sub problems (this is called branching strategy) Things to look into –Branching rule does not destroy column generation sub problem property –Efficiency of the process Ingenuity of the formulation and branching strategy Use of running bounds to prune (fathomed) This technique is called branch-and-price We see another formulation next

53. Done so far Basics of (revised) Simplex algorithm –Formulation –Corners, basic variables, non-basic variables –How to move from corner to corners –Optimal value Formulation of the CSP –LP relaxation –Bounds Delayed column generation –Restricted master problem –Sub-problem to generate column –Repeated till optimal Branch-and-price –Basic fundae

54. Agenda Introduction Basics of Simplex algorithm Formulations for the CSP (Delayed) Column Generation Branch-and-price Flow formulation of CSP & solution Conclusions

55. Flow formulation Bin packing problem Similar to the cutting stock problem binitems

56. w2w2 w2w2 w2w2 w2w2 w1w1 w1w1 w1w1 loss 0 123 45 w2w2 w2w2 0 123 45 Example Bin capacity W = 5 Item sizes – w 1 =3 and w 2 =2

57. Formulation equations Problem is equivalent to finding minimum flow between nodes 0 & W

58. Issue of symmetry in the graph 3 rules to reduce symmetry 1.Arcs are specified according to decreasing width 2.A bin can never start with a loss 3.Number of consecutive arcs corresponding to a single item size must not exceed number of orders

59. w2w2 w2w2 w2w2 w2w2 w1w1 w1w1 w1w1 loss 0 123 4 5 Invalid as per rule 2 Invalid as per rule 1 w2w2 w2w2 w2w2 w1w1 loss 0 123 45

60. Branch-and-price The sub-problem finds an longest unit flow path where cost of each arc is given by y vector Branching heuristics –Xij >= ceil(xij) –Xij <= floor(xij)

61. Few other points Loss constraints from the LP relaxation value is equated with the sum of loss variables With the above constraint it can be shown that the solution will be always integral The algorithm uses this fact by incrementally adding the lower bound This is done finite no. of times No of new constraints are added is finite Algorithm runs in pseudo-polynomial time

62. Conclusions & Future work Column generation is a success story in large scale integer programming The flexibility to work with a few columns make real life problems tractable Ingenuous formulations and branching heuristics are used to improve efficiency Need to explore more applications with different heuristics

63. Acknowledgements Valerio de Carvalho for borrowing some of his explanations exactly http://www- fp.mcs.anl.gov/otc/Guide/CaseStudies/s implex/applet/SimplexTool.html for the java applethttp://www- fp.mcs.anl.gov/otc/Guide/CaseStudies/s implex/applet/SimplexTool.html AMPL & CPLEX tools My guide

65. Thank you!