1. LINEAR PROGRAMMING

2. “ A certain wide class of practical problems appears to be just beyond the range of modern computing machinery. These problems occur in everyday life; they run the gamut from some very simple situations that confront an individual to those connected with the national economy as a whole. Typically, these problems involve a complex of different activities in which one wishes to know which activities to emphasize in order to carry out desired objectives under known limitations.” George B. Dantzig, 1948

3. LP History George Dantzig, 1947 –Introduced LP and recognized it as more than a conceptual tool: Computing answers important. –Invented “primal” simplex algorithm. –First LP solved: Laderman, 9 cons., 77 vars., 120 MAN-DAYS. First computer code – 1951 LP used commercially – Early 60s Powerful mainframe codes introduced – Early 70s Computational progress stagnated – Mid 80s Remarkable progress last 15 years (PCs, new computer science and mathematics)

4. Diet Problem * Bob wants to plan a nutritious diet, but he is on a limited budget, so he wants to spend as little money as possible. His nutritional requirements are as follows: 1.2000 kcal 2.55 g protein 3.800 mg calcium * From Linear Programming, by Vaŝek Chvátal

5. Nutritional values Diet Problem Bob is considering the following foods:

6. Variables Diet Problem We can represent the number of servings of each type of food in the diet by the variables: x 1 servings of oatmeal x 2 servings of chicken x 3 servings of eggs x 4 servings of milk x 5 servings of cherry pie x 6 servings of pork and beans

7. Nutritional values Diet Problem Bob is considering the following foods: x1x2x3x4x5x6x1x2x3x4x5x6 KCAL constraint: 110x 1 + 205x 2 + 160x 3 + 160x 4 + 420x 5 + 260x 6  2000 (110x 1 = kcals in oatmeal)

8. LP formulation Diet Problem 110x 1 + 205x 2 + 160x 3 + 160x 4 + 420x 5 + 260x 6  2000 4x 1 + 32x 2 + 13x 3 + 8x 4 + 4x 5 + 14x 6  55 2x 1 + 12x 2 + 54x 3 + 285x 4 + 22x 5 + 80x 6  800 Minimize subject to: Cost Nutritional requirements Bounds 0.3x 1 + 2.40x 2 + 1.30x 3 + 0.90x 4 + 2.0x 5 + 1.9x 6

9. Solution Diet Problem When we solve the preceding LP (using CPLEX, of course) we get a solution value of $6.71, which is achieved with the following menu: 14.24 servings of oatmeal 0 servings of chicken 0 servings of eggs 2.71 servings of milk 0 servings of cherry pie 0 servings of pork and beans

10. A linear program (LP) is an optimization problem of the form

11. Solving Linear Programs

12. x y (0.882,0.706) Objective = 0.9 x + 0.73 y = 40 million Objective = 30 million Feasible Solutions Objective = 0.9 * 0.882 + 0.73 * 0.706 = 13.1 million (0,6) (1,0) Constraint 1 (0,1) (3,0) Constraint 2 (0,1.5) (2,0) Constraint 3 A Graphical Solution (0,0) Maximize 0.90 x + 0.73 y (OBJECTIVE) Subject To Constraint 1: 0.42 x + 0.07 y <= 4200000 Constraint 2: 0.13 x + 0.39 y <= 3900000 Constraint 3: 0.35 x + 0.44 y <= 7000000 x >= 0 y >= 0

13. A graphical representation The Simplex Algorithm We now look at a graphical representation of the simplex method as it solves the following problem: Maximize 3x 1 + 2x 2 + 2x 3 Subject to x 1 + x 3  8 x 1 + x 2  7 x 1 + 2x 2  12 x 1, x 2, x 3  0

14. The Simplex Algorithm x1x1 x2x2 x3x3 (0,0,8) (0,6,8) (2,5,6) (0,6,0) (2,5,0) (7,0,1) (7,0,0) Maximize z = 3x 1 + 2x 2 + 2x 3 z = 0 z = 21 z = 23 Optimal! z = 28

15. Log started (V8.2.0a1) Tue Apr 8 21:59:46 2003 Problem name: supply_chain.lp.gz Constraints : 10810259 Variables : 19916187 Reduced LP has 3140439 rows, and 7314332 variables. Presolve time = 339.43 sec. Iteration log... Iteration: 0 Infeasibility = 1534390803.049594 Iteration: 1247301 Infeasibility = 437397607.938387... Iteration: 2348954 Infeasibility = 10965.599999 Elapsed time = 3776.48 sec. (2349000 iterations) Iteration: 2349222 Objective = 1775629743606097400.000000... Elapsed time = 5396.65 sec. (2687000 iterations) Iteration: 2687616 Objective = 1403833763253068800.000000 Removing shift (10). Primal simplex - Optimal: Objective = 1.4038337633e+18 Solution time = 5410.00 sec. Iterations = 2687616 (2348954) A Real LP: Sample Run

16. MIXED INTEGER PROGRAMMING

17. Mixed-Integer Programming (MIP) (= linear programming + restriction that some [MIP] or all variables must take on integer values.)

18. Manufacturing Cereal

19. Cereal Problem: With setup cost Minimize Total_Cost: 283 x1 + 266 x2 + 187 x3 + 188 x4 + 182 x5 + 181 x6 + 200 y1 + 250 y2 + 400 y3 + 350 y4 + 800 y5 + 1000 y6 Subject To Boxes: x1 + x2 + x3 + x4 + x5 + x6 = 200 S1: - 50 y1 + x1 <= 0 S2: - 70 y2 + x2 <= 0 S3: - 40 y3 + x3 <= 0 S4: - 70 y4 + x4 <= 0 S5: - 100 y5 + x5 <= 0 S6: - 90 y6 + x6 <= 0 Bounds x1, x2, x3, x4, x5, x6 >= 0 y1, y2, y3, y4, y5, y5 = 0 or 1 (binary) End

20. Cereal Problem: With setup cost – Solution log Problem 'cereal.lp' read. Nodes Cuts/ Node Left Objective IInf Best Integer Best Node ItCnt 0 0 38220.0000 1 38220.0000 6 * 0+ 0 0 38520.0000 38220.0000 6 38410.0000 2 38520.0000 Cuts: 4 8 38495.7143 2 38520.0000 Cuts: 4 12 * 0+ 0 0 38510.0000 38495.7143 12 cutoff 38510.0000 Flowcuts: 2 13 Cover cuts applied: 1 Flow cuts applied: 5 Gomory fractional cuts applied: 2 Integer optimal solution: Objective = 3.8510000000e+004 Solution time = 0.01 sec. Iterations = 13 Nodes = 0 Variable Name Solution Value x3 40.000000 x4 60.000000 x5 100.000000 y3 1.000000 y4 1.000000 y5 1.000000 All other variables in the range 1-12 are zero.

21. objective Linear Program LP optimum feasible solutions y x

22. objective IP optimum Integer Program rounding down optimum of LP relaxation feasible solutions = y x optimum of LP relaxation

23. Solving MIPs: Branch and Bound Consider the following integer program: Maximize x + y + 2z Subject to 7x + 2y + 3z  36 5x + 4y + 7z  42 2x + 3y + 5z  28 x, y, z  0, integer (IP 0 )

24. IP 0 IP 2 IP 1 Best IP value Best IP solution 10 / Integral IP 4 IP 3 Infeasible IP 6 IP 5 / 11 Integral IP 8 IP 7 Obj.  incumbent Branch & Bound: Example

26. The Pill Salesman Diet Problem A pill salesman offers Bob energy, protein, and calcium pills to fulfill his nutritional needs. We will represent the costs of each of the pills as follows: y 1 cost (in dollars) of 1 kcal energy pill y 2 cost (in dollars) of 1 g protein pill y 3 cost (in dollars) of 1mg calcium pill

27. LP constraints Diet Problem 110x 1 + 205x 2 + 160x 3 + 160x 4 + 420x 5 + 260x 6  2000 y 1 4x 1 + 32x 2 + 13x 3 + 8x 4 + 4x 5 + 14x 6  55 y 2 2x 1 + 12x 2 + 54x 3 + 285x 4 + 22x 5 + 80x 6  800 y 3 Nutritional requirements

28. LP constraints Diet Problem 110x 1 + 205x 2 + 160x 3 + 160x 4 + 420x 5 + 260x 6  2000 y 1 kcal 4x 1 + 32x 2 + 13x 3 + 8x 4 + 4x 5 + 14x 6  55 y 2 protein 2x 1 + 12x 2 + 54x 3 + 285x 4 + 22x 5 + 80x 6  800 y 3 calcium Nutritional requirements x 1 = servings of oatmeal: The cost of the nutrients in one serving of oatmeal shouldn’t exceed the cost of just buying one serving of oatmeal: 110y 1 + 4y 2 + 2y 3  0.3 (4 y 2 = cost of protein in oatmeal) MinimizeCost 0.3x 1 + 2.40x 2 + 1.30x 3 + 0.90x 4 + 2.0x 5 + 1.9x 6

29. The salesman’s requirements Diet Problem The pill salesman wants to make as much money as possible, given Bob’s constraints. He knows Bob wants 2000 kcal, 55g protein, and 800 mg calcium, so his problem is as follows: Maximize 2000y 1 + 55y 2 + 800y 3 Subject to 110y 1 + 4y 2 + 2y 3  0.3 205y 1 + 32y 2 + 12y 3  2.4 160y 1 + 13y 2 + 54y 3  1.3 160y 1 + 8y 2 + 285y 3  0.9 420y 1 + 4y 2 + 22y 3  2.0 260y 1 + 14y 2 + 80y 3  1.9 y 1, y 2, y 3  0

30. Solution Diet Problem Solving this LP gives us the following pill prices: $0.27 for 1 kcal energy pill $0.00 for 1 g protein pill $0.16 for 1mg calcium pill Total cost = 0.27 (2000) + 0.16 (800) = $6.71 THE SAME AS THE LOWEST COST DIET!

31. Diet Problem Weak Duality: Cost of any feas. diet  Price of any feas. pills Strong Duality: (Von Neumann, 1947) Optimal cost of diet = Optimal cost of pills

32. Diet Problem MORAL: The values of the dual variables, which are always available as part of LP solution, equal the marginal value of corresponding resources.

33. The Simplex Algorithm Consider the following linear program: Maximize 5x 1 + 4x 2 + 3x 3 Subject to 2x 1 + 3x 2 + x 3  5 4x 1 + x 2 + 2x 3  11 3x 1 + 4x 2 + 2x 3  8 x 1, x 2, x 3  0.

34. Slack Variables The Simplex Algorithm We add slack variables to each constraint to get equations: Maximize 5x 1 + 4x 2 + 3x 3 Subject to 2x 1 + 3x 2 + x 3 + x 4 = 5 4x 1 + x 2 + 2x 3 + x 5 = 11 3x 1 + 4x 2 + 2x 3 + x 6 = 8 x 1, x 2, x 3, x 4, x 5, x 6  0.

35. Getting started The Simplex Algorithm Rewriting we can express the slack variables and the objective, denoted z, in terms of the original variables. Setting x 1 =x 2 =x 3 =0, we easily read off the solution x 4 =5, x 5 =11, x 6 =8, z=0. z = 5x 1 + 4x 2 + 3x 3 x 4 = 5 - 2x 1 - 3x 2 - x 3 x 5 = 11 - 4x 1 - x 2 - 2x 3 x 6 = 8 - 3x 1 - 4x 2 - 2x 3

36. 1 st Iteration The Simplex Algorithm We want to increase the value of z from its current value of 0. Clearly, increasing the value of x 1 will increase the value of z. By how much can we increase it? z = 5x 1 + 4x 2 + 3x 3 x 4 = 5 - 2x 1 - 3x 2 - x 3 x 5 = 11 - 4x 1 - x 2 - 2x 3 x 6 = 8 - 3x 1 - 4x 2 - 2x 3 x 4  0  5 – 2x 1  0  5  2x 1  x 1  5/2 x 5  0  11 – 4x 1  0  11  4x 1  x 1  11/4 x 6  0  8 – 3x 1  0  8  3x 1  x 1  8/3

37. 2 nd Iteration The Simplex Algorithm Now we see that increasing the value of x 3 will improve the value of z. z = 25/2 - 7/2x 2 + 1/2x 3 - 1/2x 4 x 1 = 5/2 - 3/2x 2 - 1/2x 3 - 1/2x 4 x 5 = 1 + 5x 2 + 2x 4 x 6 = 1/2 + 1/2x 2 - 1/2x 3 + 3/2x 4 x 1  0  5/2 – 1/2x 3  0  5/2  1/2x 3  x 3  5 x 5  0  x 3 unbounded x 6  0  1/2 – 1/2x 3  0  1/2  1/2x 3  x 1  1

38. The Simplex Algorithm z = 13 - 3x 2 - x 4 - x 6 x 1 = 2 - 2x 2 - 2x 4 + x 6 x 5 = 1 + 5x 2 + 2x 4 x 3 = 1 + x 2 + 3x 4 - 2x 6 Optimal!!!

39. Barrier Algorithms Simplex solution path Barrier central path o Predictor o Corrector Optimum