1. Explorations in Artificial Intelligence Prof. Carla P. Gomes gomes@cs.cornell.edu Module 7 Part 2 Solving Linear Programming

2. Solving LP

3. LP: Algorithms Simplex. (Dantzig 1947) –Developed shortly after WWII in response to logistical problems: used for 1948 Berlin airlift. –Practical solution method that moves from one extreme point to a neighboring extreme point. –Finite (exponential) complexity, but no polynomial implementation known.

4. LP: Polynomial Algorithms Ellipsoid. (Khachian 1979, 1980) –Solvable in polynomial time: O(n 4 L) bit operations. n = # variables L = # bits in input –Theoretical tour de force. –Not remotely practical. Karmarkar's algorithm. (Karmarkar 1984) –O(n 3.5 L). –Polynomial and reasonably efficient implementations possible. Interior point algorithms. –O(n 3 L). –Competitive with simplex! will likely dominate on large problems soon –Extends to even more general problems.

5. Simplex Method

6. The simplex method has exponential time complexity (c n ) (worst case complexity). However, in practice it works well in general. Key advantage of simplex methods – capability of performing postoptimality / sensitivity analysis (duality); Simplex Method in Practice

7. Key Notion in Simplex: Corner Point Solutions Corner-point feasible solution – special solution that plays a key role when the simplex method searches for an optimal solution. Relationship between optimal solutions and CPF solutions: –Any LP with feasible solutions and bounded feasible region  (1) the problem must possess CPF solutions and at least one optimal solution (2) the best CPF solution must be an optimal solution  f the problem has exactly one optimal solution it must be a CFP solution If the problem has multiple optimal solutions, at least two must be CPF solutions

8. Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27

9. Simplex – Key Concepts Concept 1 – CPF solutions –Simplex methods focuses only on CPF solutions (finite set) Concept 2 – Flow of simplex method –Iterative procedure: –Initialization – find initial CPF solution –Optimality test if optimal stop if not optimal go to next iteration –Iteration – find a better CPF solution; go to 2. Concept 3– Initialization –Whenever possible pick the origin; otherwise special procedure

10. Simplex – Key Concepts (cont.) Concept 4 – Path to optimal solution –Simplex methods always chooses a CPF solution adjacent to the current one –The entire path to the optimal solution is along the edges of the feasible solution Concept 5 – Choice of new CPF solution –From a CPF solution consider all edges emanating from it but it does not solve for each adjacent CPF solution – it simply identifies the rate of improvement in Z along a given edge and than it picks the one with the largest improvement Concept 6– Optimality Test –From a given CPF solution check if there is an edge that gives a positive rate of improvement in Z. If not, current CPF is optimal.

11. Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27

12. Augmented Form for Wyndor Glass Maximize Z= 3 x 1 + 5 x 2 subject to x 1 ≤ 4 2x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 and x 1 ≥ 0, x 2 ≥ 0 Maximize Z= 3 x1 + 5 x2 + 0 s 1 + 0 s 2 + 0 s 3 subject to x 1 + s 1 =4 2x 2 + s 2 =12 3x 1 + 2x 2 + s 3 = 18 and x 1 ≥ 0, x 2 ≥ 0, s 1 ≥ 0, s 2 ≥ 0, s 3 ≥ 0

13. Simplex as an algebraic procedure System of functional constraints n (5) variables (5) and m (3) equations  2 degrees of freedom, (i.e., we can set those two variables to any arbitrary values); they are the nonbasic variables; the other variables are the basic variables; –Simplex chooses to set the non-basic variables to ZERO. –Simplex solves the simultaneous equations to set the values of the basic variables;

14. Properties of Basic Solutions A basic solution is composed of: –Non-basic variables – number of non-basic variables equals (total number of variables - number of functional constraints) They are set to ZERO –Basic variables – number of basic variables equals number of functional constraints Their values results from solving the system of functional constraints (non-basic variables set to 0) They form the Basic Feasible solution – it is a basic solution that satisfies the non-negativity constraints Adjacent basic feasible solutions – all but one of their basic (non-basic) variables are the same  moving from one basic feasible solution to an adjacent one involves switching one variable from non-basic to basic and one variable from basic to non-basic (check graph)

15. Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27 (x1,x2,s1,s2,s3) (0,0,4,12,18) Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. (4,0,0,12,6) (0,6,4,0,6) (2,4,2,0,0) (4,3,0,3,0)

16. Getting ready for the Simplex Standard form –<= constraints –Non-negativity constraints on all variables –Positive right hand sides (if these assumptions are not valid  additional adjustments need to be done) Transform the objective function and constraints into equalities (introduction of slack variables)

17. Simplex Procedure Initialization – origin whenever possible (decision variables 0) (okay if standard form with positive RHS’s; basic feasible solution (BFS): each equation has a basic variable with coefficient 1 (slack variable = RHS) and the variable does not appear in any other eq; the decision variables are the non-basic variables set to 0) Optimality test – is current BFS optimal? ( the coefficients of the objective function of the non-basic variables gives the rate of improvement in Z )

18. Iteration – move to a better adjacent BFS –a)Variable entering the basis –Consider non-basic variables (Graphically - Consider edges emanating from current CPF solution) –Pick the variable that increases Z at a faster rate –b)Variable leaving the basis –One of the basic variables will become non-basic; write all the basic variables as a function of the entering variable; the most stringent value (i.e., smallest) will be the value for the new entering variable; the basic variable associated with the most stringent constraint will become non-basic, leaving the basis. – (Graphically – where to stop? As much as possible without leaving the feasible region) –c)Solving for the new BFS –Objective of this step: convert the system of equations into a more convenient form: (1) to perform optimality test and (2) to perform next iteration if necessary –New basic variable should have coefficient 1 in the equation of the leaving variable and 0 in all the other equations, including the objective function; –Valid operations: Multiplication (or division) of an equation by a non-zero constant Addition (or subtraction) of a multiple of an equation to (from) another eqution Simplex Procedure (cont.)

19. Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27 (x1,x2,s1,s2,s3) (0,0,4,12,18) Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. (4,0,0,12,6) (0,6,4,0,6) (2,4,2,0,0) (4,3,0,3,0)

20. Wyndor Glass 1 st Simplex Tableau Optimal? Entering Variable?

21. What variable will enter the basis? Why? What variable will leave the basis? Why? What transformations do we need to perform to the tableau to get the new basic variable into the right format?

22. What operations did we perform? 1 – divide the pivot row by 2; 2 – multiplied the new pivot row by (-2) and added it to eq. 3 3 – multiplied the new pivot row by (5) and added it to eq. 0 What are the new basic / non-basic variables?

23. Optimal? Entering Variable?

24. What operations did we perform? 1 – divide the pivot row by ; 2 – multiplied the new pivot row by ( ) and added it to eq. 1 3 – multiplied the new pivot row by ( ) and added it to eq. 0 What are the new basic / non-basic variables?

25. Optimal?

26. Simplex Method in Tabular Form

27. Modeling Languages

28. A mathematical language that has been designed to efficiently formulate large mathematical models (with thousand of variables/constraints). Examples of modeling languages: AMPL; MPL; GAMS; LINGO. Textbook (CD-ROM): –MPL (student version; also from www.maximalsoftware.com) It provides an interface to to Excel (importing and exporting Excel ranges) Powerful state-of-the-art LP based solver  CPLEX –LINDO (also available from www.lindo.com) LINDO (Traditional optimizer) What’s Best – spreadsheet optimizer LINGO (Linear and Non-liner programming) –Formulations in MPL, LINGO, LINDO and What’s best for all the examples in the book.

29. MPL: Just ignore the message…

30. MPL/CPLEX Activating the license

31. Acknowledgment from Cplex

32. Email from Cplex From hillier-book-register@ilog.com Thu Feb 24 12:21:30 2005 Date: Thu, 24 Feb 2005 09:16:51 -0800 (PST) From: hillier-book-register@ilog.com To: gomes@cs.cornell.edu Subject: License Instructions for MPL and CPLEX Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 2.60 (1.212-2003-09-23-exp) on sundial.cs.cornell.edu X-Spam-Status: No, hits=-1.4 required=5.0 tests=BAYES_01,NO_REAL_NAME autolearn=no version=2.60 X-Spam-Level: Dear Carla Gomes, Thank you for registering your copy of MPL and CPLEX provided with the book Hillier and Lieberman, Introduction to Operations Research, Seventh Edition. Instructions for activating your license appear below. If you have any questions or require support, please contact Maximal Software at info@maximal-usa.com. The steps for activating your license are listed below. Please note the special activation password listed in Step 3. etc

33. MPL

34. TITLE WyndorGlass; MODEL MAX Z = 3 X1 + 5 X2; SUBJECT TO X1 <= 4; 2 X2 <= 12; 3 X1 + 2 X2 <= 18; END Note: default assumption – non-negativity constraints (can be changed)

35. MPL: Solution Options

36. Checking Syntax and Runnig

37. Syntax

38. Solving it with Cplex

39. Optimal Solution

40. Solution File MPL Modeling System - Copyright (c) 1988-2000, Maximal Software, Inc. -------------------------------------------------------------------------------- MODEL STATISTICS Problem name: WyndorGlass Filename: my-wyndor.mpl Date: February 24, 2005 Time: 11:57 Parsing time: 0.05 sec Solver: CPLEX Objective value: 36.0000000000 Iterations: 0 Solution time: 0.00 sec Constraints: 3 Variables: 2 Nonzeros: 4 Density: 67 % SOLUTION RESULT Optimal solution found MAX Z = 36.0000

41. DECISION VARIABLES PLAIN VARIABLES Variable Name Activity Reduced Cost ------------------------------------------------------ X1 2.0000 0.0000 X2 6.0000 0.0000 ------------------------------------------------------ CONSTRAINTS PLAIN CONSTRAINTS Constraint Name Slack Shadow Price ------------------------------------------------------ c1 2.0000 0.0000 c2 0.0000 1.5000 c3 0.0000 1.0000 ------------------------------------------------------ RANGES OBJECTIVE PLAIN VARIABLES Variable Name Coefficient Lower Range Upper Range ----------------------------------------------------------------------- X1 3.0000 0.0000 7.5000 X2 5.0000 2.0000 1E+020 ----------------------------------------------------------------------- RANGES RHS PLAIN CONSTRAINTS Constraint Name RHS Value Lower Bound Upper Bound ----------------------------------------------------------------------- c1 4.0000 2.0000 1E+020 c2 12.0000 6.0000 18.0000 c3 18.0000 12.0000 24.0000 ----------------------------------------------------------------------- END Solution File (cont.)

42. MPL: A more compact version (good for large-scale problems) TITLE WyndorGlass; INDEX product := (Door, Window); plant := 1..3; DATA TimeAvail[plant] := (4, 12, 18); ProdTime[plant, product] := (1, 0, 0, 2, 3, 2); Profit[product] := (3.00, 5.00); VARIABLES Produce[product] -> Prod; MODEL MAX TotalProfit = SUM(product: Profit * Produce); SUBJECT TO TimeCapacity[plant] -> TimeCap: SUM(product: ProdTime * Produce) <= TimeAvail; END

43. Excel

44. Developing a Spreadsheet Model Step #2: Changing Cells –Add a cell in the spreadsheet for every decision variable. –If you don’t have any particular initial values, just enter 0 in each. –It is a good idea to color code these “changing cells” (e.g., yellow with border). Step #1: Data Cells (parameters) –Enter all of the data (parameters) for the problem on the spreadsheet. –Make consistent use of rows and columns. –It is a good idea to color code these “data/parameter cells” (e.g., light blue).

45. Developing a Spreadsheet Model Step #3: Target Cell –Develop an equation that defines the objective of the model. –Typically this equation involves the data cells and the changing cells in order to determine a quantity of interest (e.g., total profit or total cost). –It is a good idea to color code this cell (e.g., orange with heavy border).

46. Developing a Spreadsheet Model Step #4: Constraints –For any resource that is restricted, calculate the amount of that resource used in a cell on the spreadsheet (an output cell). –Define the constraint in three consecutive cells. For example, if Quantity A ≤ Quantity B, put these three items (Quantity A, ≤, Quantity B) in consecutive cells.

47. A Trial Solution The spreadsheet for the Wyndor problem with a trial solution (4 doors and 3 windows) entered into the changing cells.

48. Identifying the Target Cell and Changing Cells Choose the “Solver” from the Tools menu. Select the cell you wish to optimize in the “Set Target Cell” window. Choose “Max” or “Min” depending on whether you want to maximize or minimize the target cell. Enter all the changing cells in the “By Changing Cells” window.

49. Adding Constraints To begin entering constraints, click the “Add” button to the right of the constraints window. Fill in the entries in the resulting Add Constraint dialogue box.

50. The Complete Solver Dialogue Box

51. Some Important Options Click on the “Options” button, and click in both the “Assume Linear Model” and the “Assume Non-Negative” box. –“Assume Linear Model” tells the Solver that this is a linear programming model. –“Assume Non-Negative” adds nonnegativity constraints to all the changing cells.

52. The Solver Results Dialogue Box

53. The Optimal Solution

54. Requesting the Sensitivity Analysis Report

55. Sensitivity Analysis Report

56. Duality

57. Every maximization LP problem in the standard form gives rise to a minimization LP problem called the dual problem Every feasible solution in one yields a bound on the optimal value of the other If one of the problems has an optimal solution, so does the other and the two optimal values coincide These results have very interesting economic interpretations

58. Bounds on the optimal value Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. Lower bound (LB) on Z*  Z* ≥ LB any feasible solution Upper bound (UB) on Z*  Z* ≤ UB – how do we come up with upper bounds for Z* for a maximization LP problem?

59. “Guessing” upper bounds Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. 1 – multiplying the 3 rd costraint by 3  9x1 + 6x2≤ 54 Z= 3 x1 + 5 x2 ≤ 9x1 + 6x2≤ 54  Z*≤ 54 2 – multiplying the 3 rd constraint by 2.5  7.5 x1 + 5 x2≤ 45 Z= 3 x1 + 5 x2 ≤ 7.5 x1 + 6x2≤ 45  Z*≤ 45 3 – multiplying the 2 nd constraint by 2 and add it to the 3 rd  3x1 + 6x2≤ 42 Z= 3 x1 + 5 x2 ≤ 3x1 + 6x2≤ 54  Z*≤ 42 4 – multiplying the 2 nd constraint by 1.5 and add it to the 3 rd  3x1 + 5x2≤ 36 Z= 3 x1 + 5 x2 ≤ 3x1 + 5x2≤ 54  Z*≤ 36

60. A principled way of finding upper-bounds on Z* Dual Problem Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1 ≤ 4  y1 2x2≤ 12  y2 3x1 + 2x2≤ 18  y3 and x1 ≥ 0, x2 ≥ 0. We construct linear combinations of constraints Minimize W= 4 y 1 + 12 y 2 + 12 y 3 subject to y 1 + 3 y 3 ≥ 3 2 y 2 + 2 y 3 ≥ 5 and y 1 ≥ 0, y 2 ≥ 0, y 3 ≥ 0 Each multiplier must be non-negative We want to minimize the biund The sum of the coefficients at least equal the corresponding one for objective function

61. Primal vs. Dual Problem Maximize Z= 3 x 1 + 5 x 2 subject to x 1 ≤ 4 2x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 and x 1 ≥ 0, x 2 ≥ 0. Minimize W= 4 y 1 + 12 y 2 + 12 y 3 subject to y 1 + 3 y 3 ≥ 3 2 y 2 + 2 y 3 ≥ 5 and y 1 ≥ 0, y 2 ≥ 0, y 3 ≥ 0

62. Primal vs. Dual Max Z = c 1 x 1 + c 2 x 2 + …. + c n x n Subject to: a 11 x 1 + a 12 x 2 + … + a 1n x n ≤ b 1 a 21 x 1 + a 22 x 2 + … + a 2n x n ≤ b 2 … a m1 x 1 + a m2 x 2 + … + a mn x n ≤ b m x 1, x 2, …, x n ≥ 0 Min W = b 1 y 1 + b 2 y 2 + …. + b m y m Subject to: a 11 y 1 + a 21 y 2 + … + a m1 y m ≥ c 1 a 12 y 1 + a 22 y 2 + … + a m2 y m ≥ c 2 … a 1n y 1 + a 2n y 2 + … + a mn y n ≥ c n y 1, y 2, …, y m ≥ 0

63. Max s.t Min Primal Dual

64. Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Z*Z* Primal feasible solutions (Maximization) Lower Bounds on Dual Optimum Dual feasible solutions (Minimization) Upper Bounds on Primal Optimum Weak duality property

65. Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual

66. Strong Duality Property

67. Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Z*Z* Primal feasible solutions (Maximization) Lower Bounds on Dual Optimum Dual feasible solutions (Minimization) Upper Bounds on Primal Optimum Weak duality property Strong duality property Primal Opt. = Dual Opt.

68. Complementary Slackness Optimal solutions Let x 1 *, x 2 *, …x n * be a primal feasible solution and a y 1 *, y 2 *, …y m * dual feasible solution. Necessary and sufficient conditions for simultaneous optimality of x 1 *, x 2 *, …x n * and y 1 *, y 2 *, …y m * are: This result is very useful for checking the optimality of solutions

69. Relationships between the primal and dual problems If one problem has feasible solutions and a bounded objective function (and therefore an optimal solution), then so does the other problem (both weak and strong duality apply) If one problem has feasible solutions and an unbounded objective function (therefore no optimal solution) then the other problem has no feasible solutions If one problem has no feasible solutions than the other problem has no feasible solutions or an unbounded objective function.

70. Complementary Slackness Optimal solutions

71. Primal-Dual Combinations

72. Adapting to Other Forms What if our problem is not in standard form? –We can always transform it to the standard form and then construct the dual;

74. Dual of the dual Note: it is not important which problem we call dual and which problem we call primal given the symmetry property of the primal dual relationships. In general we call primal the model formulated to fit the actual problem.

75. Complexity of Simplex Method Primal vs. Dual How long does it take to solve an LP using the simplex method? –Several factors but the most important one seems to be the number of functional constraints. Computation tends to be proportional to the cube of the number of functional constraints in an LP. –The number of variables is a relatively minor factor (assuming revised simplex method) –The density of the matrix of technological coefficients is also a factor – the sparser the matrix (i.e., the larger the number of zeroes) the faster the simplex method; – Real world problems tend to be sparse, i.e., “sparcity” of 5% or even 1%, which leads to fast runs.

76. Primal vs. Dual? So, the size of the problem, may determine whether to use the simplex method on the primal or dual problem. If the primal has a large number of constraints and a small number of variables it is better to apply the simplex method to the dual (since it will have a small number of constraints).

77. Dual Simplex Method This method is based on the duality results. It is a mirror image of the simplex method: –the simplex method deals with primal feasible solutions (but not dual feasible), moving toward a solution that is dual feasible; – the dual method deals with basic solutions in the primal problem that are dual feasible but not primal feasible. It moves toward an optimal solution by striving to achieve primal feasibility as well.

78. Post-optimality Analysis and Economic Interpretation of Duality

79. Post-optimality Analysis Post-optimality – very important phase of modeling. Duality plays and important role in post-optimality analysis Simplex provides several tools to perform post-optimality analysis

80. Economic Interpretation of Duality LP problems – quite often can be interpreted as allocating resources to activities. Let’s consider the standard form: x i >= 0, (i =1,2,…,n)

81. What if we change our resources – can we improve our optimal solution? Resources – m (plants) Activities – n (2 products) Wyndor Glass problem  optimal product mix --- allocation of resources to activities i.e., choose the levels of the activities that achieve best overall measure of performance

82. Sensitivity Analysis How would changes in the problem ’ s objective function coefficients or right-hand side values change the optimal solution?

83. Dual Variables (Shadow Prices) y 1 *= 0  dual variable (shadow price) for resource 1 y 2 *= 1.5  dual variable (shadow price) for resource 2 y 3 *= 1  dual variable (shadow price) for resource 3 How much does Z increase if we increase resource 2 by 1 unit (i.e., b 2 = 12  b2=13)?

84. Graphical Analysis of Dual variables – Variation in RHS Increasing level of resource 2 (b 2 ) (2,6) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 (5/3,13/2) ∆ Z=1.5 = y 2 * Why is y 1 *=0?

85. Economic Interpretation of Dual Variables The dual variable associated with resource i (also called shadow price), denoted by y i *, measures the marginal value of this resource, i.e., the rate at which Z could be increased by (slightly) increasing the amount of this resource (bi), assuming everything else stays the same. The dual variable y i * is identified by the simplex method as the coefficient of the i th slack variable in row 0 of the final simplex tableau.

86. Dual Variables: binding and non-binding constraints The shadow prices (dual variables) associated with non-binding constraints are necessarily 0 (complementary optimal slackness)  there is a surplus of non- binding resource and therefore increasing it will not increase the optimal solution. Economist refer to such resources as free resources (shadow price =0) Binding constraints on the other hand correspond to scarce resources – there is no surplus. In general they have a positive shadow price.

87. Does Z always increase at the same rate if we keep increasing the amount of resource 2? (2,6) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 (5/3,13/2) ∆ Z=1.5 = y 2 * What if b 2 > 18 (i.e., 2W>18)?  the optimal solution will stay at (0,9) for b 2 >=18 b 2 =18 (0,9)

88. If b 2 < 6 the solution will no longer vary proportionally. The optimal solution varies proportionally to the variation in b 2 only if 6 <= b 2 <=18. In other words, the current basis remains optimal for 6 ≤ b 2 ≤ 18, but the decision variable values and z-value will change. Does Z always decrease at the same rate if we decrease resource 2? (2,6) (5/3,13/2) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 ∆ Z=1.5 = y 2 * b 2 =6

89. A dual variable y i * gives us the rate at which Z could be increased by increasing the amount of resource i slightly. However this is only true for a small increase in the amount of the resource. I.e., this definition applies only if the change in the RHS of constraint i leaves the current basis optimal. It also assumes everything else stays the same. Another interpretation of y i * is: if a premium price must be paid for the resource i in the market place, y i * is the maximum premium (excess over the regular price) that would be worth paying.

90. Optimal Basis in the Wyndor Glass Problem How can we characterize (verbally) the optimal basis of the Wyndor Glass problem? –Plant 1 – unutilized capacity (non-binding constraint) –Plant 2 – fully utilized capacity (binding constraint) –Plant 3 - fully utilized capacity (binding constraint)

91. How do we interpret the intervals? If we change one coefficient in the RHS, say capacity of plant 2, by  the “basis” remains optimal, that is, the same equations remain binding. So long as the basis remains optimal, the shadow prices are unchanged. The basic feasible solution varies linearly with . If  is big enough or small enough the basis will change.

92. Sensitivity analysis for c1 How much can we vary c1 without changing the current basic optimal solution?

93. Sensitivity analysis for c1 Our objective function is: Z= c 1 D+5W=k  slope of iso-profit line is: isoprofit line How much can c1 vary until the slope of the iso-profit line equals the slope of constraint 2 and constraint 3?

94. How much can c1 vary until the slope of the iso-profit line equals the slope of constraint 2 and constraint 3? Slope of constraint 2  0 Slope of constraint 3  -3/2

95. Importance of Sensitivity Analysis Sensitivity analysis is important for several reasons: Values of LP parameters might change. If a parameter changes, sensitivity analysis shows it is unnecessary to solve the problem again. For example in the Wyndor problem, if the profit contribution of product 1 changes to 5, sensitivity analysis shows the current solution remains optimal. Uncertainty about LP parameters. In the Wyndor problem for example, if the capacity of plant 1 decreases to 2, the optimal solution remains a weekly rate of 2 doors and 6 windows. Thus, even if availability of capacity of plant 1 uncertain, the company can be fairly confident that it is still optimal to produce a weekly rate of 2 doors and 6 windows. It allows us to solve very large LP’s – column generation approaches

96. Reduced Costs

97. Glass Example x 1 = # of cases of 6-oz juice glasses (in 100s) x 2 = # of cases of 10-oz cocktail glasses (in 100s) x 3 = # of cases of champagne glasses (in 100s) max 5 x 1 + 4.5 x 2 + 6 x 3 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 3  60 (prod. cap. in hrs) 10 x 1 + 20 x 2 + 10 x 3  150 (wareh. cap. in ft 2 ) x 1  8 (6-0z. glass demand) x 1  0, x 2  0, x 3  0 (from AMP and slides from James Orlin)

98. Z* = 51.4286 Decision Variables x 1 = 6.4286 (# of cases of 6-oz juice glasses (in 100s)) x 2 = 4.2857 (# of cases of 10-oz cocktail glasses (in 100s)) x 3 = 0 (# of cases of champagne glasses (in 100s)) Slack Variables s 1 * = 0 s 2 * = 0 s 3 * = 1.5714 Dual Variables y 1 * = 11/4 = 0.7857 y 2 * = 0.0286 y 3 * = 0 Complementary optimal slackness conditions

99. Do the non-negativity constraints also have shadow prices? Yes. They are very special and are called reduced costs? Look at the reduced costs for –Juice glasses reduced cost = 0 –Cocktail glasses reduced cost = 0 –Champagne glasses red. cost = -4/7

100. What is the managerial interpretation of a reduced cost? There are two interpretations. Here is one of them. We are currently not producing champagne glasses. How much would the profit of champagne glasses need to go up for us to produce champagne glasses in an optimal solution? The reduced cost for champagne classes is –4/7. If we increase the revenue for these glasses by 4/7 (from 6 to 6 4/7), then there will be an alternative optimum in which champagne glasses are produced.

101. Why are they called the reduced costs? Nothing appears to be “reduced” The reduced costs can be obtained by treating the shadow prices are real costs. This operation is called “pricing out.”

102. Pricing Out max 5 x 1 + 4.5 x 2 + 6 x 3 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 3  60 10 x 1 + 20 x 2 + 10 x 3  150 1 x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Pricing out treats shadow prices as though they are real prices. The result is the “reduced costs.”

103. Pricing Out of x 1 max 5 x 1 + 4.5 x 2 + 6 x 3 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 3  60 10 x 1 + 20 x 2 + 10 x 3  150 1 x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 1 = 5 - 6 x 11/14 - 10 x 1/35 - 1 x 0 = 5 – 33/7 – 2/7 = 0

104. Pricing Out of x 2 max 5 x 1 + 4.5 x 2 + 6 x 3 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 3  60 10 x 1 + 20 x 2 + 10 x 3  150 1 x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 2 = 4.5 - 5 x 11/14 - 20 x 1/35 - 0 x 0 = 4.5 – 55/14 – 4/7 = 0

105. Pricing Out of x 3 max 5 x 1 + 4.5 x 2 + 6 x 3 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 3  60 10 x 1 + 20 x 2 + 10 x 3  150 1 x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 3 = 6 - 8 x 11/14 - 10 x 1/35 - 0 x 0 = 6 – 44/7 – 2/7 = -4/7

106. Can we use pricing out to figure out whether a new type of glass should be produced? max 5 x 1 + 4.5 x 2 + 7 x 4 ($100s) s.t 6 x 1 + 5 x 2 + 8 x 4  60 10 x 1 + 20 x 2 + 20 x 4  150 1 x 1  8 x 1  0, x 2  0, x 4  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 4 = 7 - 8 x 11/14 - 20 x 1/35 - 0 x 0 = 7 – 44/7 – 4/7 = 1/7

107. Pricing Out of x j max 5 x 1 + 4.5 x 2 + c j x j ($100s) s.t 6 x 1 + 5 x 2 + a 1j x j  60 10 x 1 + 20 x 2 + a 2j x j  150 ……….. ………. + a mj x j = b m x 1  0, x 2  0, x 3  0 shadow price ……y 1 ……y 2 ……… ……y m Reduced cost of x j = ?

108. Brief summary on reduced costs The reduced cost of a non-basic variable x j is the “increase” in the objective value of requiring that x j >= 1. The reduced cost of a basic variable is 0. The reduced cost can be computed by treating shadow prices as real prices. This operation is known as “pricing out.” Pricing out can determine if a new variable would be of value (and would enter the basis).

109. Summary The shadow price is the unit change in the optimal objective value per unit change in the RHS. The shadow price for a “  0” constraint is called the reduced cost. Shadow prices usually but not always have economic interpretations that are managerially useful. Non-binding constraints have a shadow price of 0. The sign of a shadow price can often be determined by using the economic interpretation Shadow prices are valid in an interval. Reduced costs can be determined by pricing out

110. Implications of Reduced Costs We can compute the reduced cost of any variable if we know the original column and if we know the “prices” for each constraint. FACT: We can compute the reduced cost of a new variable. If the reduced cost is positive, it should be entered into the basis.

111. Sensitivity Analysis Computer Analysis

112. The Computer and Sensitivity Analysis If an LP has more than two decision variables, the range of values for a rhs (or objective function coefficient) for which the basis remains optimal cannot be determined graphically. These ranges can be computed by hand but this is often tedious, so they are usually determined by a packaged computer program. MPL and LINDO will be used and the interpretation of its sensitivity analysis discussed. Note: sometimes Excel provides erroneous results

113. MPL – Sensitivity analysis info c1 b2 Dual variables Reduced cost is the amount the objective function coefficient for variable i would have to be increased for there to be an alternative optimal solution. More later… Dual or Shadow prices are the amount the optimal z-value improves if the rhs of a constraint is increased by one unit (assuming no change in basis).

114. MPL – Sensitivity analysis info c1c1 b2b2 What about c 2 ? And b 1 and b 3 ? Allowable ranges (w/o changing basis) for the x 1 coefficient (c 1 ) is: 0  c 1  7.5 Allowable range (w/o changing basis) for the rhs (b 2 ) of the second constraint is: 6  b 2  18

115. Lindo Sensitivity Analysis Allowable ranges – in terms of increase and decrease (w/o changing basis) for the x 1 coefficient (c 1 ) is: 0  c 1  7.5

116. The Computer and Sensitivity Analysis Consider the following maximization problem. Winco sells four types of products. The resources needed to produce one unit of each are: Product 1Product 2Product 3Product 4Available Raw material23474600 Hours of labor34565000 Sales price$4$6$7$8 To meet customer demand, exactly 950 total units must be produced. Customers demand that at least 400 units of product 4 be produced. Formulate an LP to maximize profit. Let x i = number of units of product i produced by Winco.

117. The Winco LP formulation: max z = 4x 1 + 6x 2 +7x 3 + 8x 4 s.t. x 1 + x 2 + x 3 + x 4 = 950 x 4 ≥ 400 2x 1 + 3x 2 + 4x 3 + 7x 4 ≤ 4600 3x 1 + 4x 2 + 5x 3 + 6x 4 ≤ 5000 x 1,x 2,x 3,x 4 ≥ 0

118. MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= 4600 5) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) 6650.000 VARIABLE VALUE REDUCED COST X1 0.000000 1.000000 X2 400.000000 0.000000 X3 150.000000 0.000000 X4 400.000000 0.000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 3.000000 3) 0.000000 -2.000000 4) 0.000000 1.000000 5) 250.000000 0.000000 NO. ITERATIONS= 4 LINDO output and sensitivity analysis example(s). Reduced cost is the amount the objective function coefficient for variable i would have to be increased for there to be an alternative optimal solution.

119. RANGES IN WHICH THE BASIS IS UNCHANGED: OBJ COEFFICIENT RANGES VARIABLE CURRENT ALLOWABLE ALLOWABLE COEF INCREASE DECREASE X1 4.000000 1.000000 INFINITY X2 6.000000 0.666667 0.500000 X3 7.000000 1.000000 0.500000 X4 8.000000 2.000000 INFINITY RIGHTHAND SIDE RANGES ROW CURRENT ALLOWABLE ALLOWABLE RHS INCREASE DECREASE 2 950.000000 50.000000 100.000000 3 400.000000 37.500000 125.000000 4 4600.000000 250.000000 150.000000 5 5000.000000 INFINITY 250.000000 LINDO sensitivity analysis example(s). Allowable range (w/o changing basis) for the x 2 coefficient (c 2 ) is: 5.50  c 2  6.667 Allowable range (w/o changing basis) for the rhs (b 1 ) of the first constraint is: 850  b 1  1000

120. Shadow prices are shown in the Dual Prices section of LINDO output. MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= 4600 5) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) 6650.000 VARIABLE VALUE REDUCED COST X1 0.000000 1.000000 X2 400.000000 0.000000 X3 150.000000 0.000000 X4 400.000000 0.000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 3.000000 3) 0.000000 -2.000000 4) 0.000000 1.000000 5) 250.000000 0.000000 NO. ITERATIONS= 4 Shadow prices are the amount the optimal z- value improves if the rhs of a constraint is increased by one unit (assuming no change in basis).

121. Shadow price signs (MAXIMIZATION) 1.Constraints with  symbols will always have nonpositive shadow prices. 2.Constraints with  will always have nonnegative shadow prices. 3.Equality constraints may have a positive, a negative, or a zero shadow price.

122. Interpretation of shadow prices for the Winco LP ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 3.000000 (overall demand) 3) 0.000000 -2.000000 (product 4 demand) 4) 0.000000 1.000000 (raw material availability) 5) 250.000000 0.000000 (labor availability) Assuming the allowable range of the rhs is not violated, shadow (Dual) prices show: $3 for constraint 1 implies that each one-unit increase in total demand will increase net sales by $3. The -$2 for constraint 2 implies that each unit increase in the requirement for product 4 will decrease revenue by $2. The $1 shadow price for constraint 3 implies an additional unit of raw material (at no cost) increases total revenue by $1. Finally, constraint 4 implies any additional labor (at no cost) will not improve total revenue.

123. Shadow price signs 1.Constraints with  symbols will always have nonpositive shadow prices. 2.Constraints with  will always have nonnegative shadow prices. 3.Equality constraints may have a positive, a negative, or a zero shadow price.

124. Managerial Use of Shadow Prices MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= 4600 5) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) 6650.000 VARIABLE VALUE REDUCED COST X1 0.000000 1.000000 X2 400.000000 0.000000 X3 150.000000 0.000000 X4 400.000000 0.000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 3.000000 3) 0.000000 -2.000000 4) 0.000000 1.000000 5) 250.000000 0.000000 NO. ITERATIONS= 4 The managerial significance of shadow prices is that they can often be used to determine the maximum amount a manager should be willing to pay for an additional unit of a resource. Reconsider the Winco to the right. What is the most Winco should be willing to pay for additional units of raw material or labor? raw material labor

125. Managerial Use of Shadow Prices MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= 4600 5) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) 6650.000 VARIABLE VALUE REDUCED COST X1 0.000000 1.000000 X2 400.000000 0.000000 X3 150.000000 0.000000 X4 400.000000 0.000000 ROW SLACK OR SURPLUS DUAL PRICES 2) 0.000000 3.000000 3) 0.000000 -2.000000 4) 0.000000 1.000000 5) 250.000000 0.000000 NO. ITERATIONS= 4 The shadow price for raw material constraint (row 4) shows an extra unit of raw material would increase revenue $1. Winco could pay up to $1 for an extra unit of raw material and be as well off as it is now. Labor constraint’s (row 5) shadow price is 0 meaning that an extra hour of labor will not increase revenue. So, Winco should not be willing to pay anything for an extra hour of labor.