Explorations in Artificial Intelligence Prof. Carla P. Gomes Module 7 Part 2 Solving Linear Programming

Solving LP

LP: Algorithms Simplex. (Dantzig 1947) –Developed shortly after WWII in response to logistical problems: used for 1948 Berlin airlift. –Practical solution method that moves from one extreme point to a neighboring extreme point. –Finite (exponential) complexity, but no polynomial implementation known.

LP: Polynomial Algorithms Ellipsoid. (Khachian 1979, 1980) –Solvable in polynomial time: O(n 4 L) bit operations. n = # variables L = # bits in input –Theoretical tour de force. –Not remotely practical. Karmarkar's algorithm. (Karmarkar 1984) –O(n 3.5 L). –Polynomial and reasonably efficient implementations possible. Interior point algorithms. –O(n 3 L). –Competitive with simplex! will likely dominate on large problems soon –Extends to even more general problems.

Simplex Method

The simplex method has exponential time complexity (c n ) (worst case complexity). However, in practice it works well in general. Key advantage of simplex methods – capability of performing postoptimality / sensitivity analysis (duality); Simplex Method in Practice

Key Notion in Simplex: Corner Point Solutions Corner-point feasible solution – special solution that plays a key role when the simplex method searches for an optimal solution. Relationship between optimal solutions and CPF solutions: –Any LP with feasible solutions and bounded feasible region  (1) the problem must possess CPF solutions and at least one optimal solution (2) the best CPF solution must be an optimal solution  f the problem has exactly one optimal solution it must be a CFP solution If the problem has multiple optimal solutions, at least two must be CPF solutions

Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27

Simplex – Key Concepts Concept 1 – CPF solutions –Simplex methods focuses only on CPF solutions (finite set) Concept 2 – Flow of simplex method –Iterative procedure: –Initialization – find initial CPF solution –Optimality test if optimal stop if not optimal go to next iteration –Iteration – find a better CPF solution; go to 2. Concept 3– Initialization –Whenever possible pick the origin; otherwise special procedure

Simplex – Key Concepts (cont.) Concept 4 – Path to optimal solution –Simplex methods always chooses a CPF solution adjacent to the current one –The entire path to the optimal solution is along the edges of the feasible solution Concept 5 – Choice of new CPF solution –From a CPF solution consider all edges emanating from it but it does not solve for each adjacent CPF solution – it simply identifies the rate of improvement in Z along a given edge and than it picks the one with the largest improvement Concept 6– Optimality Test –From a given CPF solution check if there is an edge that gives a positive rate of improvement in Z. If not, current CPF is optimal.

Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27

Augmented Form for Wyndor Glass Maximize Z= 3 x x 2 subject to x 1 ≤ 4 2x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 and x 1 ≥ 0, x 2 ≥ 0 Maximize Z= 3 x1 + 5 x2 + 0 s s s 3 subject to x 1 + s 1 =4 2x 2 + s 2 =12 3x 1 + 2x 2 + s 3 = 18 and x 1 ≥ 0, x 2 ≥ 0, s 1 ≥ 0, s 2 ≥ 0, s 3 ≥ 0

Simplex as an algebraic procedure System of functional constraints n (5) variables (5) and m (3) equations  2 degrees of freedom, (i.e., we can set those two variables to any arbitrary values); they are the nonbasic variables; the other variables are the basic variables; –Simplex chooses to set the non-basic variables to ZERO. –Simplex solves the simultaneous equations to set the values of the basic variables;

Properties of Basic Solutions A basic solution is composed of: –Non-basic variables – number of non-basic variables equals (total number of variables - number of functional constraints) They are set to ZERO –Basic variables – number of basic variables equals number of functional constraints Their values results from solving the system of functional constraints (non-basic variables set to 0) They form the Basic Feasible solution – it is a basic solution that satisfies the non-negativity constraints Adjacent basic feasible solutions – all but one of their basic (non-basic) variables are the same  moving from one basic feasible solution to an adjacent one involves switching one variable from non-basic to basic and one variable from basic to non-basic (check graph)

Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27 (x1,x2,s1,s2,s3) (0,0,4,12,18) Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. (4,0,0,12,6) (0,6,4,0,6) (2,4,2,0,0) (4,3,0,3,0)

Getting ready for the Simplex Standard form –<= constraints –Non-negativity constraints on all variables –Positive right hand sides (if these assumptions are not valid  additional adjustments need to be done) Transform the objective function and constraints into equalities (introduction of slack variables)

Simplex Procedure Initialization – origin whenever possible (decision variables 0) (okay if standard form with positive RHS’s; basic feasible solution (BFS): each equation has a basic variable with coefficient 1 (slack variable = RHS) and the variable does not appear in any other eq; the decision variables are the non-basic variables set to 0) Optimality test – is current BFS optimal? ( the coefficients of the objective function of the non-basic variables gives the rate of improvement in Z )

Iteration – move to a better adjacent BFS –a)Variable entering the basis –Consider non-basic variables (Graphically - Consider edges emanating from current CPF solution) –Pick the variable that increases Z at a faster rate –b)Variable leaving the basis –One of the basic variables will become non-basic; write all the basic variables as a function of the entering variable; the most stringent value (i.e., smallest) will be the value for the new entering variable; the basic variable associated with the most stringent constraint will become non-basic, leaving the basis. – (Graphically – where to stop? As much as possible without leaving the feasible region) –c)Solving for the new BFS –Objective of this step: convert the system of equations into a more convenient form: (1) to perform optimality test and (2) to perform next iteration if necessary –New basic variable should have coefficient 1 in the equation of the leaving variable and 0 in all the other equations, including the objective function; –Valid operations: Multiplication (or division) of an equation by a non-zero constant Addition (or subtraction) of a multiple of an equation to (from) another eqution Simplex Procedure (cont.)

Wyndor Glass CPF Edge of Feasible region 0 1 Z=0 Z=30 LetD = the number of doors to produce W = the number of windows to produce Maximize P = 3 D + 5 W 2 Z=36 Z=27 (x1,x2,s1,s2,s3) (0,0,4,12,18) Maximize P = 3 D + 5 W subject to D ≤ 4 2W ≤ 12 3D + 2W ≤ 18 and D ≥ 0, W ≥ 0. (4,0,0,12,6) (0,6,4,0,6) (2,4,2,0,0) (4,3,0,3,0)

Wyndor Glass 1 st Simplex Tableau Optimal? Entering Variable?

What variable will enter the basis? Why? What variable will leave the basis? Why? What transformations do we need to perform to the tableau to get the new basic variable into the right format?

What operations did we perform? 1 – divide the pivot row by 2; 2 – multiplied the new pivot row by (-2) and added it to eq. 3 3 – multiplied the new pivot row by (5) and added it to eq. 0 What are the new basic / non-basic variables?

Optimal? Entering Variable?

What operations did we perform? 1 – divide the pivot row by ; 2 – multiplied the new pivot row by ( ) and added it to eq. 1 3 – multiplied the new pivot row by ( ) and added it to eq. 0 What are the new basic / non-basic variables?

Optimal?

Simplex Method in Tabular Form

Modeling Languages

A mathematical language that has been designed to efficiently formulate large mathematical models (with thousand of variables/constraints). Examples of modeling languages: AMPL; MPL; GAMS; LINGO. Textbook (CD-ROM): –MPL (student version; also from It provides an interface to to Excel (importing and exporting Excel ranges) Powerful state-of-the-art LP based solver  CPLEX –LINDO (also available from LINDO (Traditional optimizer) What’s Best – spreadsheet optimizer LINGO (Linear and Non-liner programming) –Formulations in MPL, LINGO, LINDO and What’s best for all the examples in the book.

MPL: Just ignore the message…

MPL/CPLEX Activating the license

Acknowledgment from Cplex

from Cplex From Thu Feb 24 12:21: Date: Thu, 24 Feb :16: (PST) From: To: Subject: License Instructions for MPL and CPLEX Mime-Version: 1.0 Content-Transfer-Encoding: 7bit X-Spam-Checker-Version: SpamAssassin 2.60 ( exp) on sundial.cs.cornell.edu X-Spam-Status: No, hits=-1.4 required=5.0 tests=BAYES_01,NO_REAL_NAME autolearn=no version=2.60 X-Spam-Level: Dear Carla Gomes, Thank you for registering your copy of MPL and CPLEX provided with the book Hillier and Lieberman, Introduction to Operations Research, Seventh Edition. Instructions for activating your license appear below. If you have any questions or require support, please contact Maximal Software at The steps for activating your license are listed below. Please note the special activation password listed in Step 3. etc

MPL

TITLE WyndorGlass; MODEL MAX Z = 3 X1 + 5 X2; SUBJECT TO X1 <= 4; 2 X2 <= 12; 3 X1 + 2 X2 <= 18; END Note: default assumption – non-negativity constraints (can be changed)

MPL: Solution Options

Checking Syntax and Runnig

Syntax

Solving it with Cplex

Optimal Solution

Solution File MPL Modeling System - Copyright (c) , Maximal Software, Inc MODEL STATISTICS Problem name: WyndorGlass Filename: my-wyndor.mpl Date: February 24, 2005 Time: 11:57 Parsing time: 0.05 sec Solver: CPLEX Objective value: Iterations: 0 Solution time: 0.00 sec Constraints: 3 Variables: 2 Nonzeros: 4 Density: 67 % SOLUTION RESULT Optimal solution found MAX Z =

DECISION VARIABLES PLAIN VARIABLES Variable Name Activity Reduced Cost X X CONSTRAINTS PLAIN CONSTRAINTS Constraint Name Slack Shadow Price c c c RANGES OBJECTIVE PLAIN VARIABLES Variable Name Coefficient Lower Range Upper Range X X E RANGES RHS PLAIN CONSTRAINTS Constraint Name RHS Value Lower Bound Upper Bound c E+020 c c END Solution File (cont.)

MPL: A more compact version (good for large-scale problems) TITLE WyndorGlass; INDEX product := (Door, Window); plant := 1..3; DATA TimeAvail[plant] := (4, 12, 18); ProdTime[plant, product] := (1, 0, 0, 2, 3, 2); Profit[product] := (3.00, 5.00); VARIABLES Produce[product] -> Prod; MODEL MAX TotalProfit = SUM(product: Profit * Produce); SUBJECT TO TimeCapacity[plant] -> TimeCap: SUM(product: ProdTime * Produce) <= TimeAvail; END

Excel

Developing a Spreadsheet Model Step #2: Changing Cells –Add a cell in the spreadsheet for every decision variable. –If you don’t have any particular initial values, just enter 0 in each. –It is a good idea to color code these “changing cells” (e.g., yellow with border). Step #1: Data Cells (parameters) –Enter all of the data (parameters) for the problem on the spreadsheet. –Make consistent use of rows and columns. –It is a good idea to color code these “data/parameter cells” (e.g., light blue).

Developing a Spreadsheet Model Step #3: Target Cell –Develop an equation that defines the objective of the model. –Typically this equation involves the data cells and the changing cells in order to determine a quantity of interest (e.g., total profit or total cost). –It is a good idea to color code this cell (e.g., orange with heavy border).

Developing a Spreadsheet Model Step #4: Constraints –For any resource that is restricted, calculate the amount of that resource used in a cell on the spreadsheet (an output cell). –Define the constraint in three consecutive cells. For example, if Quantity A ≤ Quantity B, put these three items (Quantity A, ≤, Quantity B) in consecutive cells.

A Trial Solution The spreadsheet for the Wyndor problem with a trial solution (4 doors and 3 windows) entered into the changing cells.

Identifying the Target Cell and Changing Cells Choose the “Solver” from the Tools menu. Select the cell you wish to optimize in the “Set Target Cell” window. Choose “Max” or “Min” depending on whether you want to maximize or minimize the target cell. Enter all the changing cells in the “By Changing Cells” window.

Adding Constraints To begin entering constraints, click the “Add” button to the right of the constraints window. Fill in the entries in the resulting Add Constraint dialogue box.

The Complete Solver Dialogue Box

Some Important Options Click on the “Options” button, and click in both the “Assume Linear Model” and the “Assume Non-Negative” box. –“Assume Linear Model” tells the Solver that this is a linear programming model. –“Assume Non-Negative” adds nonnegativity constraints to all the changing cells.

The Solver Results Dialogue Box

The Optimal Solution

Requesting the Sensitivity Analysis Report

Sensitivity Analysis Report

Duality

Every maximization LP problem in the standard form gives rise to a minimization LP problem called the dual problem Every feasible solution in one yields a bound on the optimal value of the other If one of the problems has an optimal solution, so does the other and the two optimal values coincide These results have very interesting economic interpretations

Bounds on the optimal value Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. Lower bound (LB) on Z*  Z* ≥ LB any feasible solution Upper bound (UB) on Z*  Z* ≤ UB – how do we come up with upper bounds for Z* for a maximization LP problem?

“Guessing” upper bounds Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1≤ 4 2x2≤ 12 3x1 + 2x2≤ 18 and x1 ≥ 0, x2 ≥ 0. 1 – multiplying the 3 rd costraint by 3  9x1 + 6x2≤ 54 Z= 3 x1 + 5 x2 ≤ 9x1 + 6x2≤ 54  Z*≤ 54 2 – multiplying the 3 rd constraint by 2.5  7.5 x1 + 5 x2≤ 45 Z= 3 x1 + 5 x2 ≤ 7.5 x1 + 6x2≤ 45  Z*≤ 45 3 – multiplying the 2 nd constraint by 2 and add it to the 3 rd  3x1 + 6x2≤ 42 Z= 3 x1 + 5 x2 ≤ 3x1 + 6x2≤ 54  Z*≤ 42 4 – multiplying the 2 nd constraint by 1.5 and add it to the 3 rd  3x1 + 5x2≤ 36 Z= 3 x1 + 5 x2 ≤ 3x1 + 5x2≤ 54  Z*≤ 36

A principled way of finding upper-bounds on Z* Dual Problem Letx1 = the number of doors to produce x2= the number of windows to produce Maximize Z= 3 x1 + 5 x2 subject to x1 ≤ 4  y1 2x2≤ 12  y2 3x1 + 2x2≤ 18  y3 and x1 ≥ 0, x2 ≥ 0. We construct linear combinations of constraints Minimize W= 4 y y y 3 subject to y y 3 ≥ 3 2 y y 3 ≥ 5 and y 1 ≥ 0, y 2 ≥ 0, y 3 ≥ 0 Each multiplier must be non-negative We want to minimize the biund The sum of the coefficients at least equal the corresponding one for objective function

Primal vs. Dual Problem Maximize Z= 3 x x 2 subject to x 1 ≤ 4 2x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 and x 1 ≥ 0, x 2 ≥ 0. Minimize W= 4 y y y 3 subject to y y 3 ≥ 3 2 y y 3 ≥ 5 and y 1 ≥ 0, y 2 ≥ 0, y 3 ≥ 0

Primal vs. Dual Max Z = c 1 x 1 + c 2 x 2 + …. + c n x n Subject to: a 11 x 1 + a 12 x 2 + … + a 1n x n ≤ b 1 a 21 x 1 + a 22 x 2 + … + a 2n x n ≤ b 2 … a m1 x 1 + a m2 x 2 + … + a mn x n ≤ b m x 1, x 2, …, x n ≥ 0 Min W = b 1 y 1 + b 2 y 2 + …. + b m y m Subject to: a 11 y 1 + a 21 y 2 + … + a m1 y m ≥ c 1 a 12 y 1 + a 22 y 2 + … + a m2 y m ≥ c 2 … a 1n y 1 + a 2n y 2 + … + a mn y n ≥ c n y 1, y 2, …, y m ≥ 0

Max s.t Min Primal Dual

Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Z*Z* Primal feasible solutions (Maximization) Lower Bounds on Dual Optimum Dual feasible solutions (Minimization) Upper Bounds on Primal Optimum Weak duality property

Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual

Strong Duality Property

Every feasible dual solution yields an upper-bound on Z* of the primal and every feasible primal solution yields a lower-bound on Z* of the dual Z*Z* Primal feasible solutions (Maximization) Lower Bounds on Dual Optimum Dual feasible solutions (Minimization) Upper Bounds on Primal Optimum Weak duality property Strong duality property Primal Opt. = Dual Opt.

Complementary Slackness Optimal solutions Let x 1 *, x 2 *, …x n * be a primal feasible solution and a y 1 *, y 2 *, …y m * dual feasible solution. Necessary and sufficient conditions for simultaneous optimality of x 1 *, x 2 *, …x n * and y 1 *, y 2 *, …y m * are: This result is very useful for checking the optimality of solutions

Relationships between the primal and dual problems If one problem has feasible solutions and a bounded objective function (and therefore an optimal solution), then so does the other problem (both weak and strong duality apply) If one problem has feasible solutions and an unbounded objective function (therefore no optimal solution) then the other problem has no feasible solutions If one problem has no feasible solutions than the other problem has no feasible solutions or an unbounded objective function.

Complementary Slackness Optimal solutions

Primal-Dual Combinations

Adapting to Other Forms What if our problem is not in standard form? –We can always transform it to the standard form and then construct the dual;

Dual of the dual Note: it is not important which problem we call dual and which problem we call primal given the symmetry property of the primal dual relationships. In general we call primal the model formulated to fit the actual problem.

Complexity of Simplex Method Primal vs. Dual How long does it take to solve an LP using the simplex method? –Several factors but the most important one seems to be the number of functional constraints. Computation tends to be proportional to the cube of the number of functional constraints in an LP. –The number of variables is a relatively minor factor (assuming revised simplex method) –The density of the matrix of technological coefficients is also a factor – the sparser the matrix (i.e., the larger the number of zeroes) the faster the simplex method; – Real world problems tend to be sparse, i.e., “sparcity” of 5% or even 1%, which leads to fast runs.

Primal vs. Dual? So, the size of the problem, may determine whether to use the simplex method on the primal or dual problem. If the primal has a large number of constraints and a small number of variables it is better to apply the simplex method to the dual (since it will have a small number of constraints).

Dual Simplex Method This method is based on the duality results. It is a mirror image of the simplex method: –the simplex method deals with primal feasible solutions (but not dual feasible), moving toward a solution that is dual feasible; – the dual method deals with basic solutions in the primal problem that are dual feasible but not primal feasible. It moves toward an optimal solution by striving to achieve primal feasibility as well.

Post-optimality Analysis and Economic Interpretation of Duality

Post-optimality Analysis Post-optimality – very important phase of modeling. Duality plays and important role in post-optimality analysis Simplex provides several tools to perform post-optimality analysis

Economic Interpretation of Duality LP problems – quite often can be interpreted as allocating resources to activities. Let’s consider the standard form: x i >= 0, (i =1,2,…,n)

What if we change our resources – can we improve our optimal solution? Resources – m (plants) Activities – n (2 products) Wyndor Glass problem  optimal product mix --- allocation of resources to activities i.e., choose the levels of the activities that achieve best overall measure of performance

Sensitivity Analysis How would changes in the problem ’ s objective function coefficients or right-hand side values change the optimal solution?

Dual Variables (Shadow Prices) y 1 *= 0  dual variable (shadow price) for resource 1 y 2 *= 1.5  dual variable (shadow price) for resource 2 y 3 *= 1  dual variable (shadow price) for resource 3 How much does Z increase if we increase resource 2 by 1 unit (i.e., b 2 = 12  b2=13)?

Graphical Analysis of Dual variables – Variation in RHS Increasing level of resource 2 (b 2 ) (2,6) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 (5/3,13/2) ∆ Z=1.5 = y 2 * Why is y 1 *=0?

Economic Interpretation of Dual Variables The dual variable associated with resource i (also called shadow price), denoted by y i *, measures the marginal value of this resource, i.e., the rate at which Z could be increased by (slightly) increasing the amount of this resource (bi), assuming everything else stays the same. The dual variable y i * is identified by the simplex method as the coefficient of the i th slack variable in row 0 of the final simplex tableau.

Dual Variables: binding and non-binding constraints The shadow prices (dual variables) associated with non-binding constraints are necessarily 0 (complementary optimal slackness)  there is a surplus of non- binding resource and therefore increasing it will not increase the optimal solution. Economist refer to such resources as free resources (shadow price =0) Binding constraints on the other hand correspond to scarce resources – there is no surplus. In general they have a positive shadow price.

Does Z always increase at the same rate if we keep increasing the amount of resource 2? (2,6) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 (5/3,13/2) ∆ Z=1.5 = y 2 * What if b 2 > 18 (i.e., 2W>18)?  the optimal solution will stay at (0,9) for b 2 >=18 b 2 =18 (0,9)

If b 2 < 6 the solution will no longer vary proportionally. The optimal solution varies proportionally to the variation in b 2 only if 6 <= b 2 <=18. In other words, the current basis remains optimal for 6 ≤ b 2 ≤ 18, but the decision variable values and z-value will change. Does Z always decrease at the same rate if we decrease resource 2? (2,6) (5/3,13/2) Z=3(2)+5(6)=36 2w=13  Z=3(5/3)+5(13/2)=37.5 ∆ Z=1.5 = y 2 * b 2 =6

A dual variable y i * gives us the rate at which Z could be increased by increasing the amount of resource i slightly. However this is only true for a small increase in the amount of the resource. I.e., this definition applies only if the change in the RHS of constraint i leaves the current basis optimal. It also assumes everything else stays the same. Another interpretation of y i * is: if a premium price must be paid for the resource i in the market place, y i * is the maximum premium (excess over the regular price) that would be worth paying.

Optimal Basis in the Wyndor Glass Problem How can we characterize (verbally) the optimal basis of the Wyndor Glass problem? –Plant 1 – unutilized capacity (non-binding constraint) –Plant 2 – fully utilized capacity (binding constraint) –Plant 3 - fully utilized capacity (binding constraint)

How do we interpret the intervals? If we change one coefficient in the RHS, say capacity of plant 2, by  the “basis” remains optimal, that is, the same equations remain binding. So long as the basis remains optimal, the shadow prices are unchanged. The basic feasible solution varies linearly with . If  is big enough or small enough the basis will change.

Sensitivity analysis for c1 How much can we vary c1 without changing the current basic optimal solution?

Sensitivity analysis for c1 Our objective function is: Z= c 1 D+5W=k  slope of iso-profit line is: isoprofit line How much can c1 vary until the slope of the iso-profit line equals the slope of constraint 2 and constraint 3?

How much can c1 vary until the slope of the iso-profit line equals the slope of constraint 2 and constraint 3? Slope of constraint 2  0 Slope of constraint 3  -3/2

Importance of Sensitivity Analysis Sensitivity analysis is important for several reasons: Values of LP parameters might change. If a parameter changes, sensitivity analysis shows it is unnecessary to solve the problem again. For example in the Wyndor problem, if the profit contribution of product 1 changes to 5, sensitivity analysis shows the current solution remains optimal. Uncertainty about LP parameters. In the Wyndor problem for example, if the capacity of plant 1 decreases to 2, the optimal solution remains a weekly rate of 2 doors and 6 windows. Thus, even if availability of capacity of plant 1 uncertain, the company can be fairly confident that it is still optimal to produce a weekly rate of 2 doors and 6 windows. It allows us to solve very large LP’s – column generation approaches

Reduced Costs

Glass Example x 1 = # of cases of 6-oz juice glasses (in 100s) x 2 = # of cases of 10-oz cocktail glasses (in 100s) x 3 = # of cases of champagne glasses (in 100s) max 5 x x x 3 ($100s) s.t 6 x x x 3  60 (prod. cap. in hrs) 10 x x x 3  150 (wareh. cap. in ft 2 ) x 1  8 (6-0z. glass demand) x 1  0, x 2  0, x 3  0 (from AMP and slides from James Orlin)

Z* = Decision Variables x 1 = (# of cases of 6-oz juice glasses (in 100s)) x 2 = (# of cases of 10-oz cocktail glasses (in 100s)) x 3 = 0 (# of cases of champagne glasses (in 100s)) Slack Variables s 1 * = 0 s 2 * = 0 s 3 * = Dual Variables y 1 * = 11/4 = y 2 * = y 3 * = 0 Complementary optimal slackness conditions

Do the non-negativity constraints also have shadow prices? Yes. They are very special and are called reduced costs? Look at the reduced costs for –Juice glasses reduced cost = 0 –Cocktail glasses reduced cost = 0 –Champagne glasses red. cost = -4/7

What is the managerial interpretation of a reduced cost? There are two interpretations. Here is one of them. We are currently not producing champagne glasses. How much would the profit of champagne glasses need to go up for us to produce champagne glasses in an optimal solution? The reduced cost for champagne classes is –4/7. If we increase the revenue for these glasses by 4/7 (from 6 to 6 4/7), then there will be an alternative optimum in which champagne glasses are produced.

Why are they called the reduced costs? Nothing appears to be “reduced” The reduced costs can be obtained by treating the shadow prices are real costs. This operation is called “pricing out.”

Pricing Out max 5 x x x 3 ($100s) s.t 6 x x x 3  x x x 3  x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Pricing out treats shadow prices as though they are real prices. The result is the “reduced costs.”

Pricing Out of x 1 max 5 x x x 3 ($100s) s.t 6 x x x 3  x x x 3  x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 1 = x 11/ x 1/ x 0 = 5 – 33/7 – 2/7 = 0

Pricing Out of x 2 max 5 x x x 3 ($100s) s.t 6 x x x 3  x x x 3  x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 2 = x 11/ x 1/ x 0 = 4.5 – 55/14 – 4/7 = 0

Pricing Out of x 3 max 5 x x x 3 ($100s) s.t 6 x x x 3  x x x 3  x 1  8 x 1  0, x 2  0, x 3  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 3 = x 11/ x 1/ x 0 = 6 – 44/7 – 2/7 = -4/7

Can we use pricing out to figure out whether a new type of glass should be produced? max 5 x x x 4 ($100s) s.t 6 x x x 4  x x x 4  x 1  8 x 1  0, x 2  0, x 4  0 shadow price ……11/14 ……1/35 …….0 Reduced cost of x 4 = x 11/ x 1/ x 0 = 7 – 44/7 – 4/7 = 1/7

Pricing Out of x j max 5 x x 2 + c j x j ($100s) s.t 6 x x 2 + a 1j x j  x x 2 + a 2j x j  150 ……….. ………. + a mj x j = b m x 1  0, x 2  0, x 3  0 shadow price ……y 1 ……y 2 ……… ……y m Reduced cost of x j = ?

Brief summary on reduced costs The reduced cost of a non-basic variable x j is the “increase” in the objective value of requiring that x j >= 1. The reduced cost of a basic variable is 0. The reduced cost can be computed by treating shadow prices as real prices. This operation is known as “pricing out.” Pricing out can determine if a new variable would be of value (and would enter the basis).

Summary The shadow price is the unit change in the optimal objective value per unit change in the RHS. The shadow price for a “  0” constraint is called the reduced cost. Shadow prices usually but not always have economic interpretations that are managerially useful. Non-binding constraints have a shadow price of 0. The sign of a shadow price can often be determined by using the economic interpretation Shadow prices are valid in an interval. Reduced costs can be determined by pricing out

Implications of Reduced Costs We can compute the reduced cost of any variable if we know the original column and if we know the “prices” for each constraint. FACT: We can compute the reduced cost of a new variable. If the reduced cost is positive, it should be entered into the basis.

Sensitivity Analysis Computer Analysis

The Computer and Sensitivity Analysis If an LP has more than two decision variables, the range of values for a rhs (or objective function coefficient) for which the basis remains optimal cannot be determined graphically. These ranges can be computed by hand but this is often tedious, so they are usually determined by a packaged computer program. MPL and LINDO will be used and the interpretation of its sensitivity analysis discussed. Note: sometimes Excel provides erroneous results

MPL – Sensitivity analysis info c1 b2 Dual variables Reduced cost is the amount the objective function coefficient for variable i would have to be increased for there to be an alternative optimal solution. More later… Dual or Shadow prices are the amount the optimal z-value improves if the rhs of a constraint is increased by one unit (assuming no change in basis).

MPL – Sensitivity analysis info c1c1 b2b2 What about c 2 ? And b 1 and b 3 ? Allowable ranges (w/o changing basis) for the x 1 coefficient (c 1 ) is: 0  c 1  7.5 Allowable range (w/o changing basis) for the rhs (b 2 ) of the second constraint is: 6  b 2  18

Lindo Sensitivity Analysis Allowable ranges – in terms of increase and decrease (w/o changing basis) for the x 1 coefficient (c 1 ) is: 0  c 1  7.5

The Computer and Sensitivity Analysis Consider the following maximization problem. Winco sells four types of products. The resources needed to produce one unit of each are: Product 1Product 2Product 3Product 4Available Raw material Hours of labor Sales price$4$6$7$8 To meet customer demand, exactly 950 total units must be produced. Customers demand that at least 400 units of product 4 be produced. Formulate an LP to maximize profit. Let x i = number of units of product i produced by Winco.

The Winco LP formulation: max z = 4x 1 + 6x 2 +7x 3 + 8x 4 s.t. x 1 + x 2 + x 3 + x 4 = 950 x 4 ≥ 400 2x 1 + 3x 2 + 4x 3 + 7x 4 ≤ x 1 + 4x 2 + 5x 3 + 6x 4 ≤ 5000 x 1,x 2,x 3,x 4 ≥ 0

MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= ) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X ROW SLACK OR SURPLUS DUAL PRICES 2) ) ) ) NO. ITERATIONS= 4 LINDO output and sensitivity analysis example(s). Reduced cost is the amount the objective function coefficient for variable i would have to be increased for there to be an alternative optimal solution.

RANGES IN WHICH THE BASIS IS UNCHANGED: OBJ COEFFICIENT RANGES VARIABLE CURRENT ALLOWABLE ALLOWABLE COEF INCREASE DECREASE X INFINITY X X X INFINITY RIGHTHAND SIDE RANGES ROW CURRENT ALLOWABLE ALLOWABLE RHS INCREASE DECREASE INFINITY LINDO sensitivity analysis example(s). Allowable range (w/o changing basis) for the x 2 coefficient (c 2 ) is: 5.50  c 2  Allowable range (w/o changing basis) for the rhs (b 1 ) of the first constraint is: 850  b 1  1000

Shadow prices are shown in the Dual Prices section of LINDO output. MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= ) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X ROW SLACK OR SURPLUS DUAL PRICES 2) ) ) ) NO. ITERATIONS= 4 Shadow prices are the amount the optimal z- value improves if the rhs of a constraint is increased by one unit (assuming no change in basis).

Shadow price signs (MAXIMIZATION) 1.Constraints with  symbols will always have nonpositive shadow prices. 2.Constraints with  will always have nonnegative shadow prices. 3.Equality constraints may have a positive, a negative, or a zero shadow price.

Interpretation of shadow prices for the Winco LP ROW SLACK OR SURPLUS DUAL PRICES 2) (overall demand) 3) (product 4 demand) 4) (raw material availability) 5) (labor availability) Assuming the allowable range of the rhs is not violated, shadow (Dual) prices show: $3 for constraint 1 implies that each one-unit increase in total demand will increase net sales by $3. The -$2 for constraint 2 implies that each unit increase in the requirement for product 4 will decrease revenue by $2. The $1 shadow price for constraint 3 implies an additional unit of raw material (at no cost) increases total revenue by $1. Finally, constraint 4 implies any additional labor (at no cost) will not improve total revenue.

Shadow price signs 1.Constraints with  symbols will always have nonpositive shadow prices. 2.Constraints with  will always have nonnegative shadow prices. 3.Equality constraints may have a positive, a negative, or a zero shadow price.

Managerial Use of Shadow Prices MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= ) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X ROW SLACK OR SURPLUS DUAL PRICES 2) ) ) ) NO. ITERATIONS= 4 The managerial significance of shadow prices is that they can often be used to determine the maximum amount a manager should be willing to pay for an additional unit of a resource. Reconsider the Winco to the right. What is the most Winco should be willing to pay for additional units of raw material or labor? raw material labor

Managerial Use of Shadow Prices MAX 4 X1 + 6 X2 + 7 X3 + 8 X4 SUBJECT TO 2) X1 + X2 + X3 + X4 = 950 3) X4 >= 400 4) 2 X1 + 3 X2 + 4 X3 + 7 X4 <= ) 3 X1 + 4 X2 + 5 X3 + 6 X4 <= 5000 END LP OPTIMUM FOUND AT STEP 4 OBJECTIVE FUNCTION VALUE 1) VARIABLE VALUE REDUCED COST X X X X ROW SLACK OR SURPLUS DUAL PRICES 2) ) ) ) NO. ITERATIONS= 4 The shadow price for raw material constraint (row 4) shows an extra unit of raw material would increase revenue $1. Winco could pay up to $1 for an extra unit of raw material and be as well off as it is now. Labor constraint’s (row 5) shadow price is 0 meaning that an extra hour of labor will not increase revenue. So, Winco should not be willing to pay anything for an extra hour of labor.