Simplex Method Continued … from a presentation at the Fuqua School of Business MIT and James Orlin © 2003 Rev. 2.71 by M. Miccio on December 15, 2014 MIT and James Orlin © 2003 file Simplex2_AMII_05b_gr 1

Today’s Lecture Review of the simplex algorithm Alternative Optimal Solutions Unbounded Solution Degeneracy Is the simplex algorithm finite? (Answer, yes, but only if we are careful) MIT and James Orlin © 2003 2

Today’s Lecture Review of the simplex algorithm Alternative Optimal Solutions Unbounded Solution Degeneracy Is the simplex algorithm finite? (Answer, yes, but only if we are careful) MIT and James Orlin © 2003 3

Review of the Notation n number of variables m number of constraints c index of entering variable r index of pivot row note: the rth basic variable leaves the basis The original data is cj, aij, bi After performing pivots we represent the revised coefficients as cj, aij, bi The bar − indicates that it is the coefficient after some pivots MIT and James Orlin © 2003 4

m constraints, n variables Tableau Basic Variables Non-basic Variables -z x1 x2 xr xm xm+1 xs xn CV 1 cm+1 cs cn -z0 = 1 a1,m+1 a1,s a1,n b1 = 1 a2,m+1 a2,s a2,n b2 . . . . . = 1 ar,m+1 ar,s ar,n br = . . . . . 1 am,m+1 am,s am,n bm = m constraints, n variables MIT and James Orlin © 2003 5

The basic feasible solution The current values are all non-negative. This is needed for canonical form There is a basic variable associated with each constraint. in this Tableau, the basic variable associated with constraint i is xi. There are (n-m) nonbasic variables. In this Tableau, the nonbasic variables are xm+1, … xn. This bfs is as follows: x1 =b1, … , xm =bm. All other variables are 0. MIT and James Orlin © 2003 6

Today’s Lecture Review of the simplex algorithm. Alternative Optimal Solutions Unbounded Solution Degeneracy Degenerate Solutions Proving finiteness and optimality under no degeneracy MIT and James Orlin © 2003 7

Alternative Optima (maximization) Initial Tableau z = 0x1 -4x2 + 8 max! How much can x1 be increased? -z x1 x2 x3 x4 1 -4 -8 = 1 3 1 6 = -1 2 1 2 This basic feasible solution is optimal! Is there another optimal solution? Suppose that x1 entered the basis. What would leave? MIT and James Orlin © 2003 8

Alternative Optima (maximization) Initial Tableau 1 c1 c2 -4 -8 = 1 3 1 6 = -1 2 1 2 Use the min ratio rule to select the pivot row and to determine which variable leaves the basis. MIT and James Orlin © 2003 9

Alternative Optima (maximization) Tableau No.1 -4 -4 -8 -8 = 1 1 3 3 1 1 6 6 = -1 2 5 1 1 1 8 2 Let x1 enter the basis. Let the basic variable in constraint 1 leave the basis. Note: the solution is different, but the objective function value is the same. MIT and James Orlin © 2003 10

Test for infinite solutions Alternative Optima Test for infinite solutions There are alternative optima ifcj = 0 for some j where xj is non-basic MIT and James Orlin © 2003 11

Today’s Lecture Review of the simplex algorithm. Alternative Optimal Solutions Unbounded Solution Degeneracy Degenerate Solutions Proving finiteness and optimality under no degeneracy MIT and James Orlin © 2003 12

Unboundedness z = 1x1+ 0x2 + 2 max! -z x1 x2 x3 x4 1 -2 1 -3 1 -1 1 -1 1 -2 3 = x1 = 0 x2 = 1 x3 = 3 x4 = 0 z = 2 x1 = D x2 = 1 − (-2)D x3 = 3 − (-3)D x4 = 0 z = 2 + D -3 1 -1.5 = -2 1 .5 Verif. Cosa succede per a21 = +2 The entry variable is x1. Set x1 = D>0 while x4 = 0. D can be as large as we like ! If all the coefficients in the entering column are  0, then the solution is unbounded from above MIT and James Orlin © 2003 13

Unboundedness Test for Unbounded Solution If aic  0 for all i, then the solution is unbounded. MIT and James Orlin © 2003 14

Simplex Algorithm step by step (Search for Max) Step 0. The problem is in canonical form withb  0. Step 1. If c  0 then stop. The solution is optimal. If there exists some cj>0, then we continue. Step 2. Choose any non-basic variable to pivot in with cs > 0, e.g., cs = maxj {cj | cj > 0 }. If ais  0 for all i, then stop; the LP is unbounded. If we continue, then there exists some ais > 0. Step 3. Pivot out the basic variable in row r, where r is chosen by the min ratio rule, that is r = argmini (bi/ais : ais > 0 ). Step 4. Replace the basic variable in row r with variable xs and re-establish canonical form (i.e., pivot on the coefficient ars) Step 5. Go back to Step 1. MIT and James Orlin © 2003 15

Today’s Lecture Review of the simplex algorithm. Alternative Optimal Solutions Unbounded Solution Degeneracy and Degenerate Solutions Proving finiteness and optimality under no degeneracy MIT and James Orlin © 2003 16

Degeneracy Initial Tableau BV -z x1 x2 x3 x4 -z 1 -3 2 -2 = x3 -3 3 1 1 6 = x4 -4 2 1 2 A bfs is degenerate if the Tableau revealsbi = 0 for some i. Otherwise, it is non-degenerate. This bfs is degenerate. MIT and James Orlin © 2003 17

Degeneracy 3 BV -z x1 x2 x3 x4 -z 1 -3 2 -2 = x3 -3 3 1 1 6 = x4 -4 2 -2 = x3 -3 3 1 1 6 = x4 -4 2 1 3 3 2 A bfs is degenerate if the Tableau revealsbi = 0 for some i. Otherwise, it is non-degenerate. This bfs is non-degenerate. MIT and James Orlin © 2003 18

Degeneracy BV -z x1 x2 x3 x4 -z 1 -3 2 -2 = x1 = 0 x2 = D x3 = 6 - 3D x4 = 0 - 2D z = 2 + 2D x3 -3 3 1 1 6 = x4 -4 2 1 2 Suppose that variable x2 will enter the basis. Degenerate  the solution may stay the same  z stays the same MIT and James Orlin © 2003 19

A degenerate pivoting Tableau No.1 BV -z x1 x2 x3 x4 -z 1 -3 1 2 -1 -2 -2 = x1 = 0 x2 = 0 x3 = 6 x4 = 0 z = 2 x3 -3 3 3 1 1 1 -3/2 6 6 = x4 -2 -4 1 2 1 1/2 The entering variable is x2, which takes a zero value. The exiting variable is the one in constraint 2. In this case the objective function did not change. (But the tableau did !) MIT and James Orlin © 2003 20

Cycling Problem: Solution: if one of the basic variables is zero: Xi = 0 + … a pivot could be performed which does not change the corresponding value of the objective function  danger of pivoting back (“cycling”) Solution: use e.g. Bland’s anti-cycling rule: always select the candidate pivot column with smallest index, i.e., the leftmost in Tableau (e.g., x2 instead of x4) if several rows have the same minimum ratio, choose for pivoting the first one (i.e., topmost) of them this is alternative to the pivot column selection rule that was previously discussed The Simplex Algorithm 11 November, 2002 21

Degeneracy Test for Degeneracy The next bfs will be degenerate ! The Simplex Algorithm 11 November, 2002 22

Non-degeneracy leads to strict improvement BV -z x1 x2 x3 x4 -z 1 -3 2 -2 = x1 = 0 x2 = D x3 = 6 - 3D x4 = 2 - 2D z = 2+ 2D x3 -3 3 1 1 6 = x4 -4 2 1 2 If the bfs is non-degenerate, then the entering variable can strictly increase, and the objective value will strictly improve. MIT and James Orlin © 2003 23

Cycling Alternative to Bland’s anti-cycling rule Perturb the RHS just a little, in just the right way No basis is degenerate Every bfs to the perturbed problem is also a bfs for the original problem The optimal basis for the perturbed problem is optimal for the original problem MIT and James Orlin © 2003 24

An LP before perturbation Initial Tableau BV -z x1 x2 x3 x4 -z 1 -3 2 = x3 -3 3 1 1 6. = x4 -4 2 1 2. Goal: modify the problem by changing the RHS just a little Perturb it to avoid degeneracy Perturb it so little that solving the perturbed problem also solves the original problem. MIT and James Orlin © 2003 25

An example of a perturbed initial bfs BV -z x1 x2 x3 x4 -z 1 -3 2 = x3 -3 3 1 1 6.00000000000013 = x4 -4 2 1 2.00000000000041 In theory perturbations are even smaller In practice, the simplex method works fine without perturbations, that is, it really obtains the optimum basic feasible solution. MIT and James Orlin © 2003 26

Today’s Lecture Review of the simplex algorithm. Alternative Optimal Solutions Unbounded Solution Degeneracy Degenerate Solutions Proving finiteness and optimality under no degeneracy MIT and James Orlin © 2003 27

Theorem: If every feasible basic solution is non-degenerate, the simplex method is finite. The number of feasible basic solutions is at most n! / (n-m)! m! which is the number of ways of selecting m basic variables Each feasible basic solution is different Each has a better cost than the last, assuming non-degeneracy Therefore, the simplex method is finite Typical number of iterations is between m and 3m. Number of operations per iteration: O(nm) MIT and James Orlin © 2003 28

Some Remarks on Degeneracy and Alternative Optima It might seem that degeneracy would be rare. After all, why should we expect that the RHS would be 0 for some variable? In reality, degeneracy is incredibly common. The simplex method is not necessarily finite unless care is taken in the pivot rule. In reality, almost no one takes care, and the simplex method is not only finite, but it is incredibly efficient. The issue of alternative optima arises frequently, and is of importance in practice. MIT and James Orlin © 2003 29

Software for Simplex implementation and solution

Demonstration Example Maximize the Objective Function (P) P = 15 x1 + 10 x2 subject to: L1: 0.25 x1 + 1.0 x2 <= 65 L2: 1.25 x1 + 0.5 x2 <= 90 L3: 1.0 x1 + 1.0 x2 <= 85       x1 >= 0; x2 >= 0 Solved step-by-step on Operations Research-Linear Programming-Simplex Algorithm by Elmer G. Wiens http://www.egwald.ca/operationsresearch/lpsimplex.php MIT and James Orlin © 2003 31

Demonstration Simplex solver Solved step-by-step Tableau and Final Solution on Finite mathematics utility: simplex method tool http://www.zweigmedia.com/RealWorld/simplex.html MIT and James Orlin © 2003 32

SIMPLEX with MatLab® function simplex2p(type, c, A, rel, b) LINEAR PROGRAMMING WITH MATLAB course by Edward Neuman Department of Mathematics Southern Illinois University at Carbondale function simplex2p(type, c, A, rel, b) % The Two-Phase Method for solving the LP problem % implements the Bland’s rule to avoid cycling in case of degeneracy % % min (max) z = c*x % subject to Ax rel b % x >= 0 . . . % input data type = ‘min’ (OR ‘max’) % simplex2p always searches for MIN! c = [c1 c2 ... ck] % is a row vector A = [a11 a12 … a1k; ... ; am1 am2 … amk] rel = ‘<<< … >>> … ===‘ b = [b1; b2; ... bm] % must be a column vector

Exercise 5 (for home) with contraints of type ≤ z = 3x1 + 2x2 - x3 + x4 max ! x1 + 2x2 + x3 - x4  5 inequality 2x1 + 4x2 + x3 + x4  1 inequality x1  0, x2  0, x3  0 non-negativity x4 uncostrained in sign