MIT and James Orlin © Chapter 3. The simplex algorithm Putting Linear Programs into standard form Introduction to Simplex Algorithm

MIT and James Orlin © Overview of Lecture Getting an LP into standard form Getting an LP into canonical form Optimality conditions Improving a solution A simplex pivot Recognizing when an LP is unbounded Overview of what remains

MIT and James Orlin © Overview of Lecture Getting an LP into standard form

MIT and James Orlin © Linear Programs in Standard Form We say that a linear program is in standard form if the following are all true: 1. Non-negativity constraints for all variables. 2. All remaining constraints are expressed as equality constraints. 3. The right hand side vector, b, is non-negative. maximize 3x 1 + 2x 2 - x 3 + x 4 x 1 + 2x 2 + x 3 - x 4  5; -2x 1 - 4x 2 + x 3 + x 4  -1; x 1  0, x 2  0 An LP not in Standard Form not equality x 3 may be negative

MIT and James Orlin © s 1 is called a slack variable, which measures the amount of “unused resource.” Note that s 1 = 5 - x 1 - 2x 2 - x 3 + x 4. Converting Inequalities into Equalities Plus Non-negatives To convert a “  ” constraint to an equality, add a slack variable. x 1 + 2x 2 + x 3 - x 4 +s 1 = 5 s 1  0 After Before x 1 + 2x 2 + x 3 - x 4  5

MIT and James Orlin © Converting “  ” constraints Consider the inequality -2x 1 - 4x 2 + x 3 + x 4  -1; Step 1. Eliminate the negative RHS 2x 1 + 4x 2 - x 3 - x 4  1 Step 2. Convert to an equality 2x 1 + 4x 2 - x 3 - x 4 – s 2 = 1 s 2  0 The variable added will be called a “surplus variable.” To covert a “  ” constraint to an equality, subtract a surplus variable.

MIT and James Orlin © More Transformations How can one convert a maximization problem to a minimization problem? Example: Maximize 3W + 2P Subject to “constraints” Has the same optimum solution(s) as Minimize -3W -2P Subject to “constraints”

MIT and James Orlin © The Last Transformations (for now) Transforming x 1 : replace x 1 by y 1 = -x 1 ; y 1  0. One can recover x 1 from y 1. max -3 y 1 + 4x 2 +5 x 3 -2 y 1 -5 x 2 +2 x 3 =7 y 1   0, x 2 is unconstrained in sign, x 3  0 Transforming variables that may take on negative values. maximize 3x 1 + 4x x 3 subject to 2x 1 - 5x 2 + 2x 3 = 7 other constraints x 1  0, x 2 is unconstrained in sign, x 3  0

MIT and James Orlin © max -3 y 1 + 4(y 3 - y 2 ) +5 x 3 -2 y 1 -5 y 3 +5 y 2 +2 x 3 =7 all vars  0 max -3 y 1 + 4x 2 +5 x 3 -2 y 1 -5 x 2 +2 x 3 =7 y 1   0, x 2 is unconstrained in sign, x 3  0 Transforming variables that may take on negative values. Transforming x 2 : replace x 2 by x 2 = y 3 - y 2 ; y 2  0, y 3  0. One can recover x 2 from y 2, y 3. e.g., y 1 = 1, x 2 = -1, x 3 = 2 is feasible. e.g., y 1 = 1, y 2 = 0, y 3 = 1 x 3 = 2 is feasible.

MIT and James Orlin © Another Example Exercise: transform the following to standard form (maximization): Minimize x 1 + 3x 2 Subject to 2x 1 + 5x 2   12 x 1 + x 2  1 x 1  0 Perform the transformation with your partner

MIT and James Orlin © Preview of the Simplex Algorithm x1x1 x2x2 x4x4 x3x3 -z = = 0 maximize z = -3x 1 + 2x 2 subject to -3x 1 + 3x 2 +x 3 = 6 -4x 1 + 2x 2 + x 4 = 2 x 1, x 2, x 3, x 4  =

MIT and James Orlin © Overview of Lecture Getting an LP into standard form Getting an LP into canonical form

MIT and James Orlin © LP Canonical Form = LP Standard Form + Jordan Canonical Form ==== 2 6 x1x1 x2x2 x4x4 x3x3 -z = 0 The simplex method starts with an LP in LP canonical form (or it creates canonical form at a preprocess step.) z is not a decision variable x4x4 x3x3

MIT and James Orlin © LP Canonical Form ==== 2 6 x1x1 x2x2 x4x4 x3x3 -z = 0 The basic feasible solution is x 1 = 0, x 2 = 0, x 3 = 6, x 4 = 2 (set the non-basic variables to 0, and then solve) The basic variables are x 3 and x 4. The non-basic variables are x 1 and x 2. z is not a decision variable x4x4 x3x3

MIT and James Orlin © For each constraint there is a basic variable ==== 2 6 x2x2 x4x4 x3x z = 0 Constraint 1: basic variable is x 3 Constraint 1 Constraint 2: basic variable is x 4 Constraint 2 The basis consists of variables x 3 and x 4 x1x

MIT and James Orlin © Overview of Lecture Getting an LP into standard form Getting an LP into canonical form Optimality conditions

MIT and James Orlin © x1x Optimality Conditions Preview ==== 2 6 x2x2 x4x4 x3x z = 0 Obvious Fact: If one can improve the current basic feasible solution x, then x is not optimal. Idea: assign a small value to just one of the non-basic variables, and then adjust the basic variables. Note: z = -3x 1 + 2x 2

MIT and James Orlin © x1x The current basic feasible solution (bfs) is not optimal! ==== 2 6 x2x2 x4x4 x3x z = 0 Increase x 2 to  > 0. Let x 1 stay at 0. x 3 = . x 4 = . z = 2 . x1x If there is a positive coefficient in the z row, the basis is not optimal** Recall: z = -3x 1 + 2x 2 What happens to x 3, x 4 and z?.

MIT and James Orlin © Optimality Conditions ==== 2 6 x2x2 x4x4 x3x z = -8 z = -2x 1 - 4x Therefore z  8 for all feasible solutions. x1x1 Important Fact. If there is no positive coefficient in the z row, the basic feasible solution is optimal! (note that the data is different here) But z = 8 in the current basic feasible solution This basic feasible solution is optimal!

MIT and James Orlin © x 1 = 0 x 2 =  x 3 = . x 4 = . z = 2 . x 1 = 0 x 2 = 1 x 3 = 3 x 4 = 0 z = 2 x1x Let x 2 = . How large can  be? What is the solution after changing x 2 ? ==== 2 6 x2x2 x4x4 x3x z = 0 What is the value of  that maximizes z, but leaves a feasible solution? x1x  = 1. Fact. The resulting solution is a basic feasible solution for a different basis.

MIT and James Orlin © Overview of Lecture Getting an LP into standard form Getting an LP into canonical form Optimality conditions Improving a solution, a pivot

MIT and James Orlin © x1x Pivoting to obtain a better solution ==== x2x2 x4x4 x3x z = 0 If we pivot on the coefficient 2, we obtain the new basic feasible solution x 1 = 0 x 2 = 1 x 3 = 3 x 4 = 0 z = 2 New Solution: basic variables are x 2 and x 3. Nonbasics: x 1 and x 4.

MIT and James Orlin © Summary of Simplex Algorithm Start in canonical form with a basic feasible solution 1. Check for optimality conditions 2. If not optimal, determine a non-basic variable that should be made positive 3. Increase that non-basic variable, and perform a pivot, obtaining a new bfs 4. Continue until optimal (or unbounded).

MIT and James Orlin © x1x1 OK. Let’s iterate again. ==== x2x2 x4x4 x3x3 -z = The cost coefficient of x 1 is positive. Set x 1 =  and x 4 = x 1 =  x 2 =  x 3 = . x 4 = 0 z = 2 +  How large can  be? z = x 1 – x

MIT and James Orlin © Overview of Lecture Getting an LP into standard form Getting an LP into canonical form Optimality conditions Improving a solution A simplex pivot Recognizing when an LP is unbounded

MIT and James Orlin © x1x1 A Digression: What if we had a problem in which  could increase to infinity? ==== x2x2 x4x4 x3x3 -z = Suppose we change the 3 to a –3. Set x 1 =  and x 4 = x 1 =  x 2 =  x 3 = . x 4 = 0 z = 2 +  How large can  be? z = x 1 – x If the non-cost coefficients in the entering column are  0, then the solution is unbounded

MIT and James Orlin © /3-1/2 00-1/3-1/2 01/3-1/2 x1x1 End Digression: Perform another pivot ==== x2x2 x4x4 x3x3 -z = What is the largest value of  ? x 1 =  x 2 =  x 3 = . x 4 = 0 z = 2 +   x 1 =  x 2 = 3 x 3 = 0 x 4 = 0 z = 3 Pivot on the coefficient with a Variable x 1 becomes basis, x 3 becomes nonbasic. So, x 1 becomes the basic variable for constraint 1.

MIT and James Orlin © /31/2 00-1/3-1/2 101/3-1/2 x1x1 Check for optimality ==== x2x2 x4x4 x3x3 -z = There is no positive coefficient in the z-row. x 1 =  x 2 =  x 3 = . x 4 = 0 z = 2 +  The current basic feasible solution is optimal! x 1 =  x 2 = 3 x 3 = 0 x 4 = 0 z = 3 z = -x 3 /3 – x 4 /2 + 3

MIT and James Orlin © Summary of Simplex Algorithm Again Start in canonical form with a basic feasible solution 1. Check for optimality conditions Is there a positive coefficient in the cost row? 2. If not optimal, determine a non-basic variable that should be made positive Choose a variable with a positive coef. in the cost row. 3. increase that non-basic variable, and perform a pivot, obtaining a new bfs (or unboundedness) We will review this step, and show a shortcut 4. Continue until optimal (or unbounded).

MIT and James Orlin © x1x1 Performing a “Pivot”. Towards a shortcut. ==== x2x2 x4x4 x3x3 -z = Exercise: to do with your partner x 1 =  x 2 =  x 3 = . x 4 =  z =  1. Determine how large  can be. 2. Determine the next solution. 3. Determine what coefficient should be pivoted on. 4. The choice depends on ratios involving the coefficients. What is the rule for determining the coef? z = 2x = 5

MIT and James Orlin © More on performing a pivot To determine the column to pivot on, select a variable with a positive cost coefficient To determine a row to pivot on, select a coefficient according to a minimum ratio rule Carry out a pivot as one does in solving a system of equations.

MIT and James Orlin © Next Lecture Review of the simplex algorithm Formalizing the simplex algorithm How to find an initial basic feasible solution, if one exists A proof that the simplex algorithm is finite (assuming non-degeneracy)