EMGT 6412/MATH 6665 Mathematical Programming Spring 2016 Simplex Method Dincer Konur Engineering Management and Systems Engineering

Outline Chapter 3 Extreme Points and Optimality Basic Feasible Solutions Extreme Points and Basic Feasible Solutions SIMPLEX Optimality of a basic feasible solution Algebra of the Simplex Method Simplex Algorithm Finding Starting Solution Simplex Tableau Chapter 3

Outline Extreme Points and Optimality Basic Feasible Solutions Extreme Points and Basic Feasible Solutions SIMPLEX Optimality of a basic feasible solution Algebra of the Simplex Method Simplex Algorithm Finding Starting Solution Simplex Tableau

Extreme Points and Optimality Consider the following linear programming model: Let Recall that any feasible solution x can be represented as:

Extreme Points and Optimality Then, the above LP is equal to: We assume k>=1

Extreme Points and Optimality Unboundedness: Existence of optimum: The problem is feasible and it is not unbounded So, we have:

Extreme Points and Optimality Summary: Given feasibility, a LP has an optimum solution (i.e., the optimal value is finite) if and only if 𝐜 𝐝 𝐣 ≥0 for all extreme directions If there is an optimum solution, then we find the minimizing point by selecting a solution having the minimum objective function value among all extreme points So, if an optimum solution exists, there exists at least on extreme point which is optimum

Extreme Points and Optimality Example 3.1 from the book: Let’s say we minimize

Extreme Points and Optimality So we have the following equivalent model:

Extreme Points and Optimality Example 3.1 from the book: Let’s say we minimize

Extreme Points and Optimality So we have the following equivalent model:

Outline Extreme Points and Optimality Basic Feasible Solutions Extreme Points and Basic Feasible Solutions SIMPLEX Optimality of a basic feasible solution Algebra of the Simplex Method Simplex Algorithm Finding Starting Solution Simplex Tableau

Basic Feasible Solutions We have showed that: If an optimum solution exists, then there exists an extreme point which is optimum Extreme points are geometric characterization In algebraic characterization, extreme points are basic feasible solutions We will show that extreme points are basic feasible solutions

Basic Feasible Solutions Basic Feasible Solution definition: Basic solution Basic feasible solution

Basic Feasible Solutions Basic Feasible Solution definition: Rearrange A Basic Matrix (Basis) Nonbasic Matrix Basic variables Nonbasic variables

Basic Feasible Solutions Example 3.2 from the book: Consider the following polyhedral: Introducing slack variables, we have:

Basic Feasible Solutions Example 3.2 from the book:

Basic Feasible Solutions Example 3.2 from the book: Basic feasible solutions:

Basic Feasible Solutions Example 3.3 from the book: Consider the following polyhedral: Introducing slack variables, we have:

Basic Feasible Solutions Example 3.3 from the book:

Basic Feasible Solutions Example 3.3 from the book: Basic feasible solution: This basic feasible solution is degenerate since each associated basis involves a basic variable at level zero. Note that degeneracy is not always simply the result of redundant constraints

Basic Feasible Solutions Number of basic solutions is less than or equal to:

Outline Extreme Points and Optimality Basic Feasible Solutions Extreme Points and Basic Feasible Solutions SIMPLEX Optimality of a basic feasible solution Algebra of the Simplex Method Simplex Algorithm Finding Starting Solution Simplex Tableau

Basic Feasible Solutions A point is a basic feasible solution if and only if it is an extreme point x is an extreme point of a feasible region if there are n linearly independent defining hyperplanes binding at x Read Section 2.6

Basic Feasible Solutions An extreme point is a basic feasible solution Suppose x is an extreme point Then provides m linearly independent defining hyperplanes binding at x Therefore, there must be p=n-m binding defining hyperplanes coming from the nonnegativity constraints Denoting these p additional hyperplanes Then

Basic Feasible Solutions An basic feasible solution is an extreme point Suppose x is a basic feasible solution Then That is, And they are linearly independent since Therefore, by definition

Summary of Results The collection of extreme points corresponds to the collection of basic feasible solutions, and both are nonempty provided that the feasible region is not empty. Assume that the feasible region is nonempty. Then a finite optimal solution exists if and only for all extreme directions dj If an optimal solution exists, then an optimal extreme point (or equivalently an optimal basic feasible solution) exists. For every extreme point (basic feasible solution) there is a corresponding basis (not necessarily unique), and, conversely, for every basis there is a corresponding (unique) extreme point.

Outline Extreme Points and Optimality Basic Feasible Solutions Extreme Points and Basic Feasible Solutions SIMPLEX Optimality of a basic feasible solution Algebra of the Simplex Method Simplex Algorithm Finding Starting Solution Simplex Tableau

Optimality of A BFS Key to the simplex is to recognize optimality of an extreme point without having to enumerate all basic feasible solutions Suppose we have a basic feasible solution

Optimality of A BFS Due to feasibility, we have

Optimality of A BFS Objective function value reads

Optimality of A BFS Then, we can write the LP as follows Observe that the variables simply play the role of slack variables

Optimality of A BFS We can equivalently write LP in the nonbasic variable space, that is, in terms of the nonbasic variables, as follows:

Optimality of A BFS This is feasible as is feasible Since for all nonbasic variables, we have f ; and we know that the current basic feasible solution has This is feasible as is feasible

Algebra of Simplex Otherwise, current bfs is not optimal So we want to increase xj for which k >=0 Maybe the most positive

Algebra of Simplex While holding (p - 1) nonbasic variables fixed at zero, the simplex method considers increasing the remaining variable, xk. Considering feasibility, we should have

Algebra of Simplex What happens if we increase xk It is then clear that the first basic variable dropping to zero corresponds to the minimum of for positive

Algebra of Simplex We can increase xk until

Algebra of Simplex So, we have a new point Exam question: Prove that the new point is also a basic feasible solution

Algebra of Simplex Example 3.4 from the book

Algebra of Simplex Reducing the problem into nonbasic variables

Algebra of Simplex Reducing the problem into nonbasic variables Current bfs optimum?

Algebra of Simplex Current bfs is not optimum increasing x3 improves objective function The maximum value of x3 is 2 i.e., reduction by 2 Is the new bfs optimum?

Algebra of Simplex Entering variable and leaving (blocking) variable We already know that if for each nonbasic variable j, the current bfs is optimum

Algebra of Simplex Suppose that for all nonbasic variables If for all nonbasic variables, then the optimum solution is unique Proof: Let x be any feasible solution that is distinct from x*. Then there is at least one nonbasic component xj that is positive, because if all nonbasic components are zero, x would not be distinct from x*. As it follows that ; hence, x* is the unique opt. If for at least one nonbasic variable , then we have alternative optima Furthermore, if then the alternative solutions are on a ray, otherwise, they are on a line segment

Algebra of Simplex Suppose that for nonbasic variable If we can increase xk as much as we want z goes to

Simplex Algorithm Assuming a minimization problem Initialization: Choose a starting basic feasible solution with basis B Main Step:

Simplex Algorithm Main Step: … Let

Simplex Algorithm Main Step: …

Simplex Algorithm Main Step: …

Simplex Algorithm Supplement 2 has more examples on: In the absence of degeneracy (and assuming feasibility), the simplex method stops in a finite number of iterations, either with an optimal basic feasible solution or with the conclusion that the optimal objective value is unbounded. In the presence of degeneracy, however, there is the possibility of cycling in an infinite loop. This issue about preventing cycling is discussed Chapter 4. We assume that we have a starting basic feasible solution. There are methods to find starting basic feasible solutions: Big-M and Two-Phase Methods are discussed in Chapter 4 Supplement 2 has more examples on: Simplex Algebra

Finding Starting Solution To start the Simplex method, an initial BFS is needed When we only have <= constraints with positive right-side values, the slack variables as the basic variables in the augmented model will define a basic feasible solution In other cases, however, it might be the case that the slack variables do not define a BFS For instance, for = constraints we do not have slack variables! For <= constraints, we subtract a slack variable, so we might end up with negative value for that slack variable initially! In the cases we do not have an obvious BFS, we can use artificial variables to find an initial BFS in two ways: Big M method and Two-phase method Note that if we cannot find an initial BFS, the problem is infeasible!!!

Big M Method Let’s consider a simple example Unfortunately, Equations (1)-(3) do not have an obvious BFS Think what would be the case for larger and larger problems!! So we need a methodology to find a starting BFS

Big M Method The idea is to come up with an artificial model that will have the same optimum solution To do so, we use artificial variables as needed Then, we augment the model The cost of the artificial variable is very very high denoted by big M If the objective was min., we would add big M times the artificial variable The artificial variable should be zero for a feasible solution of the real problem

Big M Method The model with the big M Now, x3, x4, and 𝑥 5 are the basic variables When do we need artificial variables? For “=“ constraints For “>=“ constraints with positive right-hand sides Because, we need to subtract a slack variable And if we let that slack variable be a basic variable, its value will be negative (violates feasibility) Minimize s.t.

Big M Method At termination of the Simplex table of the artificial problem If the artificial variable is a basic variable with positive value in the termination of the Simplex (either due to finding optimum solution or unboundedness), the real problem is infeasible! Otherwise, if artificial variables are non-basic variables at termination, i.e., they have 0 value, then you have found a basic feasible solution to the real problem as soon as all artificial variables are non-basic

Two-Phase Method Consider the following example (a radiation therapy problem): Add slack variable x3>=0 for the 1st constraint Add artificial variable x4>=0 for the 2nd constraint Subtract slack variable x5>=0 for the 3rd constraint Add artificial variable x6>= for the 3rd constraint

Two-Phase Method Two-Phase method: The idea is to first solve a problem to make the artificial variables zero (Phase 1) Here, you always minimize the some of the artificial variables subject to the constraints of the augmented model you have after adding slack and artificial variables Then solve the real problem with artificial variables equal to zeroes

Two-Phase Method Phase 1 problem: If the optimum solution of Phase 1 has at least one artificial variable equal to non-zero, then the original problem is infeasible If not, the basic variables of the optimum solution of the Phase 1 problem constitute a basic feasible solution for the original Problem Phase 2: continue with the original problem eliminating the artificial variables using the bfs found at the end of Phase 1

Simplex Tableau At each iteration of the simplex algorithm, the following linear systems of equations need to be solved: These can be handled in table format.

Simplex Tableau Suppose we have a bfs x with basis B Add Multiply with cB

Simplex Tableau value of the objective function values of the basic variables Consists of the zj-cj values for the nonbasic variables The columns here are the yj vectors

Simplex Tableau Pivoting: If xk enters the basis and leaves the basis, then pivoting on can be stated as follows:

Simplex Tableau Example 3.9 from the book:

Simplex Tableau Example 3.9 from the book: Min.ratio 9/2=4.5 -- 4/1=4

Simplex Tableau Example 3.9 from the book: Min.ratio 1/3 -- ---

Next time Duality Supplement 2 has more examples on: Simplex Tableau Big-M method 2-Phase Method