1. EMGT 6412/MATH 6665 Mathematical Programming Spring 2016
Simplex Method
Dincer Konur
Engineering Management and Systems Engineering

2. Outline Chapter 3 Extreme Points and Optimality
Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau
Chapter 3

3. Outline Extreme Points and Optimality Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau

4. Extreme Points and Optimality
Consider the following linear programming model:
Let
Recall that any feasible solution x can be represented as:

5. Extreme Points and Optimality
Then, the above LP is equal to:
We assume k>=1

6. Extreme Points and Optimality
Unboundedness:
Existence of optimum:
The problem is feasible and it is not unbounded
So, we have:

7. Extreme Points and Optimality
Summary:
Given feasibility, a LP has an optimum solution (i.e., the optimal value is finite) if and only if 𝐜 𝐝 𝐣 ≥0 for all extreme directions
If there is an optimum solution, then we find the minimizing point by selecting a solution having the minimum objective function value among all extreme points
So, if an optimum solution exists, there exists at least on extreme point which is optimum

8. Extreme Points and Optimality
Example 3.1 from the book:
Let’s say we minimize

9. Extreme Points and Optimality
So we have the following equivalent model:

10. Extreme Points and Optimality
Example 3.1 from the book:
Let’s say we minimize

11. Extreme Points and Optimality
So we have the following equivalent model:

12. Outline Extreme Points and Optimality Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau

13. Basic Feasible Solutions
We have showed that:
If an optimum solution exists, then there exists an extreme point which is optimum
Extreme points are geometric characterization
In algebraic characterization, extreme points are basic feasible solutions
We will show that extreme points are basic feasible solutions

14. Basic Feasible Solutions
Basic Feasible Solution definition:
Basic solution
Basic feasible solution

15. Basic Feasible Solutions
Basic Feasible Solution definition:
Rearrange A
Basic Matrix (Basis)
Nonbasic Matrix
Basic variables
Nonbasic variables

16. Basic Feasible Solutions
Example 3.2 from the book:
Consider the following polyhedral:
Introducing slack variables, we have:

17. Basic Feasible Solutions
Example 3.2 from the book:

18. Basic Feasible Solutions
Example 3.2 from the book:
Basic feasible solutions:

19. Basic Feasible Solutions
Example 3.3 from the book:
Consider the following polyhedral:
Introducing slack variables, we have:

20. Basic Feasible Solutions
Example 3.3 from the book:

21. Basic Feasible Solutions
Example 3.3 from the book:
Basic feasible solution:
This basic feasible solution is degenerate since each associated basis involves a basic variable at level zero.
Note that degeneracy is not always simply the result of redundant constraints

22. Basic Feasible Solutions
Number of basic solutions is less than or equal to:

23. Outline Extreme Points and Optimality Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau

24. Basic Feasible Solutions
A point is a basic feasible solution if and only if it is an extreme point
x is an extreme point of a feasible region if there are n linearly independent defining hyperplanes binding at x
Read Section 2.6

25. Basic Feasible Solutions
An extreme point is a basic feasible solution
Suppose x is an extreme point
Then provides m linearly independent defining hyperplanes binding at x
Therefore, there must be p=n-m binding defining hyperplanes coming from the nonnegativity constraints
Denoting these p additional hyperplanes
Then

26. Basic Feasible Solutions
An basic feasible solution is an extreme point
Suppose x is a basic feasible solution
Then
That is,
And they are linearly independent since
Therefore, by definition

27. Summary of Results
The collection of extreme points corresponds to the collection of basic feasible solutions, and both are nonempty provided that the feasible region is not empty.
Assume that the feasible region is nonempty. Then a finite optimal solution exists if and only for all extreme directions dj
If an optimal solution exists, then an optimal extreme point (or equivalently an optimal basic feasible solution) exists.
For every extreme point (basic feasible solution) there is a corresponding basis (not necessarily unique), and, conversely, for every basis there is a corresponding (unique) extreme point.

28. Outline Extreme Points and Optimality Basic Feasible Solutions
Extreme Points and Basic Feasible Solutions
SIMPLEX
Optimality of a basic feasible solution
Algebra of the Simplex Method
Simplex Algorithm
Finding Starting Solution
Simplex Tableau

29. Optimality of A BFS
Key to the simplex is to recognize optimality of an extreme point without having to enumerate all basic feasible solutions
Suppose we have a basic feasible solution

30. Optimality of A BFS
Due to feasibility, we have

31. Optimality of A BFS
Objective function value reads

32. Optimality of A BFS Then, we can write the LP as follows
Observe that the variables simply play the role of slack variables

33. Optimality of A BFS
We can equivalently write LP in the nonbasic variable space, that is, in terms of the nonbasic variables, as follows:

34. Optimality of A BFS This is feasible as is feasible
Since for all nonbasic variables, we have f ; and we know that the current basic feasible solution has
This is feasible as is feasible

35. Algebra of Simplex Otherwise, current bfs is not optimal
So we want to increase xj for which k >=0
Maybe the most positive

36. Algebra of Simplex
While holding (p - 1) nonbasic variables fixed at zero, the simplex method considers increasing the remaining variable, xk.
Considering feasibility, we should have

37. Algebra of Simplex What happens if we increase xk
It is then clear that the first basic variable dropping to zero corresponds to the minimum of for positive

38. Algebra of Simplex
We can increase xk until

39. Algebra of Simplex So, we have a new point
Exam question: Prove that the new point is also a basic feasible solution

40. Algebra of Simplex
Example 3.4 from the book

41. Algebra of Simplex
Reducing the problem into nonbasic variables

42. Algebra of Simplex Reducing the problem into nonbasic variables
Current bfs optimum?

43. Algebra of Simplex Current bfs is not optimum
increasing x3 improves objective function
The maximum value of x3 is 2
i.e., reduction by 2
Is the new bfs optimum?

44. Algebra of Simplex Entering variable and leaving (blocking) variable
We already know that if for each nonbasic variable j, the current bfs is optimum

45. Algebra of Simplex Suppose that for all nonbasic variables
If for all nonbasic variables, then the optimum solution is unique
Proof: Let x be any feasible solution that is distinct from x*. Then there is at least one nonbasic component xj that is positive, because if all nonbasic components are zero, x would not be distinct from x*. As it follows that ; hence, x* is the unique opt.
If for at least one nonbasic variable , then we have alternative optima
Furthermore, if then the alternative solutions are on a ray, otherwise, they are on a line segment

46. Algebra of Simplex Suppose that for nonbasic variable
If we can increase xk as much as we want
z goes to

47. Simplex Algorithm Assuming a minimization problem
Initialization: Choose a starting basic feasible solution with basis B
Main Step:

48. Simplex Algorithm
Main Step: …
Let

49. Simplex Algorithm
Main Step: …

50. Simplex Algorithm
Main Step: …

51. Simplex Algorithm Supplement 2 has more examples on:
In the absence of degeneracy (and assuming feasibility), the simplex method stops in a finite number of iterations, either with an optimal basic feasible solution or with the conclusion that the optimal objective value is unbounded.
In the presence of degeneracy, however, there is the possibility of cycling in an infinite loop. This issue about preventing cycling is discussed Chapter 4.
We assume that we have a starting basic feasible solution. There are methods to find starting basic feasible solutions:
Big-M and Two-Phase Methods are discussed in Chapter 4
Supplement 2 has more examples on:
Simplex Algebra

52. Finding Starting Solution
To start the Simplex method, an initial BFS is needed
When we only have <= constraints with positive right-side values, the slack variables as the basic variables in the augmented model will define a basic feasible solution
In other cases, however, it might be the case that the slack variables do not define a BFS
For instance, for = constraints we do not have slack variables!
For <= constraints, we subtract a slack variable, so we might end up with negative value for that slack variable initially!
In the cases we do not have an obvious BFS, we can use artificial variables to find an initial BFS in two ways:
Big M method and Two-phase method
Note that if we cannot find an initial BFS, the problem is infeasible!!!

53. Big M Method Let’s consider a simple example
Unfortunately, Equations (1)-(3) do not have an obvious BFS
Think what would be the case for larger and larger problems!!
So we need a methodology to find a starting BFS

54. Big M Method
The idea is to come up with an artificial model that will have the same optimum solution
To do so, we use artificial variables as needed
Then, we augment the model
The cost of the artificial variable is very very high denoted by big M
If the objective was min., we would add big M times the artificial variable
The artificial variable should be zero for a feasible solution of the real problem

55. Big M Method The model with the big M
Now, x3, x4, and 𝑥 5 are the basic variables
When do we need artificial variables?
For “=“ constraints
For “>=“ constraints with positive right-hand sides
Because, we need to subtract a slack variable
And if we let that slack variable be a basic variable, its value will be negative (violates feasibility)
Minimize
s.t.

56. Big M Method
At termination of the Simplex table of the artificial problem
If the artificial variable is a basic variable with positive value in the termination of the Simplex (either due to finding optimum solution or unboundedness), the real problem is infeasible!
Otherwise, if artificial variables are non-basic variables at termination, i.e., they have 0 value, then you have found a basic feasible solution to the real problem as soon as all artificial variables are non-basic

57. Two-Phase Method
Consider the following example (a radiation therapy problem):
Add slack variable x3>=0 for the 1st constraint
Add artificial variable x4>=0 for the 2nd constraint
Subtract slack variable x5>=0 for the 3rd constraint
Add artificial variable x6>= for the 3rd constraint

58. Two-Phase Method Two-Phase method:
The idea is to first solve a problem to make the artificial variables zero (Phase 1)
Here, you always minimize the some of the artificial variables subject to the constraints of the augmented model you have after adding slack and artificial variables
Then solve the real problem with artificial variables equal to zeroes

59. Two-Phase Method Phase 1 problem:
If the optimum solution of Phase 1 has at least one artificial variable equal to non-zero, then the original problem is infeasible
If not, the basic variables of the optimum solution of the Phase 1 problem constitute a basic feasible solution for the original Problem
Phase 2: continue with the original problem eliminating the artificial variables using the bfs found at the end of Phase 1

60. Simplex Tableau
At each iteration of the simplex algorithm, the following linear systems of equations need to be solved:
These can be handled in table format.

61. Simplex Tableau Suppose we have a bfs x with basis B Add
Multiply with cB

62. Simplex Tableau value of the objective function
values of the basic variables
Consists of the zj-cj values for the nonbasic variables
The columns here are the yj vectors

63. Simplex Tableau Pivoting:
If xk enters the basis and leaves the basis, then pivoting on can be stated as follows:

64. Simplex Tableau
Example 3.9 from the book:

65. Simplex Tableau
Example 3.9 from the book:
Min.ratio
9/2=4.5 --
4/1=4

66. Simplex Tableau
Example 3.9 from the book:
Min.ratio
1/3 --
---

67. Next time Duality Supplement 2 has more examples on: Simplex Tableau
Big-M method
2-Phase Method