Perturbation method, lexicographic method Idea: Avoid the appearance of degenerate solutions since it is a precondition of cycling ( If the solution is not degenerate, the objective function value increases strictly in the next iteration, hence the same basis does not appear again because the objective function value never decreases during the simplex iterations (same basis  same dictionary  same objective value)). So add very small positive 𝜀 to the r.h.s of equations so that the solution values are unchanged practically but degeneracy is avoided. But the 𝜀 ′ 𝑠 added to the r.h.s may cancel out each other during elementary row operations, again causing degeneracy. Remedy: Add different values of 𝜀 𝑖 ′ 𝑠 to different r.h.s. so that cancellation does not occur. (Perturbation method) OR-1 2017

Add 𝜀 𝑖 to the 𝑖−𝑡ℎ r.h.s., 𝑖=1,…,𝑚 with the following property. 0< 𝜀 𝑚 ≪ 𝜀 𝑚−1 ≪…≪ 𝜀 2 ≪ 𝜀 1 ≪1 (3.4) Then, it can be shown that the values of the basic variables never become 0 in subsequent simplex iterations, hence no cycling occurs. (In practice, precision of computation in computer can cause problems.) (In actual implementation, 𝜀 1 =𝜀, 𝜀 2 = 𝜀 2 , 𝜀 3 = 𝜀 3 , … may be used for small 𝜀>0 or random numbers in [0,𝜀] for some fixed small 𝜀 are used. ) OR-1 2017

Perturbation Method (tableau form) (0+21 ) (0+22 ) (1+3 ) (after two iterations, optimal solution obtained) OR-1 2017

Perturbation method usually refers to the method which actually adds small 𝜀 𝑖 ′ 𝑠 to the right hand sides. For lexicographic method, the idea is the same as the perturbation method. But we do not actually add some positive numbers to the r.h.s., but treat 𝜀 𝑖 ′ 𝑠 as symbols representing indefinite quantities, which satisfy (3.4). Numbers are compared in lexicographic sense (more in later slide) Note that, in the lexicographic method, we can obtain the optimal solution and optimal value correctly after disregarding all the terms involving 𝜀 𝑖 ′ 𝑠. The coefficients of 𝜀 𝑖 ′ 𝑠 do not affect the coefficients of other variables during simplex iterations. During degenerate iterations, the actual objective value does not increase. However, considering 𝜀 terms, the objective value strictly increases even during a degenerate iteration. OR-1 2017

Lexicographic ordering of numbers : Consider 𝑟= 𝑟 0 + 𝑟 1 𝜀 1 +…+ 𝑟 𝑚 𝜀 𝑚 , 𝑠= 𝑠 0 + 𝑠 1 𝜀 1 +…+ 𝑠 𝑚 𝜀 𝑚 . If 𝑟≠𝑠, there is the smallest subscript 𝑘 such that 𝑟 𝑘 ≠ 𝑠 𝑘 . We say that 𝑟 is lexicographically smaller than 𝑠 if 𝑟 𝑘 < 𝑠 𝑘 . (Similar terminology is used for vectors too) Then if 0< 𝜀 𝑚 ≪ 𝜀 𝑚−1 ≪…≪ 𝜀 2 ≪ 𝜀 1 ≪1, 𝑟 is lexicographically smaller than 𝑠 if and only if 𝑟 is numerically smaller than 𝑠. (see problem 3.7 on page 44) Ex) r = 2 + 211 + 19 2 + 200003 s = 2 + 211 + 202 + 203 + 154 + 145 𝑟 is lexicographically smaller than 𝑠. OR-1 2017

Can cycling be prevented? Thm 3.2: The simplex method terminates as long as the leaving variable is selected by the lexicographic rule in each iteration. Pf) see the proof in the text. For an arbitrary row with r.h.s. value 𝑟 0 + 𝑟 1 𝜀 1 +…+ 𝑟 𝑚 𝜀 𝑚 , the proof shows that at least one of 𝑟 1 , 𝑟 2 , …, 𝑟 𝑚 is distinct from zero. Hence no degenerate solution appears. We may consider somewhat different logic to prove the result. The coefficient matrix for  variables in the constraints remain nonsingular after applying elementary row operations. (Elementary row operation is equivalent to premultiplying the corresponding nonsingular matrix to the coefficient matrix of constraints and the right hand sides. Since the matrix for  variables is identity matrix initially, it is nonsingular. Hence we also obtain nonsingular coefficient matrix after elementary row operations are performed to the constraint matrix). In other words, no row with all 0 elements appear in the -matrix. Hence the values of basic variables never become 0 in lexicographic sense. OR-1 2017

We can read the real solution value by ignoring the  terms in the current dictionary (tableau). It is also observed that the lexicographic method can be started and stopped at any time during the simplex iterations. We just add or drop the  terms at any time and it does not affect the real solution value and the coefficients of other variables in the tableau. OR-1 2017

Practical Implementation of the Lexicographic Method In real implementation of the lexicographic method, we do not actually add  terms to the r.h.s., but read the coefficients of  terms from the coefficients of other variables. Note that, in the example, the coefficient matrix for the basic variables x5, x6, x7 and the coefficient matrix for 1, 2, 3 are the same identity matrices in the beginning of the lexicographic method. Since we use the elementary row operations in the simplex pivots, those two coefficient matrices have the same elements in the following iterations. Hence, we can read the coefficients of 1, 2, 3 from the coefficients of x5, x6, x7 . So we do not actually need to add 1, 2, 3 to the tableau. OR-1 2017

Hence, in the example, we actually do not need to add 𝜀 𝑖 ′ 𝑠 Hence, in the example, we actually do not need to add 𝜀 𝑖 ′ 𝑠. We first perform ratio test using the entering column and the r.h.s. If ties occur, then perform ratio test (only for tied rows) again using the entering column and the column for 𝑥 5 , and then the column for 𝑥 6 , and then for 𝑥 7 , until tie is broken (lexicographic comparison of numbers). OR-1 2017

Example of real implementation (0+21 ) (0+22 ) (1+3 ) (after two iterations, optimal solution obtained) OR-1 2017

Initialization (two-phase method) We need an initial b.f.s to start the simplex method. If we have 𝑏 𝑖 <0 for some constraint 𝑖, the value of the slack variable 𝑥 𝑛+𝑖 = 𝑏 𝑖 <0 violates nonnegativitiy constraint. maximize 𝑗=1 𝑛 𝑐 𝑗 𝑥 𝑗 subject to 𝑗=1 𝑛 𝑎 𝑖𝑗 𝑥 𝑗 ≤ 𝑏 𝑖 , 𝑖=1,…,𝑚 (1) 𝑥 𝑗 ≥0, 𝑗=1, 2, …, 𝑛 Consider easily obtained solution 𝑥 𝑖 =0 for all 𝑖 in (1) and then subtract some positive number 𝑥 0 from all left-hand sides so that it becomes feasible solution to (1), i.e. 𝑥 𝑖 =0 for all 𝑖, 𝑥 0 = 𝑚𝑎𝑥 {𝑖: 𝑏 𝑖 <0} | 𝑏 𝑖 |. Now, if we can find a feasible solution to (1) with 𝑥 0 =0, it is a feasible solution to (1) using original variables. OR-1 2017

subject to 𝑗=1 𝑛 𝑎 𝑖𝑗 𝑥 𝑗 − 𝑥 0 ≤ 𝑏 𝑖 , 𝑖=1,…,𝑚 (2) Hence solve the following problem using the easily obtained initial feasible solution and simplex method to find an optimal solution with 𝑥 0 =0. Note that 𝑥 0 is a nonnegative variable. maximize −𝑥 0 (min 𝑥 0 ) subject to 𝑗=1 𝑛 𝑎 𝑖𝑗 𝑥 𝑗 − 𝑥 0 ≤ 𝑏 𝑖 , 𝑖=1,…,𝑚 (2) 𝑥 𝑗 ≥0, 𝑗=0, 1, 2, …, 𝑛 (1) has a feasible solution  (2) has an optimal solution with optimal value 0 ( 𝑥 0 =0) If we find an optimal solution with 𝑥 0 =0, we can obtain a feasible solution to (1) by disregarding 𝑥 0 . One point to be careful is that we need a basic feasible solution to start the simplex method subsequently. OR-1 2017

However a feasible dictionary can be easily obtained by one pivot. Example We cannot perform simplex iteration in this dictionary since the basic solution is not feasible (nonnegativity violated) However a feasible dictionary can be easily obtained by one pivot. OR-1 2017

Current basic solution is not feasible Current basic solution is not feasible. Let 𝑥 0 enter the basis and the slack variable with most negative value leaves the basis (It is not a simplex iteration. Just perform the pivot, not considering the objective value. It does not change the solution set.) Perform simplex method. After two iterations, we get the optimal dictionary OR-1 2017

We obtained optimal solution with value 0 We obtained optimal solution with value 0. Hence the current optimal solution gives a b.f.s. to the original problem. Drop 𝑥 0 (no more needed) and replace the objective function with the original one. Zeroth equation is used to read the objective value of a given solution, hence it can be added or dropped without affecting the feasible solution set to the LP. Note that the current b.f.s. is a b.f.s. to the original problem (We don’t have 𝑥 0 as a basic variable). OR-1 2017

Now, restart the simplex method with the current dictionary Express it in dictionary form (only nonbasic variables appear in the r.h.s.) by substituting the basic variables 𝑥 2 , 𝑥 3 appearing in the objective function. Now, restart the simplex method with the current dictionary OR-1 2017

You need to understand that (i) 𝑥 0 can be added or dropped from the dictionary, but the set of feasible solutions to LP (disregarding the variable 𝑥 0 ) doesn’t change since the coefficients of 𝑥 0 in the dictionary don’t affect the coefficients of other variables as long as we perform elementary row operations (simplex pivots). The additional variable 𝑥 0 is used temporarily for the purpose of identifying a basic feasible solution to the original problem. Once the value of 𝑥 0 becomes 0 (as a nonbasic variable), 𝑥 0 can be eliminated without affecting the values of the current solution and coefficients of the other variables in the dictionalry. Also (ii) the zeroth equation can be exchanged by another zeroth equation at any time, but the set of feasible solutions to LP doesn’t change because the set of feasible solutions to LP has not been affected by the zeroth equation during the simplex pivots. See the next slides for the two phase method, which are expressed in tableau format. OR-1 2017

-w -x0 = 0 2x1 - x2 + 2x3 - x0 + x4 = 4 2x1 - 3x2 + x3 - x0 + x5 = -5 -x1 + x2 - 2x3 - x0 + x6 = -1 tableau form -w -2x1 + 3x2 - x3 - x5 = 5 2x2 + x3 + x4 - x5 = 9 -2x1 + 3x2 - x3 + x0 - x5 = 5 -3x1 + 4x2 - 3x3 - x5 + x6 = 4 elmentary row operations -w - x0 = 0 0.2x1 + x3 +0.8x0 - 0.2x5 - 0.6x6 = 1.6 -0.6x1 + x2 +0.6x0 - 0.4x5 - 0.2x6 = 2.2 x1 -2x0 + x4 + x6 = 3 OR-1 2017

Note that the coefficients of x0 do not affect the coefficients of other variables in the simplex pivots. Suppose we perform the same pivots, disregarding variable x0. Then we obtain the same final tableau without x0. The 3 constraints in the initial tableau without variable x0 defines feasible solution set of the augmented LP (with nonnegativity of variables) Therefore, the final tableau (without x0) describes the same feasible solution set of the LP, i.e. feasible solutions to the equations have not been changed. Now, we can replace the objective function with the original one, and express it in the formal tableau form. -w - x0 = 0 0.2x1 + x3 +0.8x0 - 0.2x5 - 0.6x6 = 1.6 -0.6x1 + x2 +0.6x0 - 0.4x5 - 0.2x6 = 2.2 x1 -2x0 + x4 + x6 = 3 -z + x1 - x2 + x3 = 0 0.2x1 + x3 - 0.2x5 - 0.6x6 = 1.6 -0.6x1 + x2 - 0.4x5 - 0.2x6 = 2.2 x1 + x4 + x6 = 3 -z + 0.2x1 - 0.2x5 + 0.4x6 = 0.6 0.2x1 + x3 - 0.2x5 - 0.6x6 = 1.6 -0.6x1 + x2 - 0.4x5 - 0.2x6 = 2.2 x1 + x4 + x6 = 3 OR-1 2017

2 possible cases in phase one optimal solution Algorithm strategy in phase one: choose 𝑥 0 as leaving variable in case of ties in the minimum ratio test. 2 possible cases in phase one optimal solution 𝑤 nonzero ( 𝑤<0 ), 𝑥 0 basic  original problem is infeasible 𝑤=0, 𝑥 0 nonbasic  drop 𝑤, express original objective function 𝑧 in terms of nonbasic variables, continue the simplex method. (Note that 𝑤=0, 𝑥 0 basic can’t happen by our strategy) Similar idea can be used when the original LP is given in equality form and we do not have an initial basic feasible solution at hand. (Without replacing an equality with two inequalities, simplex method can be used directly to solve the equality form. More in Chapter 8.) OR-1 2017

(Fundamental theorem of LP) Every LP in standard form has the following properties. If no optimal solution  either unbounded or infeasible. If  feasible solution   a basic feasible solution. If  optimal solution   a basic optimal solution. OR-1 2017