Chapter 3. Pitfalls Initialization Ambiguity in an iteration Finite termination? Resolving ambiguity in each iteration Selection of the entering variable: Any nonbasic variable with can enter basis. (However, performance of the algorithm depends on the choice of the entering variable. examples of the rules : largest coefficient rule, maximum increase rule, steepest ascent rule, …) OR-1 2009

Selection of leaving variable: No restriction in minimum ratio test : can increase the value of the entering variable indefinitely while satisfying nonnegativity, hence problem is unbounded Ex) Increase x3 while keeping x1 and x4 at 0, then direction of movement should be d = ( 0, 2, 1, 0, 0 )  0 so that the new point satisfies the equations. Hence, new point x0+d  0 satisfies nonnegativity for any   0. Also objective value increases by 1 ( 1 is coefficient of x3 in z row). OR-1 2009

Also note that Note that if we want to move from x0 to x0 + d, >0 , then we must have A(x0 + d)=b so that the new point satisfies the equations Ax=b. Then A(x0 + d)=b, >0 holds if and only if Ad=0. So the direction of movement d must satisfy Ad=0 for the d vector to be a feasible direction of movement ( d is in the null space of A). OR-1 2009

b) In case of ties in the minimum ratio test : Basic variables have value 0 after pivot. OR-1 2009

Later, we will examine what this means in geometry. In the next iteration, ( x1 entering, x5 leaving ) Though x1 cannot be increased, we perform the pivot as usual, making x1 as basic and x5 as nonbasic. No change in solution, but basic and nonbasic status changes for two variables. Later, we will examine what this means in geometry. OR-1 2009

Terminology: degenerate solution (퇴화해) : basic feasible solution with one or more basic variables having 0 values. degenerate iteration (pivot) : simplex iteration that does not change the basic solution values (only basis changes). Observations: Given a nondegenerate b.f.s., we must have ties in the minimum ratio test so that we have a degenerate solution after pivot. A degenerate pivot occurs only if we have a degenerate solution, but the converse is not true (i.e. we may have a nondegenerate pivot although we have a degenerate solution). OR-1 2009

Geometric meaning of a degenerate iteration x2=0 x2=0 x5=0 x5=0 x1=0 x1=0 x6=0 x6=0 A A ( x1=0, x2=0 used) ( x2=0, x5=0 used) OR-1 2009

( provides 5 equalities together with Ax = b). (ex- continued) In the first dictionary, the point A is identified by using the three equations in Ax = b and setting x1  0, x2  0 at equalities. The nonnegativity constraints for nonbasic variables (here x1, x2) are used to indentify point A ( provides 5 equalities together with Ax = b). After pivot, the solution point A is not changed. But it is identified now by using x2  0, x5  0 at equalities together with Ax = b. Hence degenerate iteration changes the defining equations one at a time, but the solution point is not changed. OR-1 2009

Getting out of degenerate iterations: x2=0 x5=0 x1=0 x6=0 A x30 ( x3=0, x5=0 used) We changed the data in the previous dictionary a little bit. Now, x2 enters (takes value 1) and x3 leaves the basis, and it is a nondegenerate pivot. Geometrically, we still satisfy the equation x5 = 0, but do not need to satisfy x2 = 0. As we increase the value of x2 up to 1, x3 becomes 0, hence the equation x3 = 0 is now used (together with x5 = 0 and Ax=b) to define a new point. OR-1 2009

Degenerate pivot is the process of identifying the same point using different defining equations (nonnegativity constraints). If we are lucky enough to obtain defining equations that guides the moving direction, we move to a different point with a nondegenerate pivot. During the degenerate iterations, the algorithm stalls and it may hamper the performance of the algorithm. Such phenomenon is of practical concern and affects the performance of the algorithm (especially, for problems with some special structures and large problems). Later, we will see how to get out of degenerate iterations systematically. OR-1 2009

If we have a degenerate solution, pivot may continue indefinitely (Example in text p.31, pivoting rule is largest coefficient for entering variable and smallest subscript for leaving in case of ties. Then we have the initial dictionary again after 6 pivots.) Terminology: Cycling : appearance of the same dictionary again in the simplex iterations. Cycling is the only reason that simplex method may fail to terminate (i.e. simplex method terminates in a finite number of iterations as long as cycling is avoided). OR-1 2009

Thm) If simplex method fails to terminate, it must cycle. Pf) Number of ways to choose basic variables are finite. Hence, if simplex fails to terminate, same basis must appear again. Show same basis  same dictionary, Hence same dictionary appears, i.e. cycles. (1) and (2) have the same set of solutions (ignore nonnegativity constraints) Let Consider a solution to (1) given as the following: OR-1 2009

This solution satisfies (1). So it also satisfies (2), hence These equalities must hold for any real number t. Therefore, we have We can use any nonbasic variable in the above proof, so dictionaries are identical.  The theorem can be proved easily if we use matrices, but the above proof uses only algebraic arguments. OR-1 2009

The proof of the theorem shows that if we have the same basis, then the dictionaries (tableaus) are the same. Since there are only a finite number of ways to choose the basis, the simplex method terminates finitely if the same basis (the same dictionary) does not appear again, i.e. cycling is avoided. OR-1 2009

Avoid cycling Perturbation method, Lexicographic method Smallest-subscript rule (Bland’s rule) Smallest-subscript rule (Bland’s rule) : Choose the variable having the smallest index among possible candidates as the entering variable. Also, if ties occur while choosing the leaving variable, select the smallest indexed variable among the candidates as the leaving variable. See the proof in the text if time permits. OR-1 2009

Examples of smallest subscript rule: Ties (=1/2) x3 enters, x4 leaves. Ties (=0) x1 enters, x5 leaves. OR-1 2009

Perturbation method, lexicographic method Idea: Avoid the appearance of degenerate solution since it is a precondition of cycling ( If the solution is not degenerate, the objective function value increases strictly in the next iteration, hence the same basis does not appear again because the objective function value never decreases during the simplex iterations). So add very small positive  to the r.h.s of equations so that the solution values are unchanged practically but degeneracy is avoided. But the ’s added to the r.h.s may cancel out each other during elementary row operations, again causing degeneracy. Remedy: Add different values of i’s to different r.h.s. so that cancellation does not occur. OR-1 2009

Add i to the i-th r.h.s., i = 1, …, m with the following property. Then, it can be shown that the values of the basic variables never become 0 in subsequent simplex iterations, hence no cycling occurs. (In practice, precision can cause problems.) (In actual implementation, 1= , 2= 2, 3= 3, … may be used or random numbers in [0, ] for some fixed small  are used. ) We may be concerned about the correctness of the optimal solution, but in practice the revised simplex method is usually used instead of dictionary. It has the capability to recover the correct solution values if the current basis is known (Chapter 7). Then we can restart the algorithm again without adding i’s, once we escape degenerate iterations. Hence the effect of adding i’s to the r.h.s can be eliminated. OR-1 2009

Perturbation method usually refers to the methods actually adding small i‘s to the right hand sides during simplex iterations. For lexicographic method, the idea is the same as the perturbation method. But we do not actually add i‘s. However, we perform the pivots as if i‘s are present. OR-1 2009

Lexicographic ordering of numbers : Consider r = r0 + r1 1 + …. + rm m, s = s0 + s1 1 + …. + sm m If r  s, there is the smallest subscript k such that rk  sk . We say that r is lexicographically smaller than s if rk < sk . (Similar terminology is used for vectors too) Then if r is smaller than s if r is lexicographically smaller than s. Ex) r = 2 + 211 + 19 2 + 200003 s = 2 + 211 + 202 + 203 + 154 + 145 r is lexicographically smaller than s. OR-1 2009

Lexicographic method : Smart implementation of the perturbation method Lexicographic method : Smart implementation of the perturbation method. We do not actually add epsilons to the r.h.s., but epsilons are treated as symbols (like variables) denoting very small positive numbers. Moreover, during the simplex iterations, the coefficients of epsilons can be read from the simplex tableau directly without actually expressing the epsilon variables in the tableau Note that i in the above form may be regarded as additional variables in the equations like x. Any solution x,  (and z) need to satisfy the above equations. OR-1 2009

(after two iterations) Lexicographic method (0+21 ) (0+22 ) (1+3 ) (after two iterations) OR-1 2009

Note that the coefficient matrix for the basic variables x5, x6, x7 and the coefficient matrix for 1, 2, 3 are the same identity matrices in the beginning of the lexicographic method. Since we use the elementary row operations in the simplex iterations, those two coefficient matrices have the same elements in the following iterations. Hence, we can read the coefficients of 1, 2, 3 from the coefficients of x5, x6, x7 . So we do not actually need to add 1, 2, 3 to the tableau. Usually, lexicographic method means we read coefficients of i’s from the coefficients of corresponding x variables. Hence, in the example, we actually do not add i’s. We first perform ratio test using the entering column and the r.h.s. If ties occur, then perform ratio test (for tied rows) again using the entering column and the column for x5, etc. OR-1 2009

Can cycling be prevented? Since the elementary row operation matrices are nonsingular, the two matrices for  variables and corresponding x variables remain nonsingular after applying elementary row operations (elementary row operation is equivalent to premultiplying the corresponding nonsingular matrices. Since the matrix for  variables is identity matrix initially, it is nonsingular. Hence we also obtain nonsingular matrix). In other words, no row with all 0 elements appear (in the -matrix). Hence the values of basic variables never become 0. (conceptually, after adding small i values.) OR-1 2009

Simplex pivot rule guarantees that all basic variables are nonnegative at any time (treating i’s as small positive values). Hence, combined with above, it guarantees that the values of basic variables are always positive (conceptually). So strict increase of the objective value is guaranteed. It guarantees that the same tableau (or dictionary) will not appear again. We can read the real solution value by ignoring the  terms in the current dictionary. It is also observed that the lexicographic method can be started and stopped at any time during the simplex iterations. We just conceptually add or drop the  terms. See the text for more rigorous proof. It proves that rows with all 0 elements never appear in -matrix, hence no solution will have 0 value, i.e. no degeneracy occurs in subsequent iterations. OR-1 2009

Initialization (two-phase method) We need an initial b.f.s to start the simplex method. If we have bi < 0 for some constraint i, the slack variable xn+i = bi < 0 is not feasible. Consider easily obtained solution xi = 0 for all i in (1) and then subtract some positive number x0 from all l.h.s. so that it becomes feasible solution to (1), i.e. xi = 0 for all i, x0 = Now, if we can find a feasible solution to (1) with x0 = 0, it is a feasible solution to (1) using original variables. subject to (1) OR-1 2009

Hence solve the following problem using the easily obtained initial feasible solution and simplex method to find an optimal solution with x0 = 0. Note that x0 is a nonnegative variable. (1) has a feasible solution  (2) has an optimal solution with optimal value 0 (x0 = 0) If we find an optimal solution with x0 = 0, we can obtain a feasible solution to (1) by disregarding x0. One point to be careful is that we need a b.f.s. to perform the simplex method. subject to (2) OR-1 2009

However a feasible dictionary can be easily obtained by one pivot. Example We cannot perform simplex iteration in this dictionary since the basic solution is not feasible (nonnegativity violated) However a feasible dictionary can be easily obtained by one pivot. OR-1 2009

Current basic solution is not feasible Current basic solution is not feasible. Let x0 enter basis and the slack variable with most negative value leaves the basis (It is not a simplex iteration. Just perform the pivot, not considering obj. value. It does not change the solution set.) Perform simplex method. After two iterations, we get the optimal dictionary OR-1 2009

We obtained optimal solution with value 0 We obtained optimal solution with value 0. Hence the current optimal solution gives a b.f.s. to the original problem. Drop x0 (no more needed) and replace the objective function with the original one. z row is used to read the objective value of a given solution, hence it can be added or dropped without affecting the feasible solution set to the LP. Note that the current b.f.s. is a b.f.s. to the original problem. OR-1 2009

Now, restart the simplex method with the current dictionary Express it in dictionary form (only nonbasic variables appear in the r.h.s.) by substituting the basic variables in the objective function. Now, restart the simplex method with the current dictionary OR-1 2009

2 possible cases in phase one optimal solution Algorithm strategy in phase one : choose x0 as leaving variable in case of ties in the minimum ratio test. 2 possible cases in phase one optimal solution w nonzero ( w < 0 ), x0 basic  original problem is infeasible w = 0 , x0 nonbasic  drop w, express original objective function z in terms of nonbasic variables, continue the simplex method. (Note that w = 0, x0 basic can’t happen by our strategy) Similar idea can be used when the original LP is given in equality form. (Without converting a equality using two inequalities, simplex method can be used directly to solve the equality form. More in Chapter 8.) OR-1 2009

(Fundamental theorem of LP) Every LP in standard form has the following properties. No optimal solution  either unbounded or infeasible  feasible solution   a basic feasible solution  optimal solution   a basic optimal solution OR-1 2009