CS5321 Numerical Optimization 14 Linear Programming: The Interior Point Method 7/15/2019

Interior point methods for LP There are many variations of IPM for LP Primal IPM, dual IPM, primal-dual IPM Potential Reduction Methods Path Following Methods Short-step methods Long-step methods Predictor-corrector methods We will cover the path following method and the long-step method 7/15/2019

The dual problem of LP minxz=cTx subject to Ax = b, x  0 The dual problem is (see chap 12) maxbT subject to AT  +s = c, s  0 Weak duality For any x and  that are feasible in the primal and the dual problems, bT  cTx. Strong duality If x* and * are solutions to the primal and the dual problems, bT* = cTx*. 7/15/2019

Optimality conditions of LP minxz=cTx subject to Ax = b, x  0 The Lagrangian function L(x,,s) = cTx − T(Ax−b) − sTx The optimality conditions (KKT, see chap 12) The last one is the complementarity condition A T ¸ + s = c x b ; i 7/15/2019

Primal-dual method The optimality conditions can be expressed as X=diag(x), S=diag(s) and e=(1,1,…,1)T. F is a nonlinear equation: R2n+m  R2n+m. Solved by the Newton’s method: (J the Jacobian of F.) (x, , s)+=(x, , s), F ( x ; ¸ s ) = @ A T + ¡ c b X S e 1 f o r J ( x ; ¸ s ) @ ¢ 1 A = ¡ F 7/15/2019

Jacobian of F Let . The Jacobian J (x, , s) of F(x, , s) is F ( x ; ¸ s ) = @ A T + ¡ c b X S e 1 r J = @ r x c ¸ s b X S 1 A T I 7/15/2019

Solving the linear systems The Newton’s direction is obtained by solving S is an artificial variable. Let The left hand side is symmetric, but indefinite. @ A T I S X 1 ¢ x ¸ s = ¡ r c b D = S ¡ 1 2 X µ ¡ D 2 A T ¶ ¢ x ¸ = r c + X 1 S b ¢ s = ¡ X 1 r S x 7/15/2019

Centering In practice, rXS=XSe −e rather than rXS=XSe. Duality measure: Centering parameter   [0,1]: Decreasing as approaching solutions This is related to the barrier method (chap 15) ¹ = 1 n X i x s T Centered direction Without centering 7/15/2019

The path following algorithm Given (x0,0,s0) with (x0,s0)>0 For k = 0, 1, 2 …until (xk)Tsk<  Choose k[0,1] and let k=(xk)Tsk/n. Solve the Newton’s direction (xk, k, sk) Set (xk+1, k+1, sk+1)=(xk, k, sk) + k(xk, k, sk), where k is chosen to make (xk+1,sk+1)>0 7/15/2019

Central path and neighborhood Feasible set F0={(x,,s) | Ax=b, AT+s=c, (x,s)>0} Point (x,,s)F0 is on the central path C if x(i)s(i)=  for all i=1,2,…,n. An example of the neighborhood of the central path  is the duality measure (two slides before) Typically =10−3. 7/15/2019

Long-step path-following Given (x0,0,s0)N− and , min, max. For k = 0, 1, 2 …until (xk)Tsk<  Choose k[min,max] and let k=(xk)Tsk/n. Solve the Newton’s direction (xk, k, sk) Set (xk+1, k+1, sk+1)=(xk, k, sk) + k(xk, k, sk), where k is chosen to make (xk+1, k+1, sk+1)N−. 7/15/2019

Complexity The long step following method can converge in O(nlog(1/)) iterations (Theorem 14.4) The most effective interior point method for LP is the predictor-corrector algorithm (Mehrotra 1992) 7/15/2019