CS5321 Numerical Optimization 19 Interior-Point Methods for Nonlinear Programming (Barrier Methods) 2/22/2019

Problem formulation cE is a vector of ci, iE x f ( ) s . t c E = I ¡ ¸ cE is a vector of ci, iE cI is a vector of ci, iI s is the slack variables Add barrier functions for inequality constraints. m i n x f ( ) ¡ ¹ X = 1 l s . t c E I  is the barrier parameter, which may be reduced iteratively. 2/22/2019

KKT conditions The Lagrangian of the original problem is y and z are the Lagrangian multipliers. Let AE and AI denote the Jacobian of cE and cI. The KKT condition of the barrier function is L ( x ; s y z ) = f ¡ ¹ m X i 1 l n T c E I r f ( x ) ¡ A T E y I z = S ¹ e c s 2/22/2019

Line-search algorithm Solve KKT conditions by the Newton’s method The primal-dual system Update (x, s, y, z)+=(xpx,sps,ypy,zpz) and k+1 The fraction to the boundary rule: for (0,1) 2 6 4 r x L ¡ A T E I Z S 3 7 5 p s y z = f ¹ e c ® s = m a x f 2 ( ; 1 ] : + p ¸ ¡ ¿ ) g z 2/22/2019

Solving the primal-dual system Let=S−1Z and rewrite the primal dual system as The matrix become symmetric. Eliminate ps and reform the problem 2/22/2019

Avoid singularity Singularity may caused by (some elements goes to ) or ill-conditionness of xx2f and AI. Use projected Hessian and modified  Add diagonal shift 2 6 4 G A T E I ¡ 3 7 5 2/22/2019

Barrier param and step length Barrier parameters {k} need converge to zero. Static method k+1= kk for k(0,1) Adaptive methods In the linear programming, (chap 14) Use merit function to decide whether a step is productive and should be accepted. (chap 18) ¹ k + 1 = ¾ s T z m Á ( x ; s ) = f ¡ ¹ m X i 1 l n + º k c E I 2/22/2019

Trust-region SQP method Two differences from the line search approach Solve primal (x,s) and dual problem (y, z) alternately Use scaling S−1 on ps. The primal problem 2/22/2019

Solving the dual problem ^ A = · E I ¡ S ¸ Define . The dual problem is to solve Using the least-square method The solution cannot guarantee z to be positive.  replaced by a small positive number if zi0 ^ A T · y z ¸ = r f ( x ) ¡ ¹ e · y z ¸ = ( ^ A T ) ¡ 1 r f x ¹ e z i ! m n ( 1 ¡ 3 ; ¹ = s ) 2/22/2019

Scaling Let ps' = S−1ps. The problem can be rewritten as Parameter rE and rI can be computed by solving r E = A ( x ) v + c ; I ¡ S s m i n k 2 . t · : 8 ¢ ¸ V ¿ e 2/22/2019

Convergence theory Theorem 19.1: Let {xk} be the iterative points of the basic algorithm and {k}0. Suppose f and c are continuously differentiable. The all limit points x* of {xk} are feasible. Moreover, if any x* satisfies LICQ, the KKT conditions are satisfied. 2/22/2019