OR II GSLM 52800 1

Outline inequality constraint 2

NLP with Inequality Constraints min f(x) s.t. hi(x) = 0 for i = 1, …, p gj(x)  0 for j = 1, …, m in matrix form h(x) = 0 g(x) = 0 3

Inequality constraint gj(x)  0 is binding at point x0 if gj(x0) = 0 Binding Constraints Inequality constraint gj(x)  0 is binding at point x0 if gj(x0) = 0 =-constraint: always binding 4

Regular Points K: the set of binding (inequality) constraints x* subject to h(x) = 0 & g(x)  0 is a regular point if Thi(x*), i = 1, …, p, & Tgj(x*), j  K, are linearly independent 5

FONC for Inequality Constraints (Karush-Kuhn-Tucker Necessary Conditions; KKT Conditions) for a regular point x* to be a local min, there exist *  p and *  m such that In Matrix Form: f(x*) + *Th(x*) + * Tg(x*) = 0 (stationary condition; dual feasibility) h(x*) = 0, g(x*)  0 (primal feasibility) * Tg(x*) = 0 (Complementary slackness conditions) *  0 (Non-negativity of the dual variables) 6

The Convex Cone Formed by Vectors v1 = (1, 0), v2 = (0, 1) {(x, y)|v1+v2 , ,   0} The convex cone formed by (1, 0) and (0, 1): {(x, y)|v1+v2 , ,   0} v1 v2 (1, 0) (0, 1) The convex cone formed by v1 and v2: {(x, y)|v1+v2 , ,   0} 7

Geometric Interpretation of the KKT Condition constraint 1: g1(x)  0 constraint 2: g2(x)  0 g2(x) = 0 g2(x0) g1(x) = 0 g1(x0) f (x0) f (x0) x0 x0 cannot be a minimum 8

Geometric Interpretation of the KKT Condition constraint 2: g2(x)  0 constraint 1: g1(x)  0 g1(x) = 0 g2(x) = 0 g2(x0) g1(x0) x0 f (x0) f (x0) x0 cannot be a minimum 9

Geometric Interpretation of the KKT Condition f (x0) f (x0) region decreasing in f constraint 2: g2(x)  0 constraint 1: g1(x)  0 g1(x) = 0 g2(x) = 0 g2(x0) g1(x0) x0 x0 cannot be a minimum 10

Geometric Interpretation of the KKT Condition region decreasing in f f (x0) f (x0) constraint 2: g2(x)  0 constraint 1: g1(x)  0 g1(x) = 0 g2(x) = 0 g2(x0) g1(x0) x0 x0 cannot be a minimum 11

Geometric Interpretation of the KKT Condition f (x0) f (x0) region decreasing in f constraint 2: g2(x)  0 constraint 1: g1(x)  0 g1(x) = 0 g2(x) = 0 g2(x0) g1(x0) x0 x0 can be a minimum 12

A Necessary Condition for x0 to be a Minimum when f in the convex cone of g1 and g2 f(x0) = 1g1(x0) + 2g2(x0), i.e., f(x0) + 1g1(x0) + 2g2(x0) = 0 g1(x) = 0 g2(x) = 0 g2(x0) g1(x0) x0 f (x0) f (x0) 13

Effect of Convexity convex objective function, convex feasible region set  the KKT condition  necessary & sufficient convex gj & linear hi 14

Example 10.11 of JB KKT conditions 15

Example 10.11 of JB Tf(x) = (4(x1+1), 6(x24)) Tg1(x) = (2x1, 2x2) possibilities 2 = 0 > 0 1 16

Example 10.11 of JB case 1 = 0, 2 = 0 (both constraints not binding) x1 = -1, x2 = 4, violating 17

Example 10.11 of JB case 1 > 0, 2 > 0 (both constraints binding) solving g2(x0) g1(x0) f (x0) g2(x0) g1(x0) f (x0) 18

Example 10.11 of JB case 1 = 0, 2 > 0 (the second constraint binding) solving, 2 < 0 19

Example 10.11 of JB case 1 > 0, 2 = 0 (the first constraint binding) solving, 20

KKT Condition with Non-negativity Constraints min f(x), s.t. gi(x)  0, i = 1, …, m, x  0 21

KKT Condition with Non-negativity Constraints define L(x, ) = f(x) + Tg(x) 22

KKT Condition with Non-negativity Constraints define L(x, ) = f(x) + 1Tg(x) - 2Tx 23

Example 10.12 of JB L(x, ) = KKT condition (a) 2x1 8 +   0, 8x2 16 +   0 (b) x1+ x2 5  0 (c) x1(2x1 8 + ) = 0, x2(8x2 16 + ) = 0 (d) (x1+ x2 5) = 0 (e) x1 0, x2 0,  0   24

Example 10.12 of JB eight cases, depending on whether x1, x2, and = 0 or > 0 on checking, the case that x1+x2  5 is binding, x1 > 0, x2 > 0  x = (3.2, 1.8) and  = 1.6 satisfying the KKT condition regular point convex objective with linear constraint  KKT being sufficient 25