Page 1 Page 1 ENGINEERING OPTIMIZATION Methods and Applications A. Ravindran, K. M. Ragsdell, G. V. Reklaitis Book Review

Page 2 Page 2 Chapter 5: Constrained Optimality Criteria Part 1: Ferhat Dikbiyik Part 2:Yi Zhang Review Session July 2, 2010

Page 3 Page 3 Constraints: Good guys or bad guys?

Page 4 Page 4 Constraints: Good guys or bad guys? reduces the region in which we search for optimum.

Page 5 Page 5 Constraints: Good guys or bad guys? makes optimization process very complicated

Page 6 Page 6

Page 7 Page 7 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 8 Page 8 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 9 Page 9 Equality-Constrained Problems solving the problem as an unconstrained problem by explicitly eliminating K independent variables using the equality constraints GOAL

Page 10 Page 10 Example 5.1

Page 11 Page 11 What if?

Page 12 Page 12 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 13 Page 13 Lagrange Multipliers Lagrange Multipliers Converting constrained problem to an unconstrained problem with help of certain unspecified parameters known as Lagrange Multipliers

Page 14 Page 14 Lagrange Multipliers Lagrange function

Page 15 Page 15 Lagrange Multipliers Lagrange multiplier

Page 16 Page 16 Example 5.2

Page 17 Page 17

Page 18 Page 18 Test whether the stationary point corresponds to a minimum positive definite

Page 19 Page 19

Page 20 Page 20 Example 5.3

Page 21 Page 21

Page 22 Page 22

Page 23 Page 23 positive definite negative definite max

Page 24 Page 24 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 25 Page 25 Economic Interpretation of Lagrange Multipliers The Lagrange multipliers have an important economic interpretation as shadow prices of the constraints, and their optimal values are very useful in sensitivity analysis.

Page 26 Page 26 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 27 Page 27 Kuhn-Tucker Conditions

Page 28 Page 28 NLP problem

Page 29 Page 29 Kuhn-Tucker conditions (aka Kuhn-Tucker Problem)

Page 30 Page 30 Example 5.4

Page 31 Page 31 Example 5.4

Page 32 Page 32 Example 5.4

Page 33 Page 33 Outline of Part 1 Equality-Constrained Problems Lagrange Multipliers Economic Interpretation of Lagrange Multipliers Kuhn-Tucker Conditions Kuhn-Tucker Theorem

Page 34 Page 34 Kuhn-Tucker Theorems 1.Kuhn – Tucker Necessity Theorem 2.Kuhn – Tucker Sufficient Theorem

Page 35 Page 35 Kuhn-Tucker Necessity Theorem Let f, g, and h be differentiable functions x* be a feasible solution to the NLP problem. and for k=1,….,K are linearly independent

Page 36 Page 36 Kuhn-Tucker Necessity Theorem Let f, g, and h be differentiable functions x* be a feasible solution to the NLP problem. and for k=1,….,K are linearly independent at the optimum If x* is an optimal solution to the NLP problem, then there exists a (u*, v*) such that (x*,u*, v*) solves the KTP given by KTC. Constraint qualification ! Hard to verify, since it requires that the optimum solution be known beforehand !

Page 37 Page 37 Kuhn-Tucker Necessity Theorem For certain special NLP problems, the constraint qualification is satisfied: 1.When all the inequality and equality constraints are linear 2.When all the inequality constraints are concave functions and equality constraints are linear ! When the constraint qualification is not met at the optimum, there may not exist a solution to the KTP

Page 38 Page 38 Example 5.5 x* = (1, 0) and for k=1,….,K are linearly independent at the optimum

Page 39 Page 39 Example 5.5 x* = (1, 0) No Kuhn-Tucker point at the optimum

Page 40 Page 40 Kuhn-Tucker Necessity Theorem Given a feasible point that satisfies the constraint qualification If it does not satisfy the KTCs not optimal If it does satisfy the KTCs optimal

Page 41 Page 41 Example 5.6

Page 42 Page 42 Kuhn-Tucker Sufficiency Theorem Let f(x) be convex the inequality constraints g j (x) for j=1,…,J be all concave function the equality constraints h k (x) for k=1,…,K be linear If there exists a solution (x*,u*,v*) that satisfies KTCs, then x* is an optimal solution

Page 43 Page 43 Example 5.4 f(x) be convex the inequality constraints g j (x) for j=1,…,J be all concave function the equality constraints h k (x) for k=1,…,K be linear

Page 44 Page 44 Example 5.4 f(x) be convex semi-definite

Page 45 Page 45 Example 5.4 f(x) be convex the inequality constraints g j (x) for j=1,…,J be all concave function v g 1 (x) linear, hence both convex and concave negative definite

Page 46 Page 46 Example 5.4 f(x) be convex the inequality constraints g j (x) for j=1,…,J be all concave function the equality constraints h k (x) for k=1,…,K be linear v

Page 47 Page 47 Remarks For practical problems, the constraint qualification will generally hold. If the functions are differentiable, a Kuhn–Tucker point is a possible candidate for the optimum. Hence, many of the NLP methods attempt to converge to a Kuhn– Tucker point.

Page 48 Page 48 Remarks When the sufficiency conditions of Theorem 5.2 hold, a Kuhn–Tucker point automatically becomes the global minimum. Unfortunately, the sufficiency conditions are difficult to verify, and often practical problems may not possess these nice properties. Note that the presence of one nonlinear equality constraint is enough to violate the assumptions of Theorem 5.2

Page 49 Page 49 Remarks The sufficiency conditions of Theorem 5.2 have been generalized further to nonconvex inequality constraints, nonconvex objectives, and nonlinear equality constraints. These use generalizations of convex functions such as quasi-convex and pseudoconvex functions