L12 LaGrange Multiplier Method Homework Review Summary Test 1

Constrained Optimization LaGrange Multiplier Method 2 Remember: 1.Standard form 2.Max problems f(x) = - F(x)

KKT Necessary Conditions for Min 3 Regularity check - gradients of active inequality constraints are linearly independent

Prob

KKT Necessary Conditions 5

Case 1 6

Case 2 7

Case 2 cont’d, find multipliers 8

Case 2 cont’d, regular pt? 9 Regular Pt? 1. pt feasible, YES 2. active constraint gradients independent Are active constraint gradients independent i.e. parallel? Determinant of Constraint gradients non-singular? Case 2 results in a KKT point!

Graphical Solution

Constraint Sensitivity 11 Note how relaxing h increases the feasible region but is in the wrong “direction.” Recall ν can be ±! Multiply h by -1, ah ha!

Sufficient Condition 12 Is this a convex programming problem? Check f(x) and constraints: From convexity theorems: 1. H f is PD 2. All constraints are linear Therefore KKT Pt is global Min!

True/False 13

LaGrange Multiplier Method May produce a KKT point A KKT point is a CANDIDATE minimum It may not be a local Min If a point fails KKT conditions, we cannot guarantee anything…. The point may still be a minimum. We need a SUFFICIENT condition 14

15 Convex set: All pts in feasible region on a straight line(s). Convex sets Non-convex set Pts on line are not in feasible region

16 Multiple variables Fig 4.21 What if it were an equality constraint? misprint

17. Figure 4.22 Convex function f(x)=x 2 Bowl that holds water.

18 Fig 4.23 Convex function.

Test for Convex Function 19 Difficult to use above definition! However, Thm 4.8 pg 163: If the Hessian matrix of the function is PD ro PSD at all points in the set S, then it is convex. PD… “strictly” convex, otherwise PSD… “convex”

Theorem Given: S is convex if: 1. h i are linear 2. g j are convex i.e. H g PD or PSD When f(x) and S are convex= “convex programming problem”

“Sufficient” Theorem 4.10, pg The first-order KKT conditions are Necessary and Sufficient for a GLOBAL minimum….if: 1. f(x) is convex H f (x) Positive definite 2. x is defined as a convex feasible set S Equality constraints must be linear Inequality constraints must be convex HINT: linear functions are convex!

Summary LaGrange multipliers are the instantaneous rate of change in f(x) w.r.t. relaxing a constraint. Equality constraints may need tightening rather than loosening Convex sets assure contiguity and or the smoothness of f(x) KKT pt of a convex programming problem is a GLOBAL MINIMUM! 22