1. L12 LaGrange Multiplier Method Homework Review Summary Test 1

2. Constrained Optimization LaGrange Multiplier Method 2 Remember: 1.Standard form 2.Max problems f(x) = - F(x)

3. KKT Necessary Conditions for Min 3 Regularity check - gradients of active inequality constraints are linearly independent

4. Prob 4.122 4

5. KKT Necessary Conditions 5

6. Case 1 6

7. Case 2 7

8. Case 2 cont’d, find multipliers 8

9. Case 2 cont’d, regular pt? 9 Regular Pt? 1. pt feasible, YES 2. active constraint gradients independent Are active constraint gradients independent i.e. parallel? Determinant of Constraint gradients non-singular? Case 2 results in a KKT point!

10. Graphical Solution 10 2 1

11. Constraint Sensitivity 11 Note how relaxing h increases the feasible region but is in the wrong “direction.” Recall ν can be ±! Multiply h by -1, ah ha!

12. Sufficient Condition 12 Is this a convex programming problem? Check f(x) and constraints: From convexity theorems: 1. H f is PD 2. All constraints are linear Therefore KKT Pt is global Min!

13. True/False 13

14. LaGrange Multiplier Method May produce a KKT point A KKT point is a CANDIDATE minimum It may not be a local Min If a point fails KKT conditions, we cannot guarantee anything…. The point may still be a minimum. We need a SUFFICIENT condition 14

15. 15 Convex set: All pts in feasible region on a straight line(s). Convex sets Non-convex set Pts on line are not in feasible region

16. 16 Multiple variables Fig 4.21 What if it were an equality constraint? misprint

17. 17. Figure 4.22 Convex function f(x)=x 2 Bowl that holds water.

18. 18 Fig 4.23 Convex function.

19. Test for Convex Function 19 Difficult to use above definition! However, Thm 4.8 pg 163: If the Hessian matrix of the function is PD ro PSD at all points in the set S, then it is convex. PD… “strictly” convex, otherwise PSD… “convex”

20. Theorem 4.9 20 Given: S is convex if: 1. h i are linear 2. g j are convex i.e. H g PD or PSD When f(x) and S are convex= “convex programming problem”

21. “Sufficient” Theorem 4.10, pg 165 21 The first-order KKT conditions are Necessary and Sufficient for a GLOBAL minimum….if: 1. f(x) is convex H f (x) Positive definite 2. x is defined as a convex feasible set S Equality constraints must be linear Inequality constraints must be convex HINT: linear functions are convex!

22. Summary LaGrange multipliers are the instantaneous rate of change in f(x) w.r.t. relaxing a constraint. Equality constraints may need tightening rather than loosening Convex sets assure contiguity and or the smoothness of f(x) KKT pt of a convex programming problem is a GLOBAL MINIMUM! 22