1. Part 4 Nonlinear Programming
4.1 Introduction

2. Standard Form

3. An Intuitive Approach to Handle the Equality Constraints
One method of handling just one or two equality constraints is to solve for 1 or 2 variables and eliminate them from problem formulation by substitution.

4. Use of Lagrange Multipliers to Handle n Equality Constraints and m+n Variables

5. Equivalent Formulation

6. Choice of Decision Variables
For a given optimization problem, the choice of which variables to designate as the decision (control) variables is not unique.
It is only a matter of convenience to make a distinction between decision and state variables.

7. 1st Derivation of Necessary Conditions (i)

9. 1st Derivation of Necessary Conditions (ii)

10. 1st Derivation of Necessary Conditions (iii)

11. 2nd Derivation of Necessary Conditions (i)

12. 2nd Derivation of Necessary Conditions (ii)

13. 2nd Derivation of Necessary Conditions (iii)

14. 2nd Derivation of Necessary Conditions (iv)

15. 2nd Derivation of Necessary Conditions - General Formulation

16. 3rd Derivation with Lagrange Multipliers

17. Example:
Solution:

21. Sensitivity Interpretation

22. Generalized Sensitivity

23. Problems with Inequality Constraints Only

25. Two Scenarios at Minimum*

26. Area of improvement exists if
Eq (14) is not satisfied.

27. minimum

28. J Inequality Constraints and N Variables

29. Geometrical Interpretation
At any local constrained optimum, no (small) allowable change in the problem variables can improve the value of the objective function.
lies within the cone formed by the negative gradients of the active constraints.

30. General Formulation

31. Active Constraints

32. Kuhn-Tucker Conditions

33. Kuhn-Tucker Necessity Theorem

34. Remarks
The Kuhn-Tucker necessity theorem helps to identify points that are not optimal.
If the KTC are satisfied, there is no assurance that the solution is truly optimal.

37. y1 y2

39. Sensitivity

40. Constraint Qualification
When the constraint qualification is not met at the optimum, there may or may not exist a solution to the Kuhn-Tucker problem.

41. Second-Order Optimality Conditions

42. Necessary and Sufficient Conditions for Optimality
If a Kuhn-Tucker point satisfies the second-order sufficient conditions, then optimality is guaranteed.

43. Basic Idea of Penalty Methods

44. Example

47. Exact L1 Penalty Function

48. Equivalent Smooth Constrained Problem

49. Barrier Method

51. Generalized Cases

52. Mixed Penalty-Barrier Method