1. Part 4 Nonlinear Programming
4.1 Introduction

2. Standard Form

3. An Intuitive Approach to Handle the Equality Constraints
One method of handling just one or two equality constraints is to solve for 1 or 2 variables and eliminate them from problem formulation by substitution.

4. Use of Lagrange Multipliers to Handle m Equality Constraints and m+n Variables

5. Equivalent Formulation

6. Choice of Decision Variables
For a given optimization problem, the choice of which variables to designate as the decision variables is not unique.
It is only a matter of convenience to make a distinction between decision and state variables.

7. First Derivation of Necessary Conditions (i)

9. First Derivation of Necessary Conditions (ii)

10. First Derivation of Necessary Conditions (iii)

11. Second Derivation of Necessary Conditions (i)

12. Second Derivation of Necessary Conditions (ii)

13. Second Derivation of Necessary Conditions (iii)

14. Second Derivation of Necessary Conditions (iv)

15. Second Derivation of Necessary Conditions - General Formulation

16. Derivation with Lagrange Multipliers

17. Example:
Solution:

20. Sensitivity Interpretation

21. Generalized Sensitivity

22. Problems with Inequality Constraints Only

23. One Constraint and One Variable

24. Two Possibilities at Minimum*

25. One Constraint and Two Variables
Area of improvement

26. J Inequality Constraints and N Variables

27. 2-D Case

28. Kuhn-Tucker Conditions: Geometrical Interpretation
At any local constrained optimum, no (small) allowable change in the problem variables can improve the value of the objective function.
lies within the cone generated by the negative gradients of the active constraints.

29. General Formulation

30. Active Constraints

31. Kuhn-Tucker Conditions

32. Kuhn-Tucker Necessity Theorem

37. Sensitivity

38. Constraint Qualification
When the constraint qualification is not met at the optimum, there may or may not exist a solution to the Kuhn-Tucker problem.
The Kuhn-Tucker necessity theorem helps to identify points that are not optimal. On the other hand, if the KTC are satisfied, there is no assurance that the solution is truly optimal.

39. Second-Order Optimality Conditions

40. Necessary and Sufficient Conditions for Optimality
If a Kuhn-Tucker point satisfies the second-order sufficient conditions, then optimality is guaranteed.

41. Basic Idea of Penalty Methods

42. Example

45. Exact L1 Penalty Function

46. Equivalent Smooth Constrained Problem

47. Barrier Method

49. Generalized Case