1. Nonlinear Programming

2.  A nonlinear program (NLP) is similar to a linear program in that it is composed of an objective function, general constraints, and variable bounds.  A nonlinear program includes at least one nonlinear function, which could be the objective function, or some or all of the constraints.  Z = x 1 2 + 1/x 2  Many real systems are inherently nonlinear  Unfortunately, nonlinear models are much more difficult to optimize

3. Maximize Z = f(x) Subject to. g i (x) ≤ b 1 x i ≥ 0  General Form of Nonlinear  No single algorithm will solve every specific problem  Different algorithms are used for different types of problems

4.  Wyndor Glass example with nonlinear constraint Maximize Z = 3x 1 + 5x 2 Subject to x 1 ≤ 4 9x 1 2 + 5x 2 2 ≤ 216 x 1, x 2 ≥ 0  The optimal solution is no longer a CPF anymore, but it still lies on the boundary of the feasible region.  We no longer have the tremendous simplification used in LP of limiting the search for an optimal solution to just the CPF solutions.

5.  Wyndor Glass example with nonlinear objective function Maximize Z = 126x 1 – 9x 1 2 + 182x 2 – 13 x 2 2 Subject to x 1 ≤ 4 x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 x 1, x 2 ≥ 0  The optimal solution is no longer a CPF anymore, but it still lies on the boundary of the feasible region.

6.  Wyndor Glass example with nonlinear objective function  The optimal solution lies inside the feasible region.  That means we need to look at the entire feasible region, not just the boundaries. Maximize Z = 54x 1 – 9x 1 2 + 78x 2 – 13 x 2 2 Subject to x 1 ≤ 4 x 2 ≤ 12 3x 1 + 2x 2 ≤ 18 x 1, x 2 ≥ 0

7.  Unlike linear programming, solution is often not on the boundary of the feasible solution space.  Cannot simply look at points on the solution space boundary, but must consider other points on the surface of the objective function.  This greatly complicates solution approaches.  Solution techniques can be very complex.

8.  Nonlinear programming problems come in many different shapes and forms  Unconstrained Optimization  Linearly Constrained Optimization  Quadratic Programming  Convex Programming  Separable Programming  Nonconvex Programming  Geometric Programming  Fractional Programming

9.  These problems have no constraints, so the objective is simply to maximize the objective function  Basic function types  Concave  Entire function is concave down  Convex  Entire function is concave up

10.  Basic calculus  Find the critical points  Unfortunately this may be difficult for many functions  Estimate the maximum  Bisection Method  Newton’s Method

13. 

14. 

15. Gradient Search procedure: Z = 2x 1 x 2 + 2x 2 – x 1 2 – 2x 2 2  f /  x 1 = 2x 2 – 2x 1  f /  x 2 = 2x 1 + 2 – 4x 2 Initialization: set x 1 *, x 2 * = 0. 1)Set x 1 = x 1 * + t(  f /  x 1 ) = 0 + t(2×0 – 2×0) = 0. Set x 2 = x 2 * + t(  f /  x 2 ) = 0 + t(2×0 + 2 – 4×0) = 2t. f (x 1, x 2 ) = (2)(0)(2t) +(2)(2t) – (0)(0) – (2)(2t)(2t) = 4t – 8t 2. 2)f ’ (x 1, x 2 ) = 4 – 16t. Let 4 – 16t = 0 then t* = ¼. 3)ReSet x 1 * = x 1 * + t (  f /  x 1 ) = 0 + ¼(2×0 – 2×0) = 0. x 2 * = x 2 * + t(  f /  x 2 ) = 0 + ¼(2×0 + 2 – 4×0) = ½. Stopping rule:  f /  x 1 = 1,  f /  x 2 = 0

16. Gradient Search procedure: Z = 2x 1 x 2 + 2x 2 – x 1 2 – 2x 2 2  f /  x 1 = 2x 2 – 2x 1  f /  x 2 = 2x 1 + 2 – 4x 2 Iteration 2: x 1 * = 0 x 2 * = ½. 1)Set x = (0,1/2) + t(1,0) = (t,1/2). f (t,1/2) = (2)(t)(1/2) +(2)(1/2) – (t)(t) – (2)(1/2)(1/2) = t – t 2 + ½. 2)f ’ (t,1/2) = 1 - 2t. Let 1 – 2t = 0 then t* = ½. 3)ReSet x* = (0,1/2) + ½ (1,0) = (½, ½). Stopping rule:  f /  x 1 = 0,  f /  x 2 = 1.

17. Gradient Search procedure: Z = 2x 1 x 2 + 2x 2 – x 1 2 – 2x 2 2  f /  x 1 = 2x 2 – 2x 1  f /  x 2 = 2x 1 + 2 – 4x 2 Continue for a few more iterations: Iteration 1: x* = (0, 0) Iteration 2: x* = (½, ½) Iteration 3: x* = (½, ¾) Iteration 4: x* = (¾, ⅞) Iteration 5: x* = ( ⅞, ⅞) Notice the value is converging toward x* = (1, 1) This is the optimal solution since the gradient is 0 f (1,1) = (0,0)

18.  Nonlinear objective function with linear constraints  Karush-Kuhn Tucker conditions  KKT conditions