Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: LLagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints

Review of 1 st Derivatives Notation: yy = f(x), dy/dx = f’(x) f(x) = c f’(x) = 0 f(x) = x n f’(x) = n*x (n-1) f(x) = x f’(x) = 1*x 0 = 1 f(x) = x 5 f’(x) = 5*x 4 f(x) = 1/x 3 f(x) = x -3 f’(x) = -3*x 4 f(x) = c*g(x) f’(x) = c*g’(x) f(x) = 10*x 2 f’(x) = 20*x f(x) = u(x)+v(x) f’(x) = u’(x)+v’(x) f(x) = x 2 - 5x f’(x) = 2x - 5

Nonlinear Optimization Review of 2 nd Derivatives Notation: yy = f(x), d(f’(x))/dx = d 2 y/dx 2 = f’’(x) f(x) = -x 2 f’(x) = -2x f’’(x) = -2 f(x) = x -3 f’(x) = -3x -4 f’’(x) = 12x -5

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: LLagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints

Models with One Decision Variable Requires 1 st & 2 nd derivative tests

Nonlinear Optimization 1 st & 2 nd Derivative Tests Rule 1 (Necessary Condition): ddf/dx = 0 Rule 2 (Sufficient Condition): dd 2 f/dx 2 > 0Minimum dd 2 f/dx 2 < 0Maximum

Nonlinear Optimization Maximum Example Rule 1: ff(x) = y = x – 5x 2 ddy/dx = 100 – 10x = 0, x = 10 Rule 2: dd 2 y/dx 2 = -10 Therefore, since d 2 y/dx 2 < 0: f(x) has a Maximum at x=10

Nonlinear Optimization Maximum Example – Graph Solution

Nonlinear Optimization Minimum Example Rule 1: ff(x) = y = x 2 – 6x + 9 ddy/dx = 2x - 6 = 0, x = 3 Rule 2: dd 2y /dx 2 = 2 Therefore, since d 2 y/dx 2 > 0: f(x) has a Minimum at x=3.

Nonlinear Optimization Minimum Example – Graph Solution 3

Nonlinear Optimization Max & Min Example Rule 1: ff(x) = y = x 3 /3 – x 2 ddy/dx = f’(x) = x 2 – 2x = 0; x = 0, 2 Rule 2: dd 2 y/dx 2 = f’’(x) = 2x – 2 = 0 22(0) – 2 = -2, f’’(x=0) = -2 Therefore, d 2 y/dx 2 < 0: Maximum of f(x) at x=0 22(2) – 2 = 2, f’’(x=2) = 2 Therefore, d 2 y/dx 2 > 0: Minimum of f(x) at x=2

Nonlinear Optimization Max & Min Example – Graph Solution 2 0

Nonlinear Optimization Example: Cubic Cost Function Resulting in Quadratic 1 st Derivative Rule 1: ff(x) = C = 10x 3 – 200x 2 – 30x + 15,000 ddC/dx = f’(x)= 30x 2 – 400x – 30 = 0 Quadratic Form: ax 2 + bx + c

Nonlinear Optimization Rule 2: dd 2y /dx 2 = f’’(x) = 60x – 400 660(13.4) – 400 = 404 > 0 Therefore, d 2 y/dx 2 > 0: Minimum of f(x) at x = 13.4 660(-.07) – 400 = < 0 Therefore, d 2 y/dx 2 < 0: Maximum of f(x) at x = -.07

Cubic Cost Function – Graph Solution

Nonlinear Optimization Economic Order Quantity – EOQ Assumptions: DDemand for a particular item is known and constant RReorder time (time from when the order is placed until the shipment arrives) is also known TThe order is filled all at once, i.e. when the shipment arrives, it arrives all at once and in the quantity requested AAnnual cost of carrying the item in inventory is proportional to the value of the items in inventory OOrdering cost is fixed and constant, regardless of the size of the order

Nonlinear Optimization Economic Order Quantity – EOQ Variable Definitions: LLet Q represent the optimal order quantity, or the EOQ C h represent the annual carrying (or holding) cost per unit of inventory C o represent the fixed ordering costs per order D represent the number of units demanded annually

Nonlinear Optimization Economic Order Quantity – EOQ Note: If all the previous assumptions are satisfied, then the number of units in inventory would follow the pattern in the graph below: EOQ Model Q Time

Nonlinear Optimization Economic Order Quantity – EOQ At time = 0 after the initial delivery, the inventory level would be Q. The inventory level would then decline, following the straight line since demand is constant. When the inventory just reaches zero, the next delivery would occur (since delivery time is known and constant) and the inventory would instantaneously return to Q. This pattern would repeat throughout the year.

Nonlinear Optimization Economic Order Quantity – EOQ Under these assumptions: AAverage Inventory Level = Q/2 AAnnual Carrying (or Holding) Cost = (Q/2)*C h TThe annual ordering cost would be the number of orders times the ordering cost: (D/Q)* C o TTotal Annual Cost = TC = (Q/2)*C h +(D/Q)* C o

Nonlinear Optimization Economic Order Quantity – EOQ To find the Optimal Order Quantity, Q take the first derivative of TC with respect to Q: ((dTC/dQ) = (C h /2) – DC o Q -2 = 0 Solving this for Q, we find: QQ * = (2DC o /C h )^(1/2) Which is the Optimal Order Quaintly Checking the second-order conditions (Rule 2 in our text), we have: ((d 2 TC/dQ 2 )= (2DC o /Q 3) Which is always > 0, since all the quantities in the expression are positive. Therefore, Q * gives a minimum value for total cost (TC)

Nonlinear Optimization Restricted Interval Problems Step 1: FFind all the points that satisfy rules 1 & 2. These are candidates for yielding the optimal solution to the problem. Step 2: IIf the optimal solution is restricted to a specified interval, evaluate the function at the end points of the interval. Step 3: CCompare the values of the function at all the points found in steps 1 and 2. The largest of these is the global maximum solution; the smallest is the global minimum solution.

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: LLagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints

Unconstrained Models with More Than One Decision Variable Requires partial derivatives

Nonlinear Optimization Example Partial Derivatives If z = 3x 2 y 3 ∂∂z/∂x = 6xy 3 ∂∂z/∂y = 9y 2 x 2 If z = 5x 3 – 3x 2 y 2 + 7y 5 ∂∂z/∂x = 15x 2 – 6xy 2 ∂∂z/∂y = -6x 2 y + 35y 4

Nonlinear Optimization 2 nd Partial Derivatives 2 nd Partials ((∂/∂x) (∂z/∂x) = ∂ 2 z/∂x 2 ((∂/dy) (∂z/∂y) = ∂ 2 z/∂y 2 Mixed Partials ((∂/∂x) (∂z/∂y) = ∂ 2 z/(∂x∂y) ((∂/∂y) (∂z/∂x) = ∂ 2 z/(∂y∂x)

Nonlinear Optimization Example 2 nd Partial Derivatives If z = 7x 3 + 9xy 2 + 2y 5 ∂∂z/∂x = 21x 2 + 9y 2 ∂∂z/∂y = 18xy + 10y 4 ∂∂ 2 z/(∂y∂x) = 18y ∂∂ 2 z/(∂x∂y) = 18y ∂∂ 2 z/∂x 2 = 42x ∂∂ 2 z/∂y 2 = 40y 3

Nonlinear Optimization Partial Derivative Tests Rule 3 (Necessary Condition): ∂∂f/∂x 1 = 0, ∂f/∂x 2 = 0, Solve Simultaneously Rule 4 (Sufficient Condition): IIf ∂ 2 f/∂x 1 2 > 0 AAnd (∂ 2 f/∂x 1 2 )*(∂ 2 f/∂x 2 2 ) – (∂ 2 f/(∂x 1 ∂x 2) ) 2 > 0 Then Minimum IIf ∂ 2 f/∂x 1 2 < 0 AAnd (∂ 2 f/∂x 1 2 )*(∂ 2 f/∂x 2 2 ) – (∂ 2 f/(∂x 1 ∂x 2) ) 2 > 0 Then Maximum

Nonlinear Optimization Partial Derivative Tests Rule 4, continued: IIf (∂ 2 f/∂x 1 2 )*(∂ 2 f/∂x 2 2 ) – (∂ 2 f/(∂x 1 ∂x 2) ) 2 < 0 Then Saddle Point IIf (∂ 2 f/∂x 1 2 )*(∂ 2 f/∂x 2 2 ) – (∂ 2 f/(∂x 1 ∂x 2) ) 2 = 0 Then no conclusion

Nonlinear Optimization Partial Derivative Tests Rule 5 (Necessary Condition): AAll n partial derivatives of an unconstrained function of n variables, f(x 1, x 2, …, x n ), must equal zero at any local maximum or any local minimum point.

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: LLagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints

Lagrange Multipliers Nonlinear Optimization with an equality constraint MMax or Min f(x 1, x 2 ) SST: g(x 1, x 2 ) = b Form the Lagrangian Function: LL = f(x 1, x 2 ) + λ[g(x 1, x 2 ) – b]

Nonlinear Optimization Lagrange Multipliers Rule 6 (Necessary Condition): OOptimization of an equality constrained function, 1 st order conditions: ∂L/∂x 1 = 0 ∂L/∂x 2 = 0 ∂L/∂λ = 0

Nonlinear Optimization Lagrange Multipliers Rule 7 (Sufficient Condition): IIf rule 6 is satisfied at a point (x * 1, x * 2, λ * ) apply conditions (a) and (b) of rule 4 to the Lagrangian function with λ fixed at a value of λ * to determine if the point (x * 1, x * 2 ) is a local maximum or a local minimum.

Nonlinear Optimization Lagrange Multipliers Rule 8 (Necessary Condition): FFor the function of n variables, f(x 1, x 2, …, x n ), subject to m constraints to have a local maximum or a local minimum at a point, the partial derivatives of the Langrangian function with respect to x 1, x 2, …, x n and λ 1, λ 2, …, λ m must all equal zero at that point.

Nonlinear Optimization Interpretation of Lagrange Multipliers The value of the Lagrange multiplier associated with the general model above is the negative of the rate of change of the objective function with respect to a change in b. More formally, it is negative of the partial derivative of f(x 1, x 2 ) with respect to b; that is, λλ = - ∂f/∂b or ∂∂f/∂b = - λ

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality Constraints: LLagrange Multipliers Interpretation of Lagrange Multiplier Models Involving Inequality Constraints

Models Involving Inequality Constraints Step 1: AAssume the constraint is not binding, and apply the procedures of “Unconstrained Models with More Than One Decision Variable” to find the global maximum of the function, if it exists. (Functions that go to infinity do not have a global maximum). If this global maximum satisfies the constraint, stop. This is the global maximum for the inequality-constrained problem. If not, the constraint may be binding at the optimum. Record the value of any local maximum that satisfies the inequality constraint, and go on to Step 2. Step 2: AAssume the constraint is binding, and apply the procedures of “Models with Equality Constraints” to find all the local maxima of the resulting equality-constrained problem. Compare these values with any feasible local maxima found in Step 1. The largest of these is the global maximum.