Optimization in Engineering Design 1 Lagrange Multipliers

Optimization in Engineering Design 2 Lagrange Multipliers The method of Lagrange multipliers gives a set of necessary conditions to identify optimal points of equality constrained optimization problems. This is done by converting a constrained problem to an equivalent unconstrained problem with the help of certain unspecified parameters known as Lagrange multipliers. The classical problem formulation minimize f(x 1, x 2,..., x n ) Subject to h 1 (x 1, x 2,..., x n ) = 0 can be converted to minimize L(x, ) = f(x) - h 1 (x) where L(x, v) is the Lagrangian function is an unspecified positive or negative constant called the Lagrangian Multiplier

Optimization in Engineering Design 3 Finding an Optimum using Lagrange Multipliers New problem is: minimize L(x, ) = f(x) - h 1 (x) Suppose that we fix = * and the unconstrained minimum of L(x; ) occurs at x = x * and x * satisfies h 1 (x * ) = 0, then x * minimizes f(x) subject to h 1 (x) = 0. Trick is to find appropriate value for Lagrangian multiplier. This can be done by treating as a variable, finding the unconstrained minimum of L(x, ) and adjusting so that h 1 (x) = 0 is satisfied.

Optimization in Engineering Design 4 Method 1.Original problem is rewritten as: minimize L(x, ) = f(x) - h 1 (x) 2.Take derivatives of L(x, ) with respect to x i and set them equal to zero. If there are n variables (i.e., x 1,..., x n ) then you will get n equations with n + 1 unknowns (i.e., n variables x i and one Lagrangian multiplier ) 3.Express all x i in terms of Langrangian multiplier 4.Plug x in terms of in constraint h 1 (x) = 0 and solve. 5.Calculate x by using the just found value for. Note that the n derivatives and one constraint equation result in n+1 equations for n+1 variables! (See example 5.3)

Optimization in Engineering Design 5 Multiple constraints The Lagrangian multiplier method can be used for any number of equality constraints. Suppose we have a classical problem formulation with k equality constraints minimize f(x 1, x 2,..., x n ) Subject to h 1 (x 1, x 2,..., x n ) = h k (x 1, x 2,..., x n ) = 0 This can be converted in minimize L(x, ) = f(x) - T h(x) where T is the transpose vector of Lagrangian multpliers and has length k

Optimization in Engineering Design 6 In closing Lagrangian multipliers are very useful in sensitivity analyses (see Section 5.3) Setting the derivatives of L to zero may result in finding a saddle point. Additional checks are always useful. Lagrangian multipliers require equalities. So a conversion of inequalities is necessary. Kuhn and Tucker extended the Lagrangian theory to include the general classical single-objective nonlinear programming problem: minimize f(x) Subject to g j (x)  0for j = 1, 2,..., J h k (x) = 0for k = 1, 2,..., K x = (x 1, x 2,..., x N )