EML4552 - Engineering Design Systems II (Senior Design Project) Optimization Theory and Optimum Design Unconstrained Optimization (Lagrange Multipliers) Hyman: Chapter 10 EML 4550 - Spring’08

Unconstrained Optimization In 1-D the optimum is determined by: x y=f(x) df/dx=0 EML 4550 - Spring’08

Unconstrained Optimization The condition for a local optimum can be extended to multi-dimensions x2 x1 EML 4550 - Spring’08

Unconstrained Optimization Condition for local optimum in unconstrained problem However, most optimization problems are constrained EML 4550 - Spring’08

Optimization Minimize (Maximize) an Objective Function of certain Variables subject to Constraints EML 4550 - Spring’08

Lagrange Multipliers An analytical approach for solving constrained optimization problems Particularly suited for problems in which the objective function and the constraints can be expressed analytical (even if highly non-linear) Could be numerically implemented for more general cases Will present the method through a simple example, it can be generalized for more complex problems EML 4550 - Spring’08

Lagrange Multiplers: Example Determine the dimensions of a rectangular storage container to minimize fabrication costs, the container will hold a volume V, and be made of steel in the bottom (at a cost of S $/unit surface), and wood on the side (at a cost of W $/unit surface) EML 4550 - Spring’08

Lagrange Multipliers: Example In ‘principle’, we could ‘solve’ for z in terms of x and y. Substitute back into the equation for cost to obtain C(x,y) and then apply the condition dC/dx=dC/dy=0 This method, although correct in principle, could be very complex if we had many variables and constraints, or when the equations involved are difficult to solve (or involve numerical models) A more general method is needed to approach constrained optimization problems. EML 4550 - Spring’08

Lagrange Multipliers: Example Rewrite the constraint: Define the Lagrangian as: Notice that we have added “zero” to the objective function EML 4550 - Spring’08

Lagrange Multipliers: Example Have turned a 3-D constrained problem into a 4-D unconstrained problem EML 4550 - Spring’08

Lagrange Multipliers: Example The solution to the set of 4 equations in 4 unknowns is the optimum we seek. We need to solve the system, in this case: EML 4550 - Spring’08

Lagrange Multipliers: Example Substituting: EML 4550 - Spring’08

Lagrange Multipliers: Example Solutions: x=y means the optimum occurs when the bottom of the container is ‘square’ (the second solution can be shown to be the same condition x=y) EML 4550 - Spring’08

Lagrange Multipliers: Example Substituting: EML 4550 - Spring’08

Lagrange Multipliers: General Case EML 4550 - Spring’08

Lagrange Multipliers: General Case EML 4550 - Spring’08

Lagrange Multipliers: General Case EML 4550 - Spring’08

Lagrange Multipliers: General Case EML 4550 - Spring’08

Other Optimization Methods Step 1: Convert a constrained optimization problem into an unconstrained problem by use of ‘penalty’ functions EML 4550 - Spring’08

Other Optimization Methods Step 2: Use a ‘search’ method to obtain the optimum (numerical probing of the objective function) Random search Directed search Hybrid search Combination of methods (‘decomposition’, sequential application, etc.) EML 4550 - Spring’08

Search Methods The challenge is to create an ‘efficient’ search method that at the same time ensures we find the ‘global’ optimum and not just a local optimum Random search Steepest descent “Simplex” (polyhedron) search Genetic algorithm Simulated annealing EML 4550 - Spring’08