1. EML4552 - Engineering Design Systems II (Senior Design Project)
Optimization Theory and
Optimum Design
Unconstrained Optimization
(Lagrange Multipliers)
Hyman: Chapter 10
EML Spring’08

2. Unconstrained Optimization
In 1-D the optimum is determined by: x
y=f(x)
df/dx=0
EML Spring’08

3. Unconstrained Optimization
The condition for a local optimum can be extended to multi-dimensions x2 x1
EML Spring’08

4. Unconstrained Optimization
Condition for local optimum in unconstrained problem
However, most optimization problems are constrained
EML Spring’08

5. Optimization
Minimize (Maximize) an Objective Function of certain Variables subject to Constraints
EML Spring’08

6. Lagrange Multipliers
An analytical approach for solving constrained optimization problems
Particularly suited for problems in which the objective function and the constraints can be expressed analytical (even if highly non-linear)
Could be numerically implemented for more general cases
Will present the method through a simple example, it can be generalized for more complex problems
EML Spring’08

7. Lagrange Multiplers: Example
Determine the dimensions of a rectangular storage container to minimize fabrication costs, the container will hold a volume V, and be made of steel in the bottom (at a cost of S $/unit surface), and wood on the side (at a cost of W $/unit surface)
EML Spring’08

8. Lagrange Multipliers: Example
In ‘principle’, we could ‘solve’ for z in terms of x and y. Substitute back into the equation for cost to obtain C(x,y) and then apply the condition dC/dx=dC/dy=0
This method, although correct in principle, could be very complex if we had many variables and constraints, or when the equations involved are difficult to solve (or involve numerical models)
A more general method is needed to approach constrained optimization problems.
EML Spring’08

9. Lagrange Multipliers: Example
Rewrite the constraint:
Define the Lagrangian as:
Notice that we have added “zero” to the objective function
EML Spring’08

10. Lagrange Multipliers: Example
Have turned a 3-D constrained problem into a 4-D unconstrained problem
EML Spring’08

11. Lagrange Multipliers: Example
The solution to the set of 4 equations in 4 unknowns is the optimum we seek. We need to solve the system, in this case:
EML Spring’08

12. Lagrange Multipliers: Example
Substituting:
EML Spring’08

13. Lagrange Multipliers: Example
Solutions:
x=y means the optimum occurs when the bottom of the container is ‘square’
(the second solution can be shown to be the same condition x=y)
EML Spring’08

14. Lagrange Multipliers: Example
Substituting:
EML Spring’08

15. Lagrange Multipliers: General Case
EML Spring’08

16. Lagrange Multipliers: General Case
EML Spring’08

17. Lagrange Multipliers: General Case
EML Spring’08

18. Lagrange Multipliers: General Case
EML Spring’08

19. Other Optimization Methods
Step 1: Convert a constrained optimization problem into an unconstrained problem by use of ‘penalty’ functions
EML Spring’08

20. Other Optimization Methods
Step 2: Use a ‘search’ method to obtain the optimum (numerical probing of the objective function)
Random search
Directed search
Hybrid search
Combination of methods (‘decomposition’, sequential application, etc.)
EML Spring’08

21. Search Methods
The challenge is to create an ‘efficient’ search method that at the same time ensures we find the ‘global’ optimum and not just a local optimum
Random search
Steepest descent
“Simplex” (polyhedron) search
Genetic algorithm
Simulated annealing
EML Spring’08