What are optimization methods? Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Methods for Finding Engineering Solutions Evolution: There is a close parallel between technological evolution and biological evolution. Intuition: The art of engineering is the ability to make good decisions without having exact mathematical justification. Trial-and-error modeling: This refers to the usual situation in engineering design where it is recognized that the first feasible design is not necessarily the best. Numerical algorithms (Optimization): This approach has been enabled by the ready availability of fast, powerful digital computation. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Optimization Optimization has always been a goal of engineering design, but designers have not had the computational capability to perform true optimization in the mathematical sense until the last 20 years. By the term optimal design we mean the best of all feasible designs. Optimization is the process of maximizing a desired quantity or minimizing an undesired one. Optimization theory is the body of mathematics that deals with the properties of maxima and minima and how to find them. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Objective Function An objective function that defines the overall value of the design in terms of the n design variables, expressed as a vector x, is established: 𝑓 𝑥 =𝑓( 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 ) Typical objective functions can be expressed in terms of cost, weight, reliability and performance index or a combination of these. By convention, objective functions are usually written to minimize their value. Maximizing a function f(x) is the same as minimizing –f(x). Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Constraints Equality Constraints specify relations that must exist between the variables: ℎ 𝑗 𝑥 = ℎ 𝑗 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 =0;𝑗=1 𝑡𝑜 𝑝 Inequality Constraints are imposed by specific details of the problem: 𝑔 𝑖 𝑥 = 𝑔 𝑖 𝑥 1 , 𝑥 2 ,…, 𝑥 𝑛 ≤0;𝑖=1 𝑡𝑜 𝑚 There is no restriction on the number of inequality constraints. A type of inequality constraint that arises naturally in design situations is based on specifications. Specification define points of interaction with other parts of the system. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Listing of Numerical Methods Used in Optimization Problems A List of references is found on p. 715 of the text. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Optimization by Differential Calculus A characteristic property of an extremum is that the derivative of the function is 0 at that point. f(x) is momentarily stationary at that point. The familiar condition for a stationary point is: 𝑑𝑓 𝑥 𝑑𝑥 =0 If the curvature is negative, then the stationary point is a maximum. The point is a minimum if curvature is positive. 𝑑 2 𝑓 𝑥 𝑑 𝑥 2 ≤0 𝑑 2 𝑓 𝑥 𝑑 𝑥 2 ≥0 Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Different Types of Extrema in the Objective Function Curve Global Maximum Local Maximum Inflection Point Cusp Local Minimum Global Minimum Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

EXAMPLE 1 Consider the design of a cylindrical storage tank that is required to store a fixed volume of liquid V. The tank will be constructed by forming and welding thin steel plate. The design variables are the tank diameter D and height h. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Lagrange Multiplier Method The Lagrange multipliers provide a powerful method for finding optima in multivariable problems involving equality constraints. We have the objective function 𝑓(𝑥)=𝑓( 𝑥, 𝑦, 𝑧) subject to the equality constraints ℎ 1 = ℎ 1 𝑥,𝑦,𝑧 and ℎ 2 = ℎ 2 𝑥,𝑦,𝑧 . We establish a new function, the Lagrange expression (LE): 𝐿𝐸= 𝑓 𝑥 2 𝑥,𝑦,𝑧 + 𝜆 1 ℎ 1 𝑥,𝑦,𝑧 + 𝜆 2 ℎ 2 (𝑥,𝑦,𝑧) The following conditions must be satisfied at the optimum point: 𝜕𝐿𝐸 𝜕𝑥 =0, 𝜕𝐿𝐸 𝜕𝑦 =0, 𝜕𝐿𝐸 𝜕𝑧 =0, 𝜕𝐿𝐸 𝜕 𝜆 1 =0, 𝜕𝐿𝐸 𝜕 𝜆 2 =0 Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Example A total of 300 linear feet of tubes must be installed in a heat exchanger in order to provide the necessary heat transfer surface area. The total dollar cost of the installation includes: (1) the cost of the tubes, $700; (2) the cost of the shell 25D2.5L; (3) the cost of the floor space occupied by the heat exchanger = 20DL. The spacing of the tubes is such that 20 tubes must fit in a cross-sectional area of 1 ft2 inside the heat exchanger tube shell. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Search Methods When it becomes clear that there are many feasible solutions to a design problem, it is necessary to use some method of searching through the design space to find the best one. A deterministic search is one in which there is little variability of results and all problem parameters are known. In stochastic search, there is a degree of randomness in the search process that can lead to different solutions. We can have a simultaneous search, in which the conditions for every experiment are specified and all the observations are completed before any judgment regarding the location of the optima is made. Sequential search is the type of search in which future experiments are based on past outcomes. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Golden Section Search The golden section search is an efficient search method for a single variable with the advantage that it does not require an advance decision on the number of trials. The search method is based on the fact that the ratio of two successive Fibonacci numbers is 𝐹 𝑛−1 𝐹 𝑛 =0.618 for all values of n>8. A Fibonacci series, named after a 13th century mathematician, is given by 𝐹 𝑛 = 𝐹 𝑛−2 + 𝐹 𝑛−1 where 𝐹 0 =1 and 𝐹 1 =1. This same ratio was discovered by Euclid, who called it the golden mean. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Example of use of the Golden Section Search Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Multivariable Search Methods When the objective function depends on two or more variables, the geometric representation is a response surface. It usually is convenient to work with contour lines produced by the intersection of planes of constant y with the response surface and projected on the 𝑥 1 𝑥 2 plane. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Lattice Search In the lattice search, which is an analog to the single- variable search, a two dimensional grid lattice is superimposed over the projections of the contours. In the absence of the special knowledge about the location of the maximum, the starting point is selected near the center of the region, at point 1. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Univariate Search The univariate search is a one-variable-at-a-time method. All of the variable are kept constant except one, and it is varied to obtain an optimum in the objective function. That optimal value is then substituted into the function, and the function is optimized with respect to another variable. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Gradient Method A common local search method is to follow the steepest ascent (hill climbing) up the response surface. The gradient method starts with a best guess location and determines the direction with the gradient vector which by definition is normal to the local contour line. The gradient vector is expressed in terms of partial derivative of the function describing the surface and the unit vectors, i, j, and k. 𝛻𝑓 𝑥,𝑦,𝑧 = 𝜕𝑓 𝜕𝑥 𝑖+ 𝜕𝑓 𝜕𝑦 𝑗+ 𝜕𝑓 𝜕𝑧 𝑘 Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Nonlinear Optimization Methods The methods discussed previously are not practical optimization techniques for most engineering design problems where there can be a large number of design variables and constraints. Numerical methods are needed to find solutions. The solution process starts with the best estimate of the optimum design. The objective function and the constraint functions, as well as their derivatives, are evaluated at that point. Then the design is moved to a new point, and to another, and so on, until optimality conditions are met. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Multivariable Optimization Multivariable optimization of nonlinear problems has been a field of great activity, and many computer-based methods are available. Methods for unconstrained multivariable optimization are discussed first. Newton's method is an indirect technique that employs a second-order approximation of the function. The DFP (Davidon, Fletcher, and Powell) method is one of the most powerful methods. Optimization of nonlinear problems with constraints is a more difficult area. Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies

Optimization Functions Provided by MATLAB Dieter/Schmidt, Engineering Design 5e. ©2013. The McGraw-Hill Companies