5.2.1 Optimization, Search and Exploration 1 Computer-Aided Engineering Knowledge Component 5: Information Processing 5.2.1 Optimization, Search and Exploration 1 2nd Edition

Module Information Intended audience Key words Author Novice Key words Optimization, Optimality, Boundedness, Feasibility, Constraint activity Author Ian Smith, EPFL, Switzerland Reviewers (1st Edition) Esther Obonyo, U of Florida, USA Ni-Bin Chang, U of Central Florida, USA Frederic Bosche, U of Waterloo, Canada Rafal Kicinger, George Mason U, USA 2

What there is to learn At the end of these slides, see the quiz for questions and answers. This sub-module introduces the following ideas: In engineering, optimization is really an exercise in searching for good solutions within feasible solution spaces. Simple element-dimension design is a basic form of optimisation. In addition to optimization criteria (minimum cost, maximum benefit, etc.) constraints (such as maximum stress, maximum weight and minimum thickness) contain much engineering knowledge 3

Optimization Optimization is most appropriate for well defined tasks (see Knowledge Component 1, Module 1.3) Optimization is used when mathematical models exist for evaluating the goodness of solutions. In engineering, the use of the word optimization can be dangerous. Some consultants have even banned the word. The next slide explains why.

Optimization and Optimality An optimal solution is optimal only with respect to: parameters contained in the mathematical model subjective decisions of the engineer who created the model In open worlds such as most engineering contexts, optimization cannot be guaranteed.

Optimization or Search? In many situations, the word “optimize” should be avoided altogether. This is specifically true for ill- defined tasks (see Knowledge Component 1, Module 1.3). It is usually more appropriate to speak of “search” since the engineer identifies good solutions from many possibilities. Engineers are valued primarily for finding solutions to ill-structured tasks. Therefore, optimization methods are most useful when they are used to support search. Search is the key activity for exploration of good solutions.

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

Bridge Design: Formulation 1 Model 1: Includes formulas that lead to values for the volume of concrete and weight of steel Optimization task: Minimize use of concrete and steel Difficulty: This task may provide solutions that have nothing to do with minimizing costs

Bridge Design: Formulation 2 Model 2: Includes Model 1 and additional formulas to calculate costs of materials and construction Optimization task: Minimize initial cost Difficulty: This task may not provide solutions that have the minimum life-cycle cost

Bridge Design: Formulation 3 Model 3: Includes Model 2 and additional formulas for material deterioration, periodic maintenance and dismantling to calculate the life-cycle cost Optimization task: Minimize life-cycle cost Difficulty: This task may not provide solutions that provide a good compromise between factors such as low life-cycle cost, good aesthetics, low environmental impact and wide-spread societal acceptance

Discussion Models 1-3 become progressively difficult to express in mathematical form. The closed world assumption is less and less accurate. Tasks become more and more ill-structured. Therefore, complex tasks cannot be optimized reliably.

Discussion (cont’d.) In spite of the difficulties of real engineering tasks, optimization strategies can help with decision-making if appropriate caution is exercised. For example, engineers should determine to what degree variations in factors that are not modeled influence the values of modeled variables. If such interdependencies are small, then optimization may be useful. This Knowledge Component includes an introduction to optimization (Module 5.2.1), methodologies (Module 5.2.2 and Module 5.2.3) and exploration strategies (Module 5.2.3).

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

Optimization Given a set D (domain) and a function f(x) defined over D, Minimize or maximize f(x), x  D with respect to a set of constraints, C The function f(x) is called the objective function. The variable x represents an n-dimensional vector of unknown values.

Types of Optimization Problems Problem characteristic Sub-type Type of variables Continuous Discrete Mixed Size of domain Finite Infinite Shape of objective function and constraints Linear Non-linear Discontinuous Number of objectives Single Multiple

Optimization (cont’d.) Since optimization has its roots in mathematics, the word “problem” is employed. In reality, engineers do not have “problems”, they have challenges that are met by carrying out appropriate tasks. Optimization problems are combinations of one or more sub-types of each problem characteristic.

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

C = πdtL * cost/unit weight Example 2: Shaft Design Goal: Design a hollow shaft in tension Optimization task: Minimize cost C C = πdtL * cost/unit weight This function is often called the “objective function” (Its form is rarely this simple!) L d t<<d Q  = density

Example 2 (cont’d.) Constraints: Strength: stress below yield stress σy is either 235 MPa or 355 Mpa Two values are from two possible steel grades Maximum weight 100kg (so two people can carry it) Fabrication: minimum dimensions t ≥ 0.005m and d ≥ 0.10m

Example 2 (cont’d.) Accompanying formulas Stress due to axial load σ = Q/πdt (when d»t, say d>10t) Weight = πdtL This example shows the 3 concepts of feasibility, boundedness and constraint activity. These are described in the following slides. The condition d»t is a scope constraint defining the applicability of the formulas for cost, weight and stress.

Example 2 (cont’d.) Resolution Load, length and density are known Decision variables First step - find the best value for d●t Second step – choose practical values for d and t Feasible solution space : those values of the objective function that satisfy all constraints Optimal solution : those values in the feasible solution space that best satisfy the optimization task.

Example 2 (First step) (L=2m, Q = 2000 kN) σy = 235 (no feasible solution using this steel type!) σy = 355 Cost, C Objective function Optimal solution Feasible solution space 1793 2709 d•t 2024 Weight = 100kg

Example 2 (Second step) d Optimal solutions (on bottom line) 100mm Feasible solution space (between lines) 5mm t

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

Feasibility Presence of several criteria may mean that that there is no feasible domain. Although strength requirements in the example call for minimum area = Q/sy, maximum weight requirements (maximum area) may mean that there are no possible solutions for high values of Q and low values of σy. For example, given a weight restriction of 100kg, there is no solution for a shaft made of steel with a yield strength of 235 MPa.

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

Boundedness Constraints bound feasible domains Strength and fabrication requirements create lower bounds on shaft dimensions (d and t) while weight requirements place upper bounds on these values. Therefore, constraints lead to bounded feasible domains. For example, the constraint d>0.1m bounds the solution for shaft dimensions. At the other end, model validity (simplified formula for surface area) bounds the solution. This is called a soft constraint since it can be relaxed if a better model is used.

Example 1: Bridge Design Outline Example 1: Bridge Design Optimization Example 2: Shaft Design Feasibility Boundedness Constraint Activity

Constraint Activity If a constraint influences the location of the optimum, it is said to be an active constraint. In the previous example, the strength constraint   y was active. Therefore, the optimum area is determined by transforming into  = y and solving for the intersection of the constraint with the objective function. For example, the constraint σ  355 was transformed into σ = 355 in order to solve for the optimal values of dt for the shaft. Actual values chosen for d and t will depend on factors such as availability. Note: Often, optimization is not this easy.

Constraint activity (cont’d.) Often, there is no easily identifiable active constraint at the optimum. This occurs when the objective function is non-monotonic in the decision variables. In these cases, optimal values cannot be obtained by transforming inequalities into equalities and substituting them into the objective function. Other techniques are necessary in such cases. These are described in Modules 5.2.2 and 5.2.3.

Review Quiz Why is search preferable over traditional mathematical optimization in engineering contexts? Define feasibility and boundedness with respect to optimization. What is an active constraint?

Answers to Review Quiz Why is search preferable over traditional mathematical optimization in engineering contexts? Optimality depends on parameters contained in the mathematical model subjective decisions of the engineer Consequently, in open worlds, it is better to employ optimization to enhance search and exploration of good solutions.

Answers to Review Quiz Define feasibility and boundedness with respect to optimization. Feasibility refers to the possibility of finding a solution given the constraints of the task. Boundedness occurs when possible solutions are bounded by constraints within feasible domains

Answers to Review Quiz What is an active constraint? A constraint that influences the location of the optimum is an active constraint.

Summary Optimization is more suited to well-structured tasks while search and exploration of good solutions is best for engineering needs. It is hard to reliably optimize complex tasks due to difficulties associated with formulation of objective functions. Mathematically, optimization involves maximizing or minimizing a given function with respect to a given set of constraints. Feasibility, boundedness and constraint activity are aspects of optimization.

Further Reading J.S. Arora, O.A. Elwakeil, A.I. Chahande, C.C. Hsieh, “Global Optimization methods for engineering applications: a review,” Structural Optimization, 9, 137-159, 1995. B. Raphael and I.F.C. Smith, “A direct stochastic algorithm for global search,” J of Applied Mathematics and Computation, Vol 146, No 2-3, 2003, pp 729-758 Raphael, B. and Smith, I.F.C. Fundamentals of Computer-Aided Engineering, Wiley, 2003