Engineering Optimization Concepts and Applications Fred van Keulen Matthijs Langelaar CLA H21.1 A.vanKeulen@tudelft.nl

Optimization problem Design variables: variables with which the design problem is parameterized: Objective: quantity that is to be minimized (maximized) Usually denoted by: ( “cost function”) Constraint: condition that has to be satisfied Inequality constraint: Equality constraint:

Optimization problem (cont.) General form of optimization problem:

Classification Problems: Responses: Variables: Constrained vs. unconstrained Single level vs. multilevel Single objective vs. multi-objective Deterministic vs. stochastic Responses: Linear vs. nonlinear Convex vs. nonconvex Smooth vs. nonsmooth Variables: Continuous vs. discrete (integer, ordered, non-ordered)

Solving optimization problems Optimization problems are typically solved using an iterative algorithm: Responses Derivatives of responses (design sensi- tivities) Model Constants Design variables Optimizer

Optimization pitfalls! Proper problem formulation critical! Choosing the right algorithm for a given problem Many algorithms contain lots of control parameters Optimization tends to exploit weaknesses in models Optimization can result in very sensitive designs Some problems are simply too hard / large / expensive

Exercises Exercise 1: Introduction to the valve spring design problem Study analysis model Formulation of spring optimization model Exercise 2: Model behavior / optimization formulation Study model properties (monotonicity, convexity, nonlinearity) Optimization problem formulation

Course overview General introduction, problem formulation, design space / optimization terminology Modeling, model simplification Optimization of unconstrained / constrained problems Single-variable, zeroth-order and gradient-based optimization algorithms Design sensitivity analysis (FEM) Topology optimization

Defining a design model and optimization problem 1. What can be changed and how can the design be described? Dimensions Stacking sequence of laminates Ply orientation of laminates Thicknesses For structures: distinguish sizing, material and shape variables Bridgestone aircraft tire

Defining the optimization problem 2. What is “best”? Define an objective function: Weight Production cost Life-time cost Profits 3. What are the restrictions? Define the constraints: Stresses Buckling load Eigenfrequency

Defining the optimization problem (cont.) 4. Optimization: find a suitable algorithm to solve the optimization problem. Choice depends on problem characteristics: Number of design variables, constraints Computational cost of function evaluation Sensitivities available? Continuous / discrete design variables? Smooth responses? Numerical noise? Many local optima? (nonconvex) Nonsmooth response: for example maximum stresses in a structure (only C0). Other models can even be discontinuous.

Summary Defining an optimization problem: Choose design variables and their bounds Formulate objective (best?) Formulate constraints (restrictions?) Choose suitable optimization algorithm

Standard forms Several standard forms exist: Negative null form: Positive null form: Neg. unity form: Pos. unity form:

Structural optimization examples Typical objective function: weight Typical constraint: maximum stress, maximum displacement Note the scaling! Scaled vs. Unscaled

Example: minimum weight tubular column design Length l given Load P given Design variables: Radius R  [Rmin, Rmax] Wall thickness t  [tmin, tmax] Objective: minimum mass Constraints: buckling, stress R t P l

Tubular column design Design problem: Definition, substitution of elementary equations, normal form. Addition of condition associated to the modeling assumptions (thin-walled tube, A and I use this)

Tubular column design (2) Alternative formulation: Ro Ri P l

Multi-objective problems Minimize c(x) s.t. g(x)  0, h(x) = 0 Input from designer required! Popular approach: replace by weighted sum: Vector! Optimum, clearly, depends on choice of weights Pareto optimal point: “no other feasible point exists that has a smaller ci without having a larger cj”

Multi-objective problems (cont.) Examples of multi-objective problems: Design of a structure for Minimal weight and Minimal stresses Design of reduction gear unit for Minimal volume Maximal fatigue life Design of a truck for Minimal fuel consumption @ 80 km/h Minimal acceleration time for 0 – 40 km/h Minimal acceleration time for 40 – 90 km/h Minimal stress important for fatigue.

Pareto set Pareto point: “Cannot improve an objective without worsening another” c2 Pareto set Attainable set Pareto point c1

Pareto set (cont.) Alternative view: Pareto set c1 c2 x

Pareto set (cont.) Pareto set can be disjoint: Attainable set c2

Hierarchical systems Large system can be decomposed into subsystems / components: Optimization requires specialized techniques, multilevel optimization Biomass reactor for power production.

Structural hierarchical systems Example: wing box Too many design variables to treat at once Global level: global loads, global dimensions The global level simulation provides the loads for the local problems. However, changing the local properties will change the load distribution, which also affects the global level and other local problems. An iterative scheme is needed to solve this optimization problem. Local (rib / stiffner) level: plate thickness, fiber orientation

Contents Defining an optimization problem The design space & problem characteristics Model simplification

The design space Design space = set of all possible designs Example: kmax Feasible domain F k2 The cost of the springs is proportional to their stiffness, hence the objective function. k2 Optimum k1 k1 kmax

Isolines Isolines (level sets) connect points with equal function values:

The design space (cont.) No feasible domain Problem overconstrained: no solution exists. Dominated constraint (redundant) Just to introduce more terminology and concepts …

Design space (cont.) A and B inactive Interior optimum Objective function isolines A A and B active Objective function isolines Optimum B active, A inactive Optimum Objective function isolines B

Active constraint optimization Idea of constraint activity at boundary optimum sometimes used in intuitive design optimization: Fully stressed design (sizing / topology optimization) Simultaneous failure mode theory Careful: does not always give the optimal solution! F F

Problem characteristics Study of objective and constraint functions: simplify problem discover incorrect problem formulation choose suitable optimization algorithms Properties: Boundedness Linearity Convexity Monotonicity

Boundedness Proper bounds are necessary to avoid unrealistic solutions: Example: aspirin pill design Objective: minimize dissolving time = maximize surface area (fixed volume) r h

Boundedness (cont.) Volume equality constraint can be substituted, yielding: r f Restrictions on r needed. Note, surface minimization problem is well-defined!

Linearity “A function f is linear if it satisfies f(x1+ x2) = f(x1)+ f(x2) and f(a x1) = a f(x1) for every two points x1, x2 in the domain, and all a”

Linearity (2) Nonlinear objective functions can have multiple local optima: x1 x2 f x1 x2 f x Challenge: finding the global optimum.

Problem characteristics Study of objective and constraint functions: simplify problem discover incorrect problem formulation choose suitable optimization algorithms Properties: Boundedness Linearity Convexity Monotonicity

Boundedness Surface maximization of aspirin pill not well bounded: f r Restrictions on r needed. Note, surface minimization problem is well-defined!

Linearity Nonlinear objective functions can have multiple local optima: x1 x2 f x1 x2 f x Challenge: finding the global optimum.

Convexity Convex function: any line connecting any 2 points on the graph lies above it (or on it): Note difference between strictly convex and “normal” convex. Linearity implies convexity (but not strict convexity)

Convexity (cont.) Convex set [Papalambros 4.27]: “A set S is convex if for every two points x1, x2 in S, the connecting line also lies completely inside S”

Convexity (cont.) Nonlinear constraint functions can result in nonconvex feasible domains: x2 x1 Nonconvex feasible domains can have multiple local boundary optima, even with linear objective functions!

Monotonicity Papalambros p. 99: f2 f1 x1 x2 Similar: Function f is strictly monotonically increasing if: f(x2) > f(x1) for x2 > x1 weakly monotonically increasing if: f(x2)  f(x1) for x2 > x1 Similar for mon. decreasing f2 f1 x1 x2 Similar: Note: monotonicity  convexity! Linearity implies monotonicity

Optimization problem characteristics Responses: Boundedness Linearity Convexity Monotonicity Feasible domain: Convexity

Example: tubular column design f g2 t

Optimization problem analysis Motivation: Simplification Identify formulation errors early Identify under- / overconstrained problems Insight Necessary conditions for existence of optimal solution Basis: boundedness and constraint activity

Well-bounded functions – some definitions Lower bound: Greatest lower bound (glb): g f x N = set of nonnegative numbers, P = set of positive and finite numbers F(x*) = minimum, x* = minimizer Minimum: Minimizer: x*

Boundedness checking Assumption: in engineering optimization problems, design variables are positive and finite Define Boundedness check: Determine g+ for Determine minimizers Well bounded if N: non-negative numbers P: positive and finite numbers

Examples: Bounded at zero Asymptotically bounded

Air tank design Objective: minimize mass h l Not well bounded: constraints needed

Air tank constraints Minimum volume: Min. head/radius ratio (ASME code): Min. thickness/radius ratio (ASME code): Room for nozzles: min. length Space limitations: max. outside radius

Partial minimization & bounding constraints Partial minimization: keep all variables constant but one. Example: air tank wall thickness t: Conclusion: f not well bounded from below g3 bounds t from below

Constraint activity Removing constraint = relaxing problem Solution set of relaxed problem without gi is Xi 1. 2. 3. A B A and B active Activity information can simplify problem: Active: eliminate variable Inactive: remove constraint

Constraint activity checking Example: x2 f(1,x2,5) g3 g2 Partial minimization w.r.t. x1 shows that g1 bounds x1. Idem x3 shows that g4 is inactive Idem x2 shows that g = 1 (at x1=1, x3=5), X = 1, 3, 4, feasible 3, 4 Removing g3 is no problem. Removing g2 adds 1 to the solution. This means that g2 is semiactive. Note, setting g2 active and using it to eliminate x2 (I.e. x2 = 2) yields the wrong solution! Conclusion: g1 active g2 semiactive g3 and g4 inactive

Activity and Monotonicity Theorem “Constraint gi is active if and only if the minimum of the relaxed problem is lower than that of the original problem” f(x) f g(x) x g2 g1 x f g f(x) g(x) “If f(x) and gi(x) all increase or decrease (weakly) w.r.t. x, the domain is not well constrained”

First Monotonicity Principle “In a well-constrained minimization problem every variable that increases f is bounded below by at least one non-increasing active constraint” x f g This principle can be used to find active constraints. Exactly one bounding constraint: critical constraint f(x) g(x)

Air tank design Monotonicity analysis: What about l? Unclear. Critical w.r.t. r Critical w.r.t. h Critical w.r.t. t Both g1 and g4 could bound l from below. Further analysis is required. What about l? Unclear.

Optimizing variables out Critical constraints must be active:

Optimizing variables out Critical constraints must be active:

Optimizing variables out Critical constraints must be active:

Optimizing variables out Critical constraints must be active:

Optimizing variables out Critical constraints must be active:

Problem! Length not well bounded: Additional constraint from above is needed: Maximum plate width:

Air tank solution Length constraint is critical: must be active! Result of Monotonicity Analysis: Problem found, and fixed Solution found without numerical optimization

Recognizing monotonicity Some useful properties: Sums: Sums of similarly monotonic functions have the same monotonicity Products: Products of similarly monotonic functions have: same monotonicity if opposite monotonicity if

Recognizing monotonicity More properties: Powers: Positive powers of monotonic functions have the same monotonicity, negative powers have opposite monotonicity Composites:

Recognizing monotonicity f1 Integrals: w.r.t. limits: a b x w.r.t. integrand: f1 x y a b

Criticality Refined definitions: # of variables critically bounded by constraint i # of constraints possibly critically bounding variable j Uncritical constraint 1 Uniquely critical constraint >1 Multiple critical constraint >1 Conditionally critical constraint

Air tank example Multiple critical! Critical w.r.t. r Critical w.r.t. h Critical w.r.t. t Conditionally critical w.r.t. l Emphasize that the contidionally critical constraints for l SEEM to indicate that l is properly bounded. However, as it turns out, it is not! Multiple critical constraint can obscure boundedness! Eliminate if possible

Air tank example Starting with eliminating r:

? Air tank example New problem: Critical for h Critical for t Conclusion: eliminate multiply critical constraints if possible, otherwise no conclusion w.r.t. proper boundedness can be made.

Air tank example Finally, after also eliminating h and t: Not well bounded! To be honest, I used Maple to do the work. Conclusion: multiple critical constraint obscured ill-boundedness in l

Summary Optimization problem checking: Boundedness check of objective Identify underconstrained problems Monotonicity analysis Identify not properly bounded problems Identify critical constraints Eliminate variables Remove inactive constraints

But what about … Equality constraints: Example: Relaxed problem: Active if all constraint variables in objective Otherwise semi-active Example: x1 x2 f 3 1 Relaxed problem:

More on nonobjective variables Monotonicity Principle for nonobjective variables: “In a well-constrained minimization problem every nonobjective variable is bounded below by at least one non-increasing semiactive constraint and above by at least one non-decreasing semiactive constraint” g(x) gi gj x

Nonobjective variables (2) Other options: Equality constraint Single nonmonotonic constraint h(x) g(x) hi gi x x See example in book (Papalambros p. 114)

Nonmonotonic functions Monotonicity analysis difficult! Sometimes regional monotonicity can be used Concave constraints can split feasible domain: g(x) This gave a flavour of Monotonicity Analysis and what it can be used for. For special cases and more complicated situations, see book. gj gi x

Model preparation procedure (3.9) Remove dominated constraints Check boundedness for each design variable: Objective monotonic? Constraints monotonic? Critical constraints? Uniquely / conditionally / multiply? If possible, eliminate active constraints, and repeat steps Spending time on model checking usually pays off!